Realtime AI/ML game technologies *spawn

milk

Like Verified
Veteran
The "nVidia ACE" demo was built with UE5 and RTXDI:



Thats what I expect from "proper" next gen games. And I expect it to still stake some years for an actual shipping title to get all the pieces of the puzzle right, but it will eventually happen.
 
2023: 'Don't play games at launch. They're not finished yet.'
2026: 'Don't play games at launch. They are so crowded, publishers servers running the language models can't keep up, so NPCs take one minute to respond.'

Or can this run on client GPU as well? How large is the data such language models need to work?
Saw some Skyrim + ChatGPT + speech synthesis stuff before, but i guess most of that was streamed from servers.
 
BPD-player: Why are your goons trying to kill me?
EvilVillainBoss: Because you're pathetic and deserve to die
BPD-player goes and makes interesting discoveries on what counts as a noose or cellar window in certain part of the world
 
Last edited:
The "nVidia ACE" demo was built with UE5 and RTXDI:



Oh gods, the voice and response is hilariously stilted and bad. Not too mention cloud compute costs so much no gamedev in their right minds would pay for this. Consumers are never in their right mind and so would buy this only for the money to run out two years later and the entire game, even the single player portion, will shut down and their purchase will be of nothing.

UE5 looks great though. Maybe when AI models start getting efficient enough to run client side we'll see cool generative AI stuff in games. Hopefully better than this, but I'm most hopeful for animation and NPC control AI far before we ever see runtime generative, and of course we already have ML compression and NPC/Agent control. We'll probably start seeing offline generated lines pretty soon.

nVidia promotes a new asia UE5 game:



How much does Nvidia pay these companies to advertise features they have nothing to do with? Now featuring rendering tech we didn't even touch, please continue buying our hyper over priced GPUs because you're brainless sheep! Not that we care, AI hype stock valuation baybeee $:mrgreen:$!

If Jensen wanted to leave on a high note he'd quit now, before drooling stock hype people realize the AI hardware/software market has become rabidly competitive, so there's no possible way Nvidia's current market share and margins can continue for any meaningful amount of time. I'd have to dig up what company it is, but their quotes for AI power efficiency benchmarks in their winning bid for a Euro based AI super computer outclass anything even on Nvidia's roadmap.
 
Oh gods, the voice and response is hilariously stilted and bad. Not too mention cloud compute costs so much no gamedev in their right minds would pay for this. Consumers are never in their right mind and so would buy this only for the money to run out two years later and the entire game, even the single player portion, will shut down and their purchase will be of nothing.
Soon, nobody will remember your concerns, i'm afraid.
Currently the acting of AI is a bit lacking in emotion and expression, but they'll improve it quickly.
And if the future is about shared multiplayer worlds, no longer isolated single player experiences at all, the server model will surely pay off.

How much does Nvidia pay these companies to advertise features they have nothing to do with?
A joint venture in marketing. One hand helps the other.
Some people here said NV literally IS pc gaming. I'll leap ahead and claim NV has invented ChatGPT.
Praise the tech gods.
I became a believer after Jensen showed me all those good things often enough: AI accelerating medical treatment and progress is clearly a very good thing, no? Or AI cars smart enough to deal with kids crossing the street, etc.
If Jensen wanted to leave on a high note he'd quit now
Nope. AI will extend his lifetime. He'll still rule NV in 30 years, as planned. And the greatest achievements are still in front of us.

\ :D /
 
If single player games die in favor of forced multiplayer because rich arseholes like all the buzzwords, then we have truly died and gone to hell
I feel with you. And that's one of the many points of why i think gaming must and and will fragment into multiple subgroups. It's big enough, and there is no longer just one 'mainstream' to target.
For example, imagine this is what the future will give us:

Server based multiplayer 'ready player one' mainstream.
With AI NPCs as shown. And the true reason for those AI NPCs is to cultivate multiplayer. They do not only tell a story, but also act to regulate human player interactions, so they do not just silly things and yell at each other.
That's really a big opportunity, besides enabling true procedural stories.

Old school single player. For us old fashioned dudes.
Will it be worth to make games for us?
Yes! Because, as promised, AI and other new tech will bring costs down, so 10 motivated guys can make games at AAA standards.
And due to smaller risk, they can also experiment with some new ideas as well. Targeting niches becomes much better business than it is right now.

Maybe i'm too optimistic some times. But it should work.
 
Oh gods, the voice and response is hilariously stilted and bad. Not too mention cloud compute costs so much no gamedev in their right minds would pay for this. Consumers are never in their right mind and so would buy this only for the money to run out two years later and the entire game, even the single player portion, will shut down and their purchase will be of nothing.

UE5 looks great though. Maybe when AI models start getting efficient enough to run client side we'll see cool generative AI stuff in games. Hopefully better than this, but I'm most hopeful for animation and NPC control AI far before we ever see runtime generative, and of course we already have ML compression and NPC/Agent control. We'll probably start seeing offline generated lines pretty soon.

I've not followed this closely but has it been established that those AI responses were generated in the cloud as opposed to locally? I get we're a good way away from this being in a full shipping game, but as a glimpse of the future I would assume the target is for this to be locally calculated. Perhaps the next gen consoles will baseline that level of performance sufficiently for it to ship in games.

In terms of how natural the response is. Sure I agree you're not going to mistake it for a real human. But in terms of the state of the art today, especially in light of it recognizing the players natural speech, it's light years ahead and would be extremely compelling in any RPG.

Note speech recognition has been done at least once before in Star Trek Bridge Crew VR but they turned it off after a short time :( I found it missed quite a few of my commands, so wasn't great, but when it did work, it was pretty awesome. Although that could only detect pre-defined phrases and not natural speech like this.
 
Note speech recognition has been done at least once before in Star Trek Bridge Crew VR but they turned it off after a short time :( I found it missed quite a few of my commands, so wasn't great, but when it did work, it was pretty awesome.
Personally i do not want to talk to a machine. Not sure about the reasons. But it it's a real discomfort to me. A bit like admitting my humanity no longer grants me an intellectual advantage as well.
So i don't talk to phones, computers or virtual butlers, and i won't talk to NPCs in a game either.

But i have no problem with typing text to communicate with the NPC. I'll do that.
Not sure about the reasons either, but i guess more people have such talking barrier.
Though, likely just a generational habit. Probably i'm fine with typing because having played some adventure games as a kid.
 
I've not followed this closely but has it been established that those AI responses were generated in the cloud as opposed to locally? I get we're a good way away from this being in a full shipping game, but as a glimpse of the future I would assume the target is for this to be locally calculated. Perhaps the next gen consoles will baseline that level of performance sufficiently for it to ship in games.

In terms of how natural the response is. Sure I agree you're not going to mistake it for a real human. But in terms of the state of the art today, especially in light of it recognizing the players natural speech, it's light years ahead and would be extremely compelling in any RPG.

Note speech recognition has been done at least once before in Star Trek Bridge Crew VR but they turned it off after a short time :( I found it missed quite a few of my commands, so wasn't great, but when it did work, it was pretty awesome. Although that could only detect pre-defined phrases and not natural speech like this.
I think AI generated speech responses is left up to the developer but can be either cloud or local. The concept is similar to Audio2Face (UE 5) so it should either work real time or offline and is used in games like STALKER 2: Heart of Chornobyl by GSC Game World and Fort Solis by Fallen Leaf.
Audio2Face is an AI-based technology that generates facial motion and lip-sync entirely from an audio source. Audio2Face offers various ways to exploit the technology – it can be used at runtime or to generate facial animation for more traditional content creation pipelines. Audio2Face also provides a full character transfer pipeline providing the user a simplified workflow that enables them to drive their own characters with Audio2Face technologies.
...
Users can use the app to make interactive real-time applications or work offline and export caches for use in a traditional facial animation pipeline. Audio2Face can be run live or bake in and exported out, it’s up to the user.
Edit: Update url.
 
Last edited by a moderator:
Personally i do not want to talk to a machine. Not sure about the reasons. But it it's a real discomfort to me. A bit like admitting my humanity no longer grants me an intellectual advantage as well.
So i don't talk to phones, computers or virtual butlers, and i won't talk to NPCs in a game either

Oh, it's brilliant. Facade being possibly the earliest example, all be it by typing freeform text. Instead of trying to solve the relationship issues at the core of the experience, you could roll play being a zombie. I've never laughed so hard, and expect I'll apply zombie rollplay to any LLM based NPCs in this brave new future.
 
But i have no problem with typing text to communicate with the NPC. I'll do that.
Not sure about the reasons either, but i guess more people have such talking barrier.
Though, likely just a generational habit. Probably i'm fine with typing because having played some adventure games as a kid.
Can't really imagine anyone in game development feels any different from you -- this is one of those things that sounds really neat when it doesnt exist in any games, or as a mod for an old game, but once you actually try to play a game you're invested in and this is how all the dialogue works you're going to realize it sucks. I'm sure there will be use for realtime generative AI in games -- maybe something as simple as interpreting user text and returning (hand made) results? But these dialogue demos are silly.
 
I can see AI dialogue being very useful for random NPC's that otherwise aren't involved in the game interactions. Just have them say something relevant to what's going on in the current scene or location or whatever if you get near them.

I also think it's just gonna be a nice crutch for dialogue writing, much like how people use ChatGPT for for ideas and kind of getting a head start. It's easy as a creative to get burnt out when you're having to do all the work on your own and you've got so much to do.

There could also be more procedural/randomized type games where non-designed content is the point and it wouldn't hurt to have AI that provide a more unique experience on each run. I really like the idea of 'filler' side quests being more AI driven rather than the basic templates that many devs use now, so there could perhaps be more variety to what they can offer. I know people generally look down on filler quests, but I think in a big world, it's nice that they're there, whether you do them or not. Main problem is when people have the bizarre idea that you're supposed to do all of them.

Lots of potential with it when used as a tool in an appropriate context.
 
but once you actually try to play a game you're invested in and this is how all the dialogue works you're going to realize it sucks.
I can imagine it may not work well if we take any current game and replace it's NPCs and dialogue system with this. But that's not what i expect or hope to come.
It's clear this tech enables new things which are not possible now. So i guess people will play around with it, and then discover some nice mechanics.
And then they design very different future games around those newly discovered mechanics. And we get what we need so badly: New games. Because that's what new tech is good for.

Personally i always hated dialogue heavy games, so maybe those new games are not too exciting to me. In need another example.
How about using AI for locomotion or complex behavior. This surely comes with it's own problems, since we're used to predictable enemies, not smart ones.
For example, imagine we have a bunch of Bruce Lee videos, and after training our virtual character can fight as good as Bruce Lee knowing all of his moves, tricks, tactics, etc.
Can we make Street Fighter as usual with such enemies?
Likely not. Because the player can't get even in options. Even if the player is martial arts expert too, we can not map all the complexity to N buttons on a gamepad.
To compensate in a fair way, we have to give the same skilled AI to the players character. Though, then the player can only have limited or indirect control over his smart character.

In the end we have a new game, but it's not clear how much fun it is too play, and how the whole experience and impressions would change.
Still, we only need some way to have a proper input and feedback loop. With some luck it works well, and older games suddenly feel like PacMan.
This would be the ideal case. But ofc. it's naive optimism. Personally i assume it's very hard to make games work with AI.
 
If single player games die in favor of forced multiplayer because rich arseholes like all the buzzwords, then we have truly died and gone to hell

Die in what sense? What's in vogue and the focus of the market is going to shift as the demographics shift, that's unavoidable AI or not it's going to happen. People already have been complaining for years (maybe even decades can we say that now?) that games aren't what they used to be.

But AI tools are going to skyrocket the viability of content authoring for smaller projects and if anything make "niche" content design even more prevalent.

With tech demo's you really should be looking at them from the perspective of what technologies they are bringing to the table as opposed to the actual implementation. For instance what can you lift from this tech demo? Could a smaller team (or even individual at one stage) use to author dialogue and the voice acting for a single player game?

I would not be surprised if within the decade we get indy studios capable producing the equivalent art assets using such tools as major AAA projects today.
 
Back
Top