Support for Machine Learning (ML) on PS5 and Series X?

[iroboto]

This is why I always throw in the cautionary tale of the unpredictability of machine learning. Once you open that box belonging to Pandora, its difficult to close. There are things that ML could be really suited for, like multiplayer bots when there aren't enough people but this would be massively risky having bots learn from player behaviour because it can be manipulated/influenced.

yea if the goal is really to make ML/AI fun; we can introduce several policies to come up to make a decision that drives behaviour. So we can train it based a million trained games ; we can have it examine the environment and make basic AI goals, and we can have it do learned behaviour from the player. And then having the 3 Policies come up with their respective next step and then have RL AI choose the next best step.

it is of course; insanely taxing.
You’d be one hell of an engineer if you could get this to go well in real time with a small footprint in memory.

 

[BRiT]

Perhaps, but who wants bots that camp and tea-bag -- if it's purely learned from live MP games ?

 

[manux]

Could keep learning in multilayer games and improve bot ai etc

I would imagine at least for now developers would want to use highly curated models. This is to prevent the models becoming not fun, offensive or otherwise buggy.

It's super easy for the AI to become messed up if it got retrained on arbitrary input. Another practical issue is that trained models today rarely are good for production unless they go through a lot of manual optimization/tweaking. It's a whole art in itself to bridge the training and move to production gap as of today.

 

[iroboto]

Perhaps, but who wants bots that camp and tea-bag -- if it's purely learned from live MP games ?

You can assign negative points for tea-bagging and poor manners. By default reinforced learning AI is incentive to maximize the highest score possible.

 

[Deleted member 11852]

yea if the goal is really to make ML/AI fun; we can introduce several policies to come up to make a decision that drives behaviour.

If you referring to making algorithms that are adaptive but only in certain ways, speaking from experience, this is really difficult to do. Insanely difficult. Because algorithms adapt to data and stimulus, how you do know which aspects of an adapting algorithm are changing for good or bad reasons? You can limit change, but only by limiting the positives as well.

 

[iroboto]

If you referring to making algorithms that are adaptive but only in certain ways, speaking from experience, this is really difficult to do. Insanely difficult. Because algorithms adapt to data and stimulus, how you do know which aspects of an adapting algorithm are changing for good or bad reasons? You can limit change, but only by limiting the positives as well.

Deep Learned models all adapt based on what you train it and which type of statistical method you use. Say monte-carlo is pretty good for this type of thing. If you put in a setup it's never seen before, it will still come up with an answer based on how you've trained it. It can do all sorts of things on its own just based on its training. And it will adapt to the player behaviour, without retraining it's own behaviour.

But if the goal it to actually _learn_ from the player, as in the the players unique signature is being adapted to, sort of like how 2 tennis players will slowly adapt to each others 'tells', and strategies, to know what their opponent is both weak and strong at and to take advantage of this (though this is different from taking advantage of injury) this type of real time reinforcement learning is going to need to come with a lot of additional stuff to keep the AI from biasing too hard one way in one direction or the other. And if you keep biasing the training set enough, you'll result with hitlertwitter over a long enough time. So this is where we need to start putting in competing algorithms that come up with their own move, and the reinforcement learning algorithm will pick based on past history, which move is likely going to result in a higher point score.

 

[Shifty Geezer]

Maybe, it depends what the AI is adapting too. You don't want the AI adapting to the actions of other stupid AI, but perhaps to player's behaviour? Even that could go wobbly quickly, i.e. you go on a killing spree for 30 minute and now all the AI pedestrians run away from you on sight you all the time. It'd be like a hideous bug you could never fix unless you start over.

I am reminded of Black and White, and how the pet was untrainable and in the end, one just ignored it and used direct powers.

 

[PSman1700]

I am reminded of Black and White

The memories, that was a different but great game for it's time. For anyone intrested

 

[Shifty Geezer]

I don't remember any of that. All I remember is a dumb panda/monkey/lion who'd keep shitting in the villages and eating stuff he wasn't supposed to. It'd be nice to see the idea developed in a style that actually works. TBH I think I'd like to see a resurgence of god-games with ML driving stuff like minions, peeps, and pets so they actually could be trained.

 

[iroboto]

I don't remember any of that. All I remember is a dumb panda/monkey/lion who'd keep shitting in the villages and eating stuff he wasn't supposed to. It'd be nice to see the idea developed in a style that actualyl works. TBH I think I'd like to see a resurgence of god-games with ML driving stuff like minions, peeps, and pets so they actually could be trained.

Yea B&W would be a good fit. If you want them to teabag after carrying a harvest for approval points this makes sense lol.

 

[iroboto]

I don't remember any of that. All I remember is a dumb panda/monkey/lion who'd keep shitting in the villages and eating stuff he wasn't supposed to. It'd be nice to see the idea developed in a style that actualyl works. TBH I think I'd like to see a resurgence of god-games with ML driving stuff like minions, peeps, and pets so they actually could be trained.

you make it in unity, I'll work on the AI P

 

[Deleted member 11852]

I don't remember any of that. All I remember is a dumb panda/monkey/lion who'd keep shitting in the villages and eating stuff he wasn't supposed to.

The trick with getting the creature in B&W to behave how you wanted, was to punish/reward immediately after it had done an action. This was the flaw, because you had to watch the damn thing all the time, which just isn't fun. But you're right with the powers, once it had done something bad a few times without punished (because you didn't see it) it seemed really difficult to untrained it.

An interesting game-design experiment for sure, but how is it better than just being to able set behavioural options from a menu? Don't eat, villagers, heal the sick etc. Or, just be good/bad!?! It was a convoluted means to an and end unless creative interaction was the draw.

 

[Silent_Buddha]

Maybe, it depends what the AI is adapting too. You don't want the AI adapting to the actions of other stupid AI, but perhaps to player's behaviour? Even that could go wobbly quickly, i.e. you go on a killing spree for 30 minute and now all the AI pedestrians run away from you on sight you all the time. It'd be like a hideous bug you could never fix unless you start over.

But isn't that what the pedestrians should be doing? I find it annoying if I go on a killing spree around civilians and then the civilians just walk around like I didn't just slaughter a bunch of people.

Maybe one of the reasons that I just can't get into open world games. Granted there are some games that attempt to have NPC reacts as you would expect them to based on players actions, but even then they'll either easily forget what you did or you just give them something and they forget what you did.

I do, however, like the handful of games which actually remember your actions and rewards or punishes you appropriately in game. Games where actions have weight and consequence are games I love.

ML pedestrians like in your example would be fantastic for someone like me.

Regards,
SB

 

[Deleted member 11852]

But isn't that what the pedestrians should be doing? I find it annoying if I go on a killing spree around civilians and then the civilians just walk around like I didn't just slaughter a bunch of people.

During and immediately after the event, yes but not in perpetuity. Don't bank on behaviour adaptions being nuanced or temporary. It's all this little things where you're saddening creating lots of additional adaptive behaviour policies.

 

[Shifty Geezer]

The trick with getting the creature in B&W to behave how you wanted, was to punish/reward immediately after it had done an action.

That's how I started playing the game, just watching him rather than doing anything else. It didn't work. The interface was also a bit troublesome in that a stroke could end up a slap - mixed signals or what!

An interesting game-design experiment for sure, but how is it better than just being to able set behavioural options from a menu? Don't eat, villagers, heal the sick etc. Or, just be good/bad!?! It was a convoluted means to an and end unless creative interaction was the draw.

I guess the idea is a virtual animal you'd feel more a connection with because it wasn't a robot. If you follow your distillation to its greatest end, why bother with all the stick and button inputs rather than having a script where the player can type in "I win"? Games are about creating challenges for the user to solve. Training an animal is a good challenge, just so long as it works right. And I think AI entities could overall be a good change in the gaming landscape.

 

[Deleted member 11852]

I guess the idea is a virtual animal you'd feel more a connection with because it wasn't a robot. If you follow your distillation to its greatest end, why bother with all the stick and button inputs rather than having a script where the player can type in "I win"? Games are about creating challenges for the user to solve. Training an animal is a good challenge, just so long as it works right. And I think AI entities could overall be a good change in the gaming landscape.

The first Black and White was the moment when Peter Molyneux began overpromising gimmicks as revolutionary gameplay. Remove the creature from Black & White and you're left with an iteration of Populous. It wasn't a bad game but by six hours in you had seen all the gameplay you were ever going to see.

I don't know what B&W2 did differently.

 

[Silent_Buddha]

During and immediately after the event, yes but not in perpetuity. Don't bank on behaviour adaptions being nuanced or temporary. It's all this little things where you're saddening creating lots of additional adaptive behaviour policies.

IMO, it should also persist. In something like GTA V, almost everyone in the city would likely know about what you did and immediately report you to the police and run and hide whenever you came around. The news would spread via, TV, radio, print, social media, etc. People with cell phones would be able to take your picture and spread it around.

In something like RDR 2, knowledge of your exploits would spread slower, if at all. It's more likely to be localized due to there not being good, easy, or even wide spread news accounts. But people moving around the map would still be able to spread word of your violent killing spree. After all, back then people still liked to talk, but there wasn't a whole lot to talk about. So, in essence your actions when known should carry far more weight than say in GTA V where gang violence is more the order of the day. ML actors would be good for something like this.

I think it's one of the great failings of open world gaming that NPCs have the memory of a wet noodle. At least for me, it means that I very quickly lose interest in most open world games. Ones that have at least some level of consequence to actions generally can at least hold my interest longer.

Granted, this isn't for everyone. Many people enjoy acting irresponsibly in games and would be annoyed if their actions had long lasting consequences because the NPCs weren't just the idiot dolls that they've come to expect.

Regards,
SB

 

[AlNom]

I don't suppose ML would be the next step for Euphoria

 

[arhra]

Offline training of AI models based on player behaviour has existed this entire generation - Forza's drivatars supposedly use a model that's trained on uploaded player data, to generate AI actors who are supposed to drive somewhat similarly to the player they're based on.

They even ran into some of the issues people here have been raising, with the AI behaviour becoming somewhat of an issue in FM6, because they were driving too much like actual human players (ie, complete assholes - they eventually added some controls to artificially limit drivatar aggression, to stop them just running everyone off the road).

 

[AlNom]

Ah..... so that's what that setting was about. xD