Xbox Scarlet Hybrid Game-Streaming Version, Kahawai

[Shifty Geezer]

Standard game steaming a la XCloud is simple and proven. The locally enhanced version designed to compensate for latencies (Kahawai) is the interesting one and the one bgroovy challenges viability of due to dev effort. The demo's were online games I think.

Someone watch that. It has a slide on implementation.

 

[iroboto]

Standard game steaming a la XCloud is simple and proven. The locally enhanced version designed to compensate for latencies (Kahawai) is the interesting one and the one bgroovy challenges viability of due to dev effort. The demo's were online games I think.

Someone watch that. It has a slide on implementation.

It seems like they are making progress and there are some applicable use cases here. But it still seems further out before it becomes a useful reality.

 

[Jay]

Standard game steaming a la XCloud is simple and proven. The locally enhanced version designed to compensate for latencies (Kahawai) is the interesting one and the one bgroovy challenges viability of due to dev effort. The demo's were online games I think.

Someone watch that. It has a slide on implementation.

Yea, I meant kahawai is interesting, xcloud not so much to me anyway. Until information comes out that says it's anything more than just what everyone else does I consider it basic streaming that may work better due to being able to leverage own data ceneters and locations. Even working better isn't a given to me.

I'll try watch that a bit later, there was 2 distinct technology though, that may just be one of them?

 

[Silent_Buddha]

How do you account for the synchronisation issues between local client and server?

I can imagine something akin to DVD keyframes at an OS level, but used for synchronization. Say the OS auto-generates (picks) a keyframe every 5, 10, 20, whatever frames to be used as a key frame for synchronization between the local machine and network data stream. The local machine then has a local delay in the presentation of frames (delay determined by periodic pings or whatever). Due to the low resolution local rendering, the memory burden will be mitigated on the local machine (having to store that many frames).

There'd have to be some kind of latency hiding built in, however, to prevent a minor hiccup on the internet that delays several frames from desynchronizing the entire thing, however. So I doubt there would be a large interval between frames if they were to use something analogous to this.

Perhaps it's able to use data for a past frame in some X number of future frames (with quality degrading each frame forward) in cases of minor latency spikes delaying the arrival of expected frame data.

This wouldn't require special effort from a game developer.

Regards,
SB

 

[Shifty Geezer]

The issue is game state. You launch a rocket at a wall. The local machine calculates the explosion and the rubble falling on moving objects. The server does the same a moment later and ends up with a slightly different result. How do you sync the two? To date, all such physics is run server side as I understand it. Crackdown 3 was set to take that to a whole new level by increasing the amount of synchronised data, which is the limiting factor (how many objects can you position and orientate per ms over the internet?). Running phsysics locally and on the server requires complex netcode including things like time warping to rewind game state.

Note this is tallying with what the quote says about using local compute for things like physics.

The cloud console will have a limited amount of compute locally for specific tasks like controller input, image processing, and importantly, collision detection.

Keyframing just the images is an idea but it doesn't match the report, leaving the collision detection to the mercy of your internet latency.

 

[Silent_Buddha]

Keyframing just the images is an idea but it doesn't match the report, leaving the collision detection to the mercy of your internet latency.

Yes, so perhaps not a keyframe of graphics data but a "keyframe" of game state. I'd imagine that the OS should be able to handle something like that. Similar in many ways to what cheat engine or a game profiler does.

Of course that has a penalty of increasing latency, but no more so than current streaming solutions which are round robin as well (streaming taking user input sending it to the servers, rendering then sending back the rendered image).

Regards,
SB

 

[Shifty Geezer]

There's two issues with that. Firstly, how do you send huge amounts of data? That was the discussion around Crackdown 3, and at the moment it looks like the data has been dramatically simplified. Secondly, how exactly does the OS keyframe state? Do you enforce an object representation for all games so the OS can generate data itself? Or is there only one physics engine, Xbox Physics, and all games have to use that? How does that with cross-platform titles? Will a game use Havok on PS5 and have to ported to Xbox Physics to run on all Xboxes?

 

[Silent_Buddha]

There's two issues with that. Firstly, how do you send huge amounts of data? That was the discussion around Crackdown 3, and at the moment it looks like the data has been dramatically simplified. Secondly, how exactly does the OS keyframe state? Do you enforce an object representation for all games so the OS can generate data itself? Or is there only one physics engine, Xbox Physics, and all games have to use that? How does that with cross-platform titles? Will a game use Havok on PS5 and have to ported to Xbox Physics to run on all Xboxes?

As long as MS can run the game locally, I don't image there'd be a need for a special version of any game component or code.

The game state can be as simple as the user input, similar to a streamed game (for this it would be reliant on the game being deterministic, however) or a slice of the memory state of the game. I'd imagine to reduce data transmitted from the user to MS, they'd profile the game (perhaps using AI do it?) to determine just what variables in memory are essential to replicate the state of the game as it exists on the users machine.

For example, in game (in engine) recordings/replays of user actions take very little space. Replays of StarCraft matches lasting minutes I seem to recall only taking up a few KB of storage space. The engine uses that data to basically replay the match. My memory is a bit fuzzy on that, but even if it was a few MB for an entire match, the per frame data would still be relatively miniscule.

Regards,
SB

 

[Jay]

Will a game use Havok on PS5 and have to ported to Xbox Physics to run on all Xboxes?

Not that it invalidates your point, but doesn't MS own havok?

 

[Tkumpathenurpahl]

What if it just rotoscopes bits of the higher quality server image onto the basic, local image?

That way, it doesn't need to transmit entire frames, just select surfaces/objects/characters.

Or a futuristic throwback to the original Resident Evil? The local machine processes all collisions, and renders only the player's avatar, maybe anything with which the avatar physically interacts, and the cloud machine renders the background at an adaptable resolution?

There's two issues with that. Firstly, how do you send huge amounts of data? That was the discussion around Crackdown 3, and at the moment it looks like the data has been dramatically simplified. Secondly, how exactly does the OS keyframe state? Do you enforce an object representation for all games so the OS can generate data itself? Or is there only one physics engine, Xbox Physics, and all games have to use that? How does that with cross-platform titles? Will a game use Havok on PS5 and have to ported to Xbox Physics to run on all Xboxes?

When you put it like that, yeah, some sort of hybrid method's way more challenging to pull off than standard streaming, with a focus on low latency.

Maybe the hybrid focus is a more just a case of fancy upscaling? Using ML on either end of the stream to do things like alternate each frame between high and low resolution, and use the high resolution frame to intelligently upscale the lower resolution successor.

 

[cheapchips]

What if it just rotoscopes bits of the higher quality server image onto the basic, local image?

That way, it doesn't need to transmit entire frames, just select surfaces/objects/characters.

This is litterally how Kahawai works (in one of it's demos).

 

[Tkumpathenurpahl]

Shit, I was gonna patent that ...