Streaming visual information from the Cloud

[Alucardx23]

Hi guys, I want to share an idea that I had and I want to hear your opinion. Since the really big problem that needs to be tackled with cloud gaming/processing is lag, why not make the combination of streaming visual information with local processing? This would solve the problem of input lag and would give the advantage of the processing power that is available remotely.

Think of this as an evolution of the way graphics were handled on Resident Evil 2 or any other game with pre processed backgrounds from Donkey Kong to Fear Effect. These type of games managed to give the impression of higher detail by Pre processing the backgrounds on stronger hardware. On my example instead of the pre processed backgrounds saved on the disk as pictures, it would be a stream of visual information from the server.

Taking this God of War 3 picture as an example of this, only the elements that are inside the red line would be rendered locally; anything outside of the red line would be rendered on a remote server.

From the console perspective it would just need to replace the green background with the video stream.

This could also be used to stream texture information as a video, think of it as an evolution of parallax mapping, to stream 3D textures. Why would I need to draw and save the textures locally in the example below, if I can just stream the information from the server as a video? If I’m not mistaken this would mean faster loading times and it would free up a lot of local processing to be used on all things that are sensitive to lag.

Since local memory would not be a limitation you can put all the details that you want.

 

[Shifty Geezer]

I'm pretty sure this has already been mentioned in the cloud computing thread as an option. It's definitely doable, although you'd have an issue synchronising cameras. If there's 100 ms lag on the streamed imagery, and 30 ms in game, there'll be a 70ms (4 frames) delay when you move the camera. More latency discrepancy would be worse. It could be worked around, I'm sure, but it'd affect camera design/choices or game style choices. A shooter is probably the worst possible example. Running down a corridor with the walls being 4 frames behind your location, and then rotating and having them warped into place to adjust for the wrong perspective, would probably akin to a narcotics experience, I imagine.

 

[Pixel]

in the given examples why couldn't the videosteamed environment be a 'preprocessed background' stored as a video or images and stored locally in the install folders on the hdd.

These examples you give have non-controllable perspectives. If latency negates interaction with cloud render nonusercontrolled perspective environments, then why not just store them the old fashion way as prerended backgrounds or videos on the hdd.

 

[Shifty Geezer]

You could of course do that if you wanted, but then you're stuck with static renders. Great for an on rails shooter, say, or a scene-based detective game, but otherwise pretty limited to fixed cameras. Alucardx23's idea is to create the backgrounds in realtime using cloud computing, so the cameras could be fully dynamic as long as you can deal with the latency.

 

[Alucardx23]

Thanks Shifty, that is exactly what I was thinking about. @Pixel Of course it can also be done like that, but like Shifty already said, it would be less dynamic. I actually started thinking about this when I was watching some Resident Evil 2 videos and I thought about how a game like that would look using all of the available Blu Ray space and processing power that we now have, but we all know how static these games can be, so streaming a dynamic video feed would solve that problem.

I know the challenge would be synchronizing both cameras perspectives, specially for things that are close, but it should be easier to do for objects that are far away in the distance, since the perspective changes are less pronounced. Lets take the castle below.

If we could take this castle and convert it to a video feed of that same area, we would be offloading the local resources needed to render it. The video feed for the castle would just slowly update as you move around it. As you move closer to it the video feed could change to local processing for the castle, so in that case it would also work as a LOD solution.

 

[brunogm]

Maybe things could be even better if IPv6 multicast is used so many players share the feed if applicable like co-op bird view.

 

[Alucardx23]

Maybe things could be even better if IPv6 multicast is used so many players share the feed if applicable like co-op bird view.

Something like streaming the Skybox in an open world game? That information can be broadcasted to a group of players regardless of what they are doing locally.

 

[brunogm]

Yep

 

[fehu]

Something like streaming the Skybox in an open world game? That information can be broadcasted to a group of players regardless of what they are doing locally.

even if doable, why spend bw and researce in something so light on the console?

 

[Alucardx23]

even if doable, why spend bw and researce in something so light on the console?

I guess it all depends on what is happening in that Skybox. If you're just doing 2D clouds it wouldn't help a lot to stream that information, since it does not require a lot of processing power, but let's say you want to simulate several kilometers of volumetric clouds, with physics and lighting simulations, then you're doing something that is not feasible to calculate locally. You can have servers making the calculations for different weather simulations at the sky level and then broadcasting that information to several users.

Halo Reach also comes to mind as a good use for streaming the sky box, since it has big battles happening in the background, you could make part of that battle simulated in a remote server and then broadcast it (replacing the Green with the remote video feed).

 

[Shifty Geezer]

Although true, one does have to wonder on the economic feasibility of expending so much effort on some clouds. I'm not sure the questions 'can it be done?' and 'should it be done?' have the same answer.

 

[Alucardx23]

Although true, one does have to wonder on the economic feasibility of expending so much effort on some clouds. I'm not sure the questions 'can it be done?' and 'should it be done?' have the same answer.

Well I'm just brainstorming a little bit , on this example a server is making the calculations and broadcasting the same information to several clients, so is not like every client will have a dedicated server. It doesn't need to be clouds as I mentioned, though I would find it funny to use the power of the cloud to improve the clouds (bad joke) :smile:, but you can surely see how this can be used to expand the game world beyond what can be rendered locally.

 

[Shifty Geezer]

Certainly to a degree. The idea of a background battle-zone would work, calculated in real time from the player's actions in their own part of it.

 

[brunogm]

Let's hope for amazing weather. Game design could also go crazy in a space environment, like the background "skybox" could render billions of stars with heavy effects of all kinds.

 

[Arwin]

I'm actually successfully streaming DriveClub from my PS4 at home to my Vita at work - this is the first time it works for me remotely, and it's actually driveable (at least with not too fast cars ). So I do see the potential here, but at the same time that game does clouds (and apparently soon weather) very well on its own. And the game is also unwittingly a good example of the limitations of online (lots of issues so far).

 

[Rodéric]

The problem is always time, it's fine to have incredible power on a remote server but you have to send it data, wait for the computation to happen, get it back then display it.
Displaying it is already lagging, should improve with future API (at last), sending & receiving will depend on your and the server internet connection speed and "ping" (bandwidth and latency) and computing it depend on the server power and the program running there.

We can do a lot of things, the problem is to work within the boundaries of those constraints, for some players it will be fine, but not for everyone, as not everyone has the same quality of internet connection, so I don't really see it as a reliable solution.

I'd rather run some background A.I. or such, some popular games only update the A.I. at 10Hz, which should work for almost any internet connection. (Maybe the internet connection characteristics will become requirements on the box in the future...)

I think a fair amount of game logic could be run on the server, but I'd keep frame computation local, then again it depends how much is going on, but it could be a solution.

And of course we could go completely client/server like Gaikai, but then why have such a high cost of entry (ie the console) ?

 

[Alucardx23]

The problem is always time, it's fine to have incredible power on a remote server but you have to send it data, wait for the computation to happen, get it back then display it.
Displaying it is already lagging, should improve with future API (at last), sending & receiving will depend on your and the server internet connection speed and "ping" (bandwidth and latency) and computing it depend on the server power and the program running there.

We can do a lot of things, the problem is to work within the boundaries of those constraints, for some players it will be fine, but not for everyone, as not everyone has the same quality of internet connection, so I don't really see it as a reliable solution.

I'd rather run some background A.I. or such, some popular games only update the A.I. at 10Hz, which should work for almost any internet connection. (Maybe the internet connection characteristics will become requirements on the box in the future...)

I think a fair amount of game logic could be run on the server, but I'd keep frame computation local, then again it depends how much is going on, but it could be a solution.

And of course we could go completely client/server like Gaikai, but then why have such a high cost of entry (ie the console) ?

I think services like Gaikai/On Live with all of the processing happening remotely will be the future, but for now a combination of local/remote processing should be the steeping stone in to that. Since not everybody has the required internet connection, the data that is streamed from the server should not directly affect gameplay, but that's just a matter of time until high speed/low latency connections are widespread.