Xbox Scarlet Hybrid Game-Streaming Version, Kahawai

[Shifty Geezer]

You don't need speculative execution and many outcomes to be processed. That would be terrible. You render remotely, providing cues such as motion vectors, object type ID etc, then use what's provided to the client to generate the displayed image based on the things you've calculated locally e.g camera and object position.

I can imagine plenty of cases in this scenario of the graphics 'swimming' and being fudged over several frames to poor effect. Any significant change in direction would likely struggle to transition smoothly. I guess one thing to do would be render a larger-than-screen buffer so you've got some room to slide in info at the borders instead of trying to warp out data from a framebuffer that's moved off the edge.

 

[function]

Running part of the game engine locally, and the graphics remotely, is going to require some effort from game devs.

If engines need to be written specifically for the cloud, they must mandate all games to implement it. I don't see how they would support the whole back catalog. It will need a traditional streaming method for most.

If we're going to have cloud based gaming in the future (if!) there may well be something of a pain barrier for developers. I think it will be optional, at least at first. Just as mutlithreading caused issues that are now (largely!) considered (mostly!) historical, the move to centralised processing of graphics data (which will have a significant efficiency payoff for multiplayer online games) may well require some additional unexpected effort at first....

 

[function]

I can imagine plenty of cases in this scenario of the graphics 'swimming' and being fudged over several frames to poor effect. Any significant change in direction would likely struggle to transition smoothly. I guess one thing to do would be render a larger-than-screen buffer so you've got some room to slide in info at the borders instead of trying to warp out data from a framebuffer that's moved off the edge.

Yeah, that was one of my first thoughts too. In most games the camera that follows or represents the player has momentum, especially when controlled by a progressive modifier like a joypad's thumbsticks. You may well be able to plan for this with an "overscan" like area representing where current trajectory places the camera plus a 5% bounds (for example).

 

[MrFox]

If we're going to have cloud based gaming in the future (if!) there may well be something of a pain barrier for developers. I think it will be optional, at least at first. Just as mutlithreading caused issues that are now (largely!) considered (mostly!) historical, the move to centralised processing of graphics data (which will have a significant efficiency payoff for multiplayer online games) may well require some additional unexpected effort at first....

If it starts with only some games it would be a reasonable launch. But supposing the rumored streaming-only SKU already needs enough local power to run the game engine, it won't be a chromecast type of product. It will be relatively expensive.

Streaming games can be complementary to all other distribution choices, it's the streaming-only console rumor I find strange if it can't play anything which wasn't designed for streaming. So maybe it will be at least XB1-level of power to at least play BC. And OTOH, if it can play normal games in streaming it will have the same problems as PS-Now, which doesn't work well unless you are close to the servers, and your ISP is a good one.

 

[bgroovy]

In Europe DVB T2 is coming and 100 dollar android boxes will be saled as bread to equip not so old tvs.... MS adds the possibility to play good games to this kind of boxes by producing one special type ?!? Good idea....

Those devices can already run any number of game streaming services. You don't need more than a $50 device for that. I don't see any scenario where this MS solution is enough of an improvement to justify spending 2-4X as much on the streaming client device.

It's the grunt work of most of the rendering pipeline that's remote. Any latency sensitive logic is local, and you do the best job you can in manipulating the graphic data you have from the cloud to make it fit what's happening in the instant.

What does it matter if the "latency sensitive" stuff is done locally if you can't see any of the results until you hear back from the server? By dramatically increasing the cost of the client hardware, and making game development significantly more complicated you gain what? A barely perceptible improvement in the end user experience? It's no different than the power of the cloud pipedream for single player stuff. It is always, alway more cost effective to prebake your physic in as many variations as you could conceivably imagine than to do it fresh every single time for every single user on a server. The end user will not notice a difference between a good faked experience and a server driven one, except when their wifi goes down and the cloud component breaks. Likewise all this tech will accomplish less than adding a game server 500 miles closer to the user would.

 

[turkey]

Listening back to Phil, he states a game streaming network to play console quality on any device (image on the screen is a phone)

Emphasis mine.

Could this just be a program of rolling dedicated fiber to all major territory ISPs? Or are there more than one streaming solutions in flight?

 

[Tkumpathenurple]

I can imagine plenty of cases in this scenario of the graphics 'swimming' and being fudged over several frames to poor effect. Any significant change in direction would likely struggle to transition smoothly. I guess one thing to do would be render a larger-than-screen buffer so you've got some room to slide in info at the borders instead of trying to warp out data from a framebuffer that's moved off the edge.

Good idea. So render an 8K image, but only stream a 1080-4K image located in the centre thereof.

 

[Shifty Geezer]

You wouldn't need 8K. Just like VR, you'd want an extra 5% border for a total of only 10% more pixels. That should be enough to cover a few frames. I guess you could also optimise the 'overscan' with more in the horizontal and less in the vertical for most games. Well, as everything these days, you could dynamically select overscan amount in each direction depending on context and content - the more the player swings the camera up and down, the more vertical overscan you add.

 

[cheapchips]

Listening back to Phil, he states a game streaming network to play console quality on any device (image on the screen is a phone)

Emphasis mine.

Could this just be a program of rolling dedicated fiber to all major territory ISPs? Or are there more than one streaming solutions in flight?

Microsoft have to convince console buyers that their hybrid streaming console is a good proposition. We have to assume at this point it is, or why would MS bother. Given they have that acceptance hurdle to jump, I'd suspect the higher latency 'steam to any old device' will happen later.

 

[Lalaland]

We have to assume at this point it is, or why would MS bother.

The Xbox One came about because MS decided that the additional TV integration, the Kinect experience and innovative always online connectivity would blow away games running a bit faster on an "old style" box. Businesses aren't always aligned to the same incentives consumers are, MS has a lot of data centres and this would provide a "base load" of usage to help offset those costs, if the experience isn't the same or is noticeably worse but "good enough" in the right people's judgement then this could happen regardless of whether anyone on here would touch it with a bargepole.

I hadn't considered the challenges of making games work in this hybrid model, designing your game engine to support being bifurcated like that for a subset of your users sound non-trivial to me.

 

[BRiT]

Would anyone's opinion of Xbox Scarlet change entirely if it was a mobile console with base resolution of HD but scalable up to 4K with no noticable latency or lag, with the right scenarios with the cloud?

It seems like everyone seems to be downplaying it already and comparing it to the current stationary consoles, yet we have large success of Nintendo Switch which doesnt even do HD at times.

 

[McHuj]

The Xbox One came about because MS decided that the additional TV integration, the Kinect experience and innovative always online connectivity would blow away games running a bit faster on an "old style" box. Businesses aren't always aligned to the same incentives consumers are, MS has a lot of data centres and this would provide a "base load" of usage to help offset those costs, if the experience isn't the same or is noticeably worse but "good enough" in the right people's judgement then this could happen regardless of whether anyone on here would touch it with a bargepole.

Yup, often times the people making decisions make really terrible decisions. Microsoft has probably made more than any other company (Xbox one, Windows 8, Windows mobile/Nokia purchase) so I wouldn’t put it past them to completely miss the mark on this. I hope they can deliver something revolutionary, but I’m not raising my hopes.

 

[Shifty Geezer]

Microsoft have to convince console buyers that their hybrid streaming console is a good proposition.

They have to convince buyers. That might not be console buyers in the traditional sense. MS's predictions could be anything, like 15% streaming and 85% conventional console. Remember that there's the standard box still coming as part of this rumour, so this streaming thing could be as much an experiment as a real next-gen gameplan.

Would anyone's opinion of Xbox Scarlet change entirely if it was a mobile console with base resolution of HD but scalable up to 4K with no noticable latency or lag, with the right scenarios with the cloud?

That would be something fairly different if you're talking about hardware driven 1080p games mobile, and then cloud-enhanced up to 4K when 'docked'. At the moment we're discussing a streaming console rather than an Xbox handheld.

 

[BRiT]

That would be something fairly different if you're talking about hardware driven 1080p games mobile, and then cloud-enhanced up to 4K when 'docked'. At the moment we're discussing a streaming console rather than an Xbox handheld.

Why would it need to be 'docked' to take advantage of 'cloud enhanced' rendering? Plenty of tablets already have higher than 1080p screens.

 

[cheapchips]

They have to convince buyers. That might not be console buyers in the traditional sense. MS's predictions could be anything, like 15% streaming and 85% conventional console. Remember that there's the standard box still coming as part of this rumour, so this streaming thing could be as much an experiment as a real next-gen gameplan.

It's the hybrid bit I'm hung up on. If they're targeting outside of the core audience, then little Chromecast type thing or iOS/Android/SmartTV app is a much lower barrier to entry than a "more expensive" streaming box.

Would anyone's opinion of Xbox Scarlet change entirely if it was a mobile console with base resolution of HD but scalable up to 4K with no noticable latency or lag, with the right scenarios with the cloud?

Or the home streaming Scarlett is limited to 1080p native and 4k streaming.

Does that get you cheap enough though? You could ditch the optical drive but you'd still need local storage.

 

[Shifty Geezer]

Why would it need to be 'docked' to take advantage of 'cloud enhanced' rendering? Plenty of tablets already have higher than 1080p screens.

It wouldn't need to be, but that's the better place for it. 4K on an 8" portable screen is mostly overkill. It only has real value for presenting tiny text. You also have the issues of connectivity - will the thing be using mobile BW? If you're out and about playing games in 1080p, there's very little need to go higher. Unless the cloud can tween framerate which would be more beneficial.

Assuming a portable is powerful enough to play 1080p games in hardware, the cloud feature, you could enable whenever, but you'd mostly want it for TV output. Assuming the portable isn't powerful enough to play any games without connectivity, you've got a device that'd probably fail hard.

 

[cheapchips]

Having thought about it a bit more, I think a 1080p native home console would really make sense as a "more expensive" streaming box. It's only enhanced to 4k with Kahawai. It'll always have a fairly high quality frame to fall back on if the streamed frame enhancement misses a latency threshold.

They can halve or more the silicon budget for the GPU relative to the native 4k box, keep the CPU the same, ditch the optical drive, reduce ram a bit. Should add up to a reasonable cost saving relative to the full fat version.

I don't see why Microsoft can't make syncing of the local 1080p game and 1080p+4k Azure instance and frame delivery/blending more or less invisible to developers.

 

[Shifty Geezer]

Could also use binned parts to improve economies on the main box.

I don't think it could be invisible to devs unless devs use a defined way of doing things and set of API calls. You'll want IO and syncing to be standard, which means being able to sync your game to the Kahawai way.

 

[cheapchips]

Yeah, maybe I should have used the phrase fairly simple rather than invisible.

 

[function]

What does it matter if the "latency sensitive" stuff is done locally if you can't see any of the results until you hear back from the server?

In the example I'm giving you *do* see many of the results *before* you've heard back from the server.

By dramatically increasing the cost of the client hardware, and making game development significantly more complicated you gain what? A barely perceptible improvement in the end user experience?

Cost of client hardware goes down, not up. Game development is constantly getting more complicated, if (if!) this business model works it'll be adapted to. Once you know how to do it, it should be very manageable.

Many of the most popular games are already based around their online experiences with janky, client hosted servers on domestic broadband lines.

The potential for improvement will be vast for a given client box's cost, size, power consumption. Depending on how work is split and the requirements of the client device, it could allow a better than X1X experience on something like a phone, while using less power than a traditional game pushing it to the limit.

A 99$ system that can provide a similar experience to a $499 system would have potential. And AAA developers ultimately want some that that mobile market. Hybrid cloud / client can go places.