Remote game services (OnLive, Gaikai, etc.)

[Arwin]

While the games run at 60fps server side, that doesn't mean they have to on the client side, where they video stream could be 30fps (and I think it currently is). You need 33ms latency to make response time within a 30fps frame (that's 16ms one way). Encoding at 1ms latency exists already as far as I know, so that's possible too. Note that the controller can send data at a much higher rate and separate from the video stream. Also I think you can predict some of the controller input intelligently to hide lag more. If the server is close enough (and you need to distribute them across the country to make this work anyway).

I still think this is going to work, at least on a technical level - at the very least for a (n ever increasing) subset of games. On a commercial level I'm less certain, but I'm sure we'll find out.

 

[hoho]

One of the devs mentioned that on the GDC booths they are using a single 6mbit connection for 3 OnLive users.

The videos they showed there are looked far better quality than even the best 2mbit video streams can offer.

You need 33ms latency to make response time within a 30fps frame (that's 16ms one way). Encoding at 1ms latency exists already as far as I know, so that's possible too.

If game runs at 30fps and the stream is sent at 5mbit/s it takes around 30ms to send a single frame over that connection while completely capping the connection throughput.

 

[function]

Look at the games being mooted in the front end demo. How many of those could you run at 720p60 simultaneously even on the most high-end PC hardware? And you need to be realistic here - the encoder card isn't encoding at 1ms. Heavily parallelised MJPEG encoding done on hardware, yes. HD at 5mbps? That's just ridiculous. It's when claims like this are being made that you have to question the whole thing.

But you said that a hardware encoder card intimated that each client had its own dedicated PC to connect to. Why would this be the case?

720p60 at 5mbps is barely watchable... 2mbps would be like FMV on the Sega CD on all but the most static of scenes.

Have you heard of BBC iPlayer?

Their "high quality" option is 796 kbps (at least on the stuff I've watched) at 640 * 360 and while it's not perfect it's perfectly watchable and looks okay even on a tv. It's light years beyond even the best 32X enhanced Sega CD stuff.

With lower quality compression but higher bit rates I don't see why an imperfect but workable solution is as physically impossible as you are insisting.

I do see some potentially huge problems for the service, but these are mostly related to inconsistent service delivery with jerks and hangs wrecking gameplay, and total latency (input->send input->processing and rendering->send image->display image etc) becoming unpleasant.

 

[grandmaster]

In one of the interviews with Perlmen he says that the hardware encoding is done via an expansion card in each PC. They are not refered to as dedicated standalone encoder units with multiple inputs, although that's the way I'd do it personally, so the only info I have is what they have told us.

With regards the iPlayer, there is a world of difference on several points.

1. Bi-directional prediction - info can be taken from the past and the future of each individual frame. This represents incredible potential for compression and the buffer can be as big as you like.
2. Multiple-pass quality-based encoding - allocate the bandwidth to where it's needed most, not possible with OnLive
2. Content - fast-moving game footage is a world apart from the sort of talking heads stuff you get from TV. TV also typically has far less in the way of bright, vibrant colours, less stuff going on, far less dynamic cameras etc. In short, more predictable and easier to compress.

I'll leave you with the bare maths of the situation.

5000kbps = 625K/s. Divided by 60 = 10.41K/s. That's how much bandwidth they have for a 720p intra frame with 5.1 surround sound. Sorry, I just don't buy it. You can double that 10.41K/s if they are running at 30fps as I suspect they are (maybe using interlacing or even frame blending, though both would hurt efficiency) but it's still going to look poor.

 

[Sindre]

Don't know if you've seen this.

Richard Leadbetter's article on why OnLive "can't possibly work".

I'm very excited about the whole idea, but I'm becoming more and more skeptical the more I read.

http://www.eurogamer.net/articles/gdc-why-onlive-cant-possibly-work-article

 

[Quaz51]

it's probably Jpeg2000 compression

 

[RancidLunchmeat]

I'm very excited about the whole idea, but I'm becoming more and more skeptical the more I read.

http://www.eurogamer.net/articles/gdc-why-onlive-cant-possibly-work-article

Why? It's word for word what grandmaster wrote on page 4.

 

[hoho]

it's probably Jpeg2000 compression

Have you tried compressing anything with jpeg2k? First, it takes ages. Second to fit 60 720p frames into 4Mbit pipe you have to compress each frame to around 11kB, including sound. Good luck with that.

 

[Sindre]

Why? It's word for word what grandmaster wrote on page 4.

Why what?

 

[Quaz51]

Have you tried compressing anything with jpeg2k? First, it takes ages. Second to fit 60 720p frames into 4Mbit pipe you have to compress each frame to around 11kB, including sound. Good luck with that.

need a spatial compression for reduce latency and for others reason and there are lot of realtime Jpeg2k compression hardware (digital cinema use Jpeg2000 for 4k and 2k film)
and i think onlive service is 30fps locked. but yes, need more and more bitrate for good quality

JPEG2000 Enables Wireless HD Video Distribution in the Home

JPEG2000 Is Ideal for Wireless Video
• Low latency—less than one frame
• More robust to errors than MPEG-X
• Error resiliency increases range
• Fixed bit rate simplifies design

JPEG2000 Is Ideal for Consumer Applications
• Capable of real-time HD compression
• Symmetric encode and decode
• Low cost, low complexity
• No external memory required

JPEG2000 Code Stream Is Scalable for Resolution and Quality
• Single encoded stream can supply different resolution displays
• Free thumbnail images available
• Dynamically adjustable bit rate: trade quality for bandwidth
• Reduce file size by reducing quality, without transcoding

from a CES2006 demo

Q: How do UWB and JPEG2000 enable wireless HD gaming? Why can’t MPEG-2 be used?
A: The video from a gaming console must be compressed in real time before being wirelessly transmitted. JPEG2000 provides better image quality than MPEG-2 for real-time compression of either SD or HD at the price points of interest.
Real-time compression is not the same thing as low latency. Because it must wait for several compressed frames to display a single decompressed frame, MPEG-2 has high latency. Latency and bandwidth can be traded by going to I-frame only, but, as discussed earlier, this is an expensive, complex proposition even for SD; recall that real-time HD MPEG-2 encoders are thousands of dollars.

 

[RancidLunchmeat]

Why what?

Why would you become 'more and more skeptical' after reading a Eurogamer article that doesn't say anything more than what was already stated on page 4 of this thread almost word for word?

 

[hoho]

need a spatial compression for reduce latency and for others reason

So basically you think they compress each frame individually? It could be done but resulting image will not fit through the pipe they claim it would.



http://upload.wikimedia.org/wikipedia/commons/7/78/JPEG_2000_Artifacts_Demonstration.png

That last one is 1:100 compression. To fit 720p even at only 30fps through 5Mbit pipe (they clamed 60fps through 4mbit) they would need around 21kb per frame or 1:1000 compression.

 

[Sindre]

Why would you become 'more and more skeptical' after reading a Eurogamer article that doesn't say anything more than what was already stated on page 4 of this thread almost word for word?

Oh, sorry;-) I didn't mean it directly with regards to the article, which you are right, is almost word by word what has been discussed here. Just in general, based on the feedback from this thread, hands-on impressions and articles on other sites.

 

[Quaz51]

So basically you think they compress each frame individually? It could be done but resulting image will not fit through the pipe they claim it would.



http://upload.wikimedia.org/wikipedia/commons/7/78/JPEG_2000_Artifacts_Demonstration.png

That last one is 1:100 compression. To fit 720p even at only 30fps through 5Mbit pipe (they clamed 60fps through 4mbit) they would need around 21kb per frame or 1:1000 compression.

jpeg2000 is good for HD resolution, not for very low resolution like this exemple (190x190)
and i think they merge many frames in one frame for simulate better framerate (motion blur) maybe they merge 60fps source in a 20fps video flux

4Mbps jpeg2k video is very low but temporal compression (like Mpeg) for this type of service would be bad. maybe they have a very good new intermediate solution (surely)

 

[obonicus]

Natta.

 

[Richard]

while noting problems, this sounds like a more positive experience ... http://i.gizmodo.com/5184502/onlive-streaming-games-hands+on-impressions

A few points:

I played Bioshock using the PC setup, which involved an average looking Dell laptop, and a Logitech control pad. The game was running 50 miles away on a server in Santa Clara, and load times are pretty much the same as running the game on your PC.

Seems like the test was under near-optimal conditions:

1) Bioshock, a game so slow-paced I actually beat with the XBOX controller instead of the mouse/keyb.

2) 50 miles away is pratically local computing these days.

Why the same load times? If they have video hardware and memory to display the game so much better than a local pc, are they skimping on fast HDDs?

Once I got started with gameplay, I noticed the slightest bit of lag between controller and screen. Just enough to not feel natural, but hardly enough to really detract from gameplay.

It would have been nice to have a point of reference. Same lag as triple-buffering? 60Hz rodent with Quake 3? 100ms of MP lag?

Some of the indoor textures and environments looked pretty close to running on your own console, with minor dropoffs in sharpness and clarity.

I consider many of the textures in Bioshock to be a tad low rez already... hmm..

Anyway, I hope the bit discussed in the previous page about high-end games requiring more bandwidth to be a misunderstanding otherwise that comment, if coming from them, would seal this as a hoax.

 

[xatnys]

VNC for video games?

 

[RenegadeRocks]

I also mentioned the possibility of jpeg2000 compression a few pages back, but would the resulting quality be enough to satisfy a gamer ?
I am not a movie-goer, I am a gamer who likes to see how good a game looks. With colours being clamped to web standards, reduction in blacks and contrast, will I really like what I see on my screen?
Add to that loss of the sharpness I see in my games right now.........

 

[hoho]

I also mentioned the possibility of jpeg2000 compression a few pages back, but would the resulting quality be enough to satisfy a gamer ?

Using JPEG2000 encoder from here I created three sets of images. They are compressed to match the required bandwidth for 720p@30fps for 5, 10 and 20 Mbit/s. I converted the images from jpeg2k back to png so people without jpeg2k support could view them.

Original:

5Mbit

10Mbit

20Mbit


Even at 20Mbit the quality difference is quite significant.
Original jpeg2k images are here:
http://hoho.bafsoft.net/jpeg2k/test-5mbit.jp2
http://hoho.bafsoft.net/jpeg2k/test-10mbit.jp2
http://hoho.bafsoft.net/jpeg2k/test-20mbit.jp2

I understand that the source image I used was already in lossy format (jpeg) but I don't think it matters much.

 

[RenegadeRocks]

Well, 5mb and 10 mb ones are totally unacceptable ! I think they must have come up with something better than that to even announce the service ! Have they elaborated on their compression tech further?