3D Gaming*

[ERP]

Natal seems really promising, as it constructs the skeleton of the user. With the 3d skeleton you have both the head orientation and rotation (in 3d space). With a simple web camera it's much harder to get good quality head position and rotation. Stereo 3d with properly calculated two eye positions with head tracking data should look stunning. If Natal gets as popular as Microsoft predicts, I am sure we will see some Xbox games supporting stereo 3d + head tracking in the future.

If all you care about is basic head position and rotation you can do a pretty good job with just a web cam (assuming you can get high enough resolution data fast enough), and it's not even that hard.
If someone is going to be wearing 3D glasses anyway, it's almost trivial.

 

[MfA]

If you really want to use a normal display as a virtual window with realistic parallax for head movement you will only be able to do ~30 degree FOV (unless you can convince people to sit really close to their displays).

 

[patsu]

Bleeding-edgers. A very limited market of folks that do little to dictate future trends. Most of us end up wasting money on stuff that quickly loses support as the market disappears. I still don't know at this point if the 3D DLPs that have been shipping for over a year will be compatible with Blu-Ray 3D and console 3D. The 120Hz LCD monitors out today have inherent flaws that should not be present in widely-marketed CE products.

DLP came up a few times in various 3D Blu-ray articles. The display tech is independent of the underlying format. It should be supported. In fact, Mitsubishi is releasing a 3D adaptor box for DLP TVs to sync with the 3D glasses (may be supporting its own TV brand only).

3D is young. I don't think they want to throw away the early 3D adopters. In fact, the initial base is one of the reasons the 3D vendors are keen to move ahead (existing enthusiastic base, according to the vendors).

I added a 9800GT to my 5850 to do PhysX, because I'm a (foolish) bleeding-edger. I only have 2 games that it works with, and the effects have been less than impressive. I'll be the first to tell you that PhysX should either be drastically changed to a point where my card probably won't be able to handle it, or simply killed off entirely. When I get my 3D projector later this year, I'll be prepared to make that same evaluation, because I hope that whatever technology eventually becomes the standard will be widely accessible and of reasonable value for my family and friends to adopt.

You picked the wrong trend to bet on.

Natal seems really promising, as it constructs the skeleton of the user. With the 3d skeleton you have both the head orientation and rotation (in 3d space). With a simple web camera it's much harder to get good quality head position and rotation. Stereo 3d with properly calculated two eye positions with head tracking data should look stunning. If Natal gets as popular as Microsoft predicts, I am sure we will see some Xbox games supporting stereo 3d + head tracking in the future.

Yap, I think Microsoft will have a breakthrough with Natal. The concept is sound. The rest is execution (Just don't lean on speech recognition too much). Stereo 3D + head tracking and other augmented/alternate reality stuff will be interesting to track.

 

[patsu]

Nintendo's take:
http://www.siliconera.com/2010/02/0...to-work-with-graphical-circuits-for-3d-games/

When asked for his opinion on 3D technology in games, Nintendo president Satoru Iwata stated that he felt 3D technology wasn’t suited to multiplayer or family-oriented games, which is what Nintendo have been primarily focused on.



However, he went on to reveal that the idea of developing 3D games on a home console has been tossed around at Nintendo before, and that there were even measures taken to help realize it on the Gamecube if the need ever arose.



“In fact there have already been discussions for a possibility of a 3D video game for a long time,” stated Iwata. “To tell you the truth, GameCube is secretly designed to load graphical circuits which display graphics for right and left eyes respectively, for a future possibility of realizing 3D gaming experience. So actually we have had interest on this technology, but I have some doubt about everyone needing glasses to play.”



As final note, he added: “To wrap up, we have interests but are also aware that there are so many hurdles to overcome, thus we don’t believe every kind of game will become 3D environment in no time.”

 

[patsu]

3D specs of HDMI 1.4 made public

http://www.blu-ray.com/news/?id=4136

HDMI Licensing, LLC, the agent responsible for licensing the HDMI specification, has announced, on behalf of the HDMI Founders, that it has made the 3-D portion of the HDMI Specification Version 1.4 available for public download on the HDMI Web site at the official HDMI site. This will make it possible even for non-HDMI licensees to access that part of the spec and make their products compatible with it.

The specification defines common 3-D formats and resolutions, with 3-D support for up to 1080p resolution, and under many 3D structures: full side-by-side; half side-by-side; frame packing; field alternative; line alternative; Left + Depth; and Left + Depth + Gfx + Gfx Depth.

[size=-2]All these people are making frightening progress.[/size]

 

[Squilliam]

Natal seems really promising, as it constructs the skeleton of the user. With the 3d skeleton you have both the head orientation and rotation (in 3d space). With a simple web camera it's much harder to get good quality head position and rotation. Stereo 3d with properly calculated two eye positions with head tracking data should look stunning. If Natal gets as popular as Microsoft predicts, I am sure we will see some Xbox games supporting stereo 3d + head tracking in the future.

How would that effect the performance and reproduction of content on the console? If they are already taking a hit from CPU performance with Natal, would 3D not also give them an even more significant performance penalty and magnify the already steep 15% CPU runtime loss for Natal?

If they truely want to target 3D + Natal it would make more sense to me to refine the console somewhat to be able to do both at once by increasing the performance of the GPU and giving the console the ability to seamlessly spit out Xbox 360 content in 3D with insignificant developer intervention.

 

[patsu]

At the very least, the Natal cam and the Xbox CPU/GPU should be able to rig the photo taking process to synthesize a 3D picture (from the IR depth info).

As I understand, the rest will depend on rendering techniques. If they can do it on major PS3 games, I am sure it is possible on 360 also. I think the only uncertainty is the HDMI 1.2 (or lesser) output.

EDIT: sebbbi, do you have to turn off the 360's auto-scaler ? Will it work well with auto-scaling 2 slightly different pictures and combine them together in our eyes when the scene is dynamic ?

 

[Iron Tiger]

You picked the wrong trend to bet on.

Nah. I never expected PhysX to amount to anything. I was just curious, so I waited for a sale. I figured I can someday give it to my brother, and feel no personal loss. I've secretly wanted PhysX to fail so we can get an efficient, open standard, but that's a subject for another discussion.

 

[patsu]

Nah. I never expected PhysX to amount to anything. I was just curious, so I waited for a sale. I figured I can someday give it to my brother, and feel no personal loss. I've secretly wanted PhysX to fail so we can get an efficient, open standard, but that's a subject for another discussion.

Hmm... then it's not even bleeding edge, just a waste of money. Early adopters usually dictate the trend *a lot*. The HD movie war would be a good example.

 

[patsu]

Capcom on 3D gaming:
http://www.siliconera.com/2010/02/05/capcoms-comments-on-3d-gaming/

“We can produce 3D games now from a technological standpoint. Although we are prepared to start research involving some titles, our decision about adding 3D to games will be made from a strategic perspective,” said a Capcom representative during an investors Q&A session.

Research scope:

Resident Evil 5 supports Nvidia’s Geforce 3D vision glasses and a 3D demo of Dark Void was at this past CES.

 

[Blazkowicz]

Natal + 3D has a problem. it quickly gets as expensive as the console itself.
also Natal is the experience without a controller or add-on you have to carry on yourself, glasses would be an impediment. another factor is glasses won't do 3D blurays on the X360.

so. next time?

 

[Neb]

Nah. I never expected PhysX to amount to anything. I was just curious, so I waited for a sale. I figured I can someday give it to my brother, and feel no personal loss. I've secretly wanted PhysX to fail so we can get an efficient, open standard, but that's a subject for another discussion.

But it has in several games adding quite some that adds to the immersion. I wouldn't call ~30k dynamic water particles with complex physics and interaction something small in Cryostasis which adds alot to immersion. Not either when it is near 60fps with a GTS250 budget card (standalone). But yeah go open standard but until then it delivers very good stuff for millions of gamers something I cant say about 3D glasses so far from what I've seen. :smile:

 

[DeadlyNinja]

Nintendo's take:
http://www.siliconera.com/2010/02/0...to-work-with-graphical-circuits-for-3d-games/

Isn't that weird. Do developers know of this feature and kept under NDA, or were they also in the dark about this feature?

 

[patsu]

Natal + 3D has a problem. it quickly gets as expensive as the console itself.
also Natal is the experience without a controller or add-on you have to carry on yourself, glasses would be an impediment. another factor is glasses won't do 3D blurays on the X360.

They don't need Natal equipments to support 3D gaming right ? Just like we don't need PSEye for playing stereo 3D gaming.

Also, I think Natal is bundled for new customers.

 

[ergem]

Unlikely, it hurts resolution too much I think.

I could see them using an LCD barrier though (stereoscopy without glasses, but it only works in a sweetspot). There are already mobile phones with that kind of screen.

Viewing angle should not be an issue with portable gaming though. I mean you hold it with your hand fixedly.

Well I only thought that 3D screens whether OLED or LCD could really be a neat idea for Sony. I hope the internal developers are at least considering the idea.

 

[ergem]

Technically: yes.
Financially: probably no.

I salute your creative idea, but the screen would require so many pixels (read the technical pages of your link) that it would probably not fit the low cost market of the handhelds.

Edit: beaten by MfA

Edit: Been given it some more thinking, it does not have to go as far as the alioscopy screen with regard to pixels, for a handheld it may be sufficient to just double the number of pixels as a handheld unit does not require a wide view angle. It would be a really neat idea actually, given that there will be much more 3D content (movies, pictures) in the future.

Yes handheld unit does not require wide view angle. Do you really think it's financially impractical? Considering Sony's stance toward 3D standards on the coming years, don't you think maybe Sony is at least considering putting one?

 

[sebbbi]

If they can do it on major PS3 games, I am sure it is possible on 360 also. I think the only uncertainty is the HDMI 1.2 (or lesser) output.

Avatar already supports stereo 3d on Xbox 360 and PS3, and it looks really good.

EDIT: sebbbi, do you have to turn off the 360's auto-scaler ? Will it work well with auto-scaling 2 slightly different pictures and combine them together in our eyes when the scene is dynamic ?

Checkerboard (or odd/even scanline) stereo 3d encoding pattern needs to be pixel precise. Otherwise the TV/projector cannot separate the pixels to both eyes. The TV and projector I have tested support both 720p and 1080p checkerboard input. Naturally if the player has set 1080p mode active in his/her console settings (or the TV only supports stereo 3d with 1080p), your game has to output in 1080p to make the 3d work (as upscaled checkerboard pattern does not work). In Avatar they have handled this by stating it in the 3d stereo settings menu ("set your console resolution to a supported TV 3d resolution"). The game could of course handle this by blitting both 720p eye images to the final 1080p image with a shader that reads every other pixel from different 720p eye texture (but with a standard blit you only get bilinear filtering - hardware scaler produces better upscale quality).

If the TV/projector takes the 3d input in "side by side" encoding, then there is no need to output in the TV/projector native resolution. Hardware scalers work fine for this mode. And this mode doesn't require any fancy combination shader either (just output pixels directly to half horizontal res viewport and change the viewport rectangle for the other eye).

I am not sure what stereo encodings the forthcoming LCD/LED stereo 3d televisions support. I would be really happy if "side by side" encoding was supported, as it requires least extra performance for image combining (odd/even scanline is not that texture cache friendly) and it trasparently supports the hardware scalers in the current consoles.

Also, I think Natal is bundled for new customers.

This is a key point. Currently ps-eye and xbox-vision are only owned by really few players, so it's not commercially viable to support these in AAA games. If a large portion of players have Natal then head tracking using it might be a good solution (depending of performance hit of course). And the head tracking should be trivial to program, as you already have the skeleton constructed (including head position & rotation). With a web camera you have to analyze the picture by your own program code (takes time and money to implement an efficient solution that works really well on various lighting conditions).

If all you care about is basic head position and rotation you can do a pretty good job with just a web cam (assuming you can get high enough resolution data fast enough), and it's not even that hard.
If someone is going to be wearing 3D glasses anyway, it's almost trivial.

The 3d glasses do not help the head position detection at all. Each TV/projector manufacturer glasses have different outlook, and polarized glasses are even completely passive (no communication at all between glasses & TV/projector). The DLP TV and projector systems I have tested all support same stardard interface to connect the glasses IR emitter and there are several glass manufacturers (each having it's own IR emitter technology). So the glass and output device manufacturers are not always even same. The key point here being that the console has no connection and no information of the glasses at all (and it doesn't even know it's outputting stereo 3d signal). The TV set gives the glasses the shutter signal (depending on the connection system used between the TV and glasses).

 

[Shifty Geezer]

Regardless of what design glasses you have, they should provide a very easy reference point to detect and track. You could also have target LEDs in them for tracking purposes, although you'd need a standardised design for that so different TVs all come with useable specs. Tracking would be no harder than Wii if there were two spots to follow, providng a rough Z position too.

 

[Arwin]

Regardless of what design glasses you have, they should provide a very easy reference point to detect and track. You could also have target LEDs in them for tracking purposes, although you'd need a standardised design for that so different TVs all come with useable specs. Tracking would be no harder than Wii if there were two spots to follow, providng a rough Z position too.

True, but on the downside, all specs will be roughly the same size and prevent detecting the eyes. What I think Sebbbi failed to realise is that Sony already has shown code for tracking the head-box as well as the eyes and mouth, and if I remember correctly this is also already available to developers through the SDK. However, with the glasses the actual eye tracking will have to be handled differently, and instead focus on tracking the glasses. I don't think that will be a big problem though. I also can't determine how important it would be to be able to tell the distance of the head from the screen and the eyes from each other. This could help perfect the 3D, but it will pose some serious demands on programmers to get it right and respond to changes fast enough.

As for HDMI 1.2 limitations versus HDMI 1.3, I'm not sure ... HDMI 1.2 can do 1920x1080px60fps too, I think? 3.96Gbit/s vs 10.2Gbit/s means that we have 8MB per frame at 60fps for 1.2, and 20MB per frame at 60fps for 1.3. 1920x1080 = 2.073.600, and at 32bit per pixel, or 4 bytes, that is exactly 8MB. As I'm not seeing much higher bandwidth happening for games it probably won't matter a great deal - in best case scenarios we'll probably see 1280x1440 games at 60fps (30fps per eye) and that is doable on both. 1.3 could theoretically output something like 1920x2160p for Full HD 3D output at 60fps, but I'm not so sure there will be a lot of games that can meaningfully make use of that resolution? Videostreaming might be possible, but I don't know if the BD datarates could support it at a good enough compression rate. It might just be possible on the PS3, but it's not going to happen any time soon, that's for sure.

 

[patsu]

Avatar already supports stereo 3d on Xbox 360 and PS3, and it looks really good.

Was it 720p or something else ? And what 3D encoding format did they use ? (side by side ?).

If the TV/projector takes the 3d input in "side by side" encoding, then there is no need to output in the TV/projector native resolution. Hardware scalers work fine for this mode. And this mode doesn't require any fancy combination shader either (just output pixels directly to half horizontal res viewport and change the viewport rectangle for the other eye).

I am not sure what stereo encodings the forthcoming LCD/LED stereo 3d televisions support. I would be really happy if "side by side" encoding was supported, as it requires least extra performance for image combining (odd/even scanline is not that texture cache friendly) and it trasparently supports the hardware scalers in the current consoles.

I believe side-by-side encoding is supported by HDMI 1.4.

This is a key point. Currently ps-eye and xbox-vision are only owned by really few players, so it's not commercially viable to support these in AAA games. If a large portion of players have Natal then head tracking using it might be a good solution (depending of performance hit of course). And the head tracking should be trivial to program, as you already have the skeleton constructed (including head position & rotation). With a web camera you have to analyze the picture by your own program code (takes time and money to implement an efficient solution that works really well on various lighting conditions).

Yes, bundling is critical for Natal and Arc to proliferate.

As Arwin mentioned, head tracking has already been implemented in the PS3 SDK. I believe there are at least 2 forms:
(A) Omega Tracking tracks the head shape and movement (e.g., nodding)
(B) Facial recognition will detect more (e.g., orientation, with or without glasses, estimated age, gender)

The 3d glasses do not help the head position detection at all.

If necessary, something small can be clipped on to help tracking. Johnny Lee's IR glasses experiment should be available in the Playstation developer network too. The guy who ported the idea to PS3 said that he made it available there.