3D Gaming*

[Iron Tiger]

Head tracking is a considerably cheaper proposition, which should be doable with nothing more than a $20-$50 webcam (Natal?). The scene doesn't have to be rendered twice, so there's a much smaller performance hit, and the image filtering workload can be managed on the CPU or the GPU.

I'd love it if MS added a headtracking API to DirectX with the introduction of Natal, so it would be GPU vendor-agnostic and get supported by more games. PC gamers typically sit much closer to the screen/camera, so they could go as far as tracking eyes for a more granular input, and minimal neck strain.

 

[patsu]

Depends on what you want I guess. There are cheaper propositions than head tracking too, doesn't always mean they are substitutes (to everyone) !

I think ARToolkit will be part of OS moving forward. We should be expecting them on Windows, Linux, Macs, phone, consoles and other CE devices, including TVs.

 

[PeterT]

I really don't understand why head tracking and 3D are lumped together so often, or even presented as alternatives. They are completely orthogonal. IMHO head tracking is little more than a nice checkbox feature, while well-implemented 3D is a qualitative difference in experience. Though admittedly I've not tried head tracking in any "real" game.

 

[MfA]

Headtracking with a fixed display is pretty retarded, I don't see what motion control (whether with your head or otherwise) has to do with the topic at hand though.

 

[Iron Tiger]

IMHO head tracking is little more than a nice checkbox feature, while well-implemented 3D is a qualitative difference in experience.

I'm still waiting to see what "well-implemented 3D" looks like. I need a real sense of volume, while everything out right now uses stereoscopic trickery. The illusion takes a toll on your eyes or your brain after a while, and my idea of "well-implemented" doesn't have that drawback.

 

[patsu]

If you think about it, the very concept of animation/framerate (and lighting, and many other graphics techniques) is a trickery too. They approximate the real thing at reasonable cost. Some folks will find it hard to perceive 3D (headache, can't focus, hate weight of glasses, blah). I think the key point is head tracking and 3D are orthogonal. They should not be mixed up together. They don't replace each other. They may complement each other though.

 

[ERP]

Headtracking with a fixed display is pretty retarded, I don't see what motion control (whether with your head or otherwise) has to do with the topic at hand though.

When people talk about head tracking with fixed displays they are usually talking about tracking the position of the head relative to the display, which provides additional depth information, and can look very 3 dimensional even without stereo display.
It's like looking through a window I get a different view based on where I look from.

There is an interesting effect with static stereo displays where moving your head left and right even relatively small ammounts makes the world appear to bend because your brain is expecting what's visible to change.

I think it can add to an experience, but I'm not entirely sold on its value, and I have mocked it up with a web cam admittedly with relatively low framerates (cause my web cam is crap).

 

[patsu]

That's not very different from a FPS shooter walking in a battlefield though. The 3D-ness is at the same level. There is no additional information. In your window example, all you need to do is to jump out of the window like a FPS soldier and you're in a "deeper" 3D world.

In 3D visualization, you don't have to move your head or body to realize/perceive/visualize the 3D-ness. Your eyes "alone" (One quick look) should be sufficient to perceive the volume.

Head tracking (or just walking around) would be good to admire the 3D objects from different perspectives/views.

 

[Iron Tiger]

That's not very different from a FPS shooter walking in a battlefield though. The 3D-ness is at the same level. There is no additional information. In your window example, all you need to do is to jump out of the window like a FPS soldier and you're in a "deeper" 3D world.

Ideally, head tracking will also work on the Z axis, and convey a realistic sense of depth as in the Johnny Chung Lee demo.

In 3D visualization, you don't have to move your head or body to realize/perceive/visualize the 3D-ness. Your eyes "alone" (One quick look) should be sufficient to perceive the volume.

There's just several things wrong with it. First, with the illusion of depth, your eyes will try to focus the way they normally do. It doesn't work right because you're staring at a 2D surface, and you're trying to look closer or farther away than where it really is.

Second, you are going to move around, even if unintentionally, and when your perspective doesn't shift, your brain doesn't understand. I found myself sitting very still and telling myself to focus to the depth of where the screen was to keep the illusion working in Avatar, and I had to do that every couple of minutes after the first 15 or so.

 

[PeterT]

I disagree with calling stereoscopic 3D "trickery". It's not more so than anything else we do in 3D rendering, and one could argue it's actually less "fake" than many other approximations -- you only see reality with 2 eyes after all.
Regarding headaches etc., I think that varies from person to person, like motion sickness in FPS games. I've used 3D vision for 5 hours with only a small break and suffered no ill effects.

I think it can add to an experience, but I'm not entirely sold on its value, and I have mocked it up with a web cam admittedly with relatively low framerates (cause my web cam is crap).

Get a PS3 eye if you can/care, it's rather cheap and great for vision applications compared to most webcams (at least for what I did).

 

[Iron Tiger]

You're showing two different images to your eyes to trick your brain into perceiving perspective and depth. Yeah, all 3D representations on 2D surfaces are trickery, but stereoscopic trickery isn't consistently effective. For some people it just doesn't work. For some, it causes headaches and/or nausea. For some, the illusion wears off in time.

 

[patsu]

Ideally, head tracking will also work on the Z axis, and convey a realistic sense of depth as in the Johnny Chung Lee demo.

Ha ha, still not any different from controlling a character to step forward. The ideal distance between the player and the TV is 4-6 feet any way.

There's just several things wrong with it. First, with the illusion of depth, your eyes will try to focus the way they normally do. It doesn't work right because you're staring at a 2D surface, and you're trying to look closer or farther away than where it really is.

Yes, one of the articles mentioned this, but with more practices and money pouring in, the vendors should be able to improve their techniques.

Second, you are going to move around, even if unintentionally, and when your perspective doesn't shift, your brain doesn't understand. I found myself sitting very still and telling myself to focus to the depth of where the screen was to keep the illusion working in Avatar, and I had to do that every couple of minutes after the first 15 or so.

They solve different problems. Instead of your head moving, you can control the game character moving. It's less tiring that way.

You're showing two different images to your eyes to trick your brain into perceiving perspective and depth. Yeah, all 3D representations on 2D surfaces are trickery, but stereoscopic trickery isn't consistently effective. For some people it just doesn't work. For some, it causes headaches and/or nausea. For some, the illusion wears off in time.

Sure, but they have to start somewhere. If the consumers, content providers and tech suppliers work together, I'm sure things can only improve. There are already existing 3D gaming consumers on the market, despite the expensive 3D glasses.

 

[Iron Tiger]

Sure, but they have to start somewhere. If the consumers, content providers and tech suppliers work together, I'm sure things can only improve. There are already existing 3D gaming consumers on the market, despite the expensive 3D glasses.

Bleeding-edgers. A very limited market of folks that do little to dictate future trends. Most of us end up wasting money on stuff that quickly loses support as the market disappears. I still don't know at this point if the 3D DLPs that have been shipping for over a year will be compatible with Blu-Ray 3D and console 3D. The 120Hz LCD monitors out today have inherent flaws that should not be present in widely-marketed CE products.

I added a 9800GT to my 5850 to do PhysX, because I'm a (foolish) bleeding-edger. I only have 2 games that it works with, and the effects have been less than impressive. I'll be the first to tell you that PhysX should either be drastically changed to a point where my card probably won't be able to handle it, or simply killed off entirely. When I get my 3D projector later this year, I'll be prepared to make that same evaluation, because I hope that whatever technology eventually becomes the standard will be widely accessible and of reasonable value for my family and friends to adopt.

 

[ergem]

Hi Beyond3d, I only registered to ask a question as I don't understand most of your posts here. Of the forums I go to, I think you guys are the only ones who can give sensible answer.

With the psp2 imminent from release now, or at least according to the latest rumor (Fall 2010).

Do you think it's feasible for Sony to use a 4" 3D (glassesless) OLED for the device during the launch financially and technically? Perhaps using this technology.

 

[Blazkowicz]

At least there's decoupling between 3D content, be it movies or real-time rendered, and the 3D display tech.
It was that way even a decade ago, with the nvidia stereo driver supporting the various techniques.

thus you shouldn't really suffer about compatibility problems. for non standard 3D DLPs, there's a workaround - some converter box. you don't get to suffer the physics situation - a DX11 card that can't run physx games, and a physx card that won't run DX Compute level 11 games.

 

[jlippo]

If you think about it, the very concept of animation/framerate (and lighting, and many other graphics techniques) is a trickery too. They approximate the real thing at reasonable cost. Some folks will find it hard to perceive 3D (headache, can't focus, hate weight of glasses, blah). I think the key point is head tracking and 3D are orthogonal. They should not be mixed up together. They don't replace each other. They may complement each other though.

In perfect world one would use the eye locations from the head tracking to render the stereo images, this way brain would get images as it would expect even tough one would tilt his head.
This would solve lot of the eye strain and images could be quite natural.

 

[MfA]

Do you think it's feasible for Sony to use a 4" 3D (glassesless) OLED for the device during the launch financially and technically? Perhaps using this technology.

Unlikely, it hurts resolution too much I think.

I could see them using an LCD barrier though (stereoscopy without glasses, but it only works in a sweetspot). There are already mobile phones with that kind of screen.

 

[Crossbar]

Do you think it's feasible for Sony to use a 4" 3D (glassesless) OLED for the device during the launch financially and technically? Perhaps using this technology.

Technically: yes.
Financially: probably no.

I salute your creative idea, but the screen would require so many pixels (read the technical pages of your link) that it would probably not fit the low cost market of the handhelds.

Edit: beaten by MfA

Edit: Been given it some more thinking, it does not have to go as far as the alioscopy screen with regard to pixels, for a handheld it may be sufficient to just double the number of pixels as a handheld unit does not require a wide view angle. It would be a really neat idea actually, given that there will be much more 3D content (movies, pictures) in the future.

 

[Arwin]

In perfect world one would use the eye locations from the head tracking to render the stereo images, this way brain would get images as it would expect even tough one would tilt his head.
This would solve lot of the eye strain and images could be quite natural.

Yes, this is what I was thinking too - the Sony demoes have shown that the eyes can be tracked. That means you can also track the distance they are apart and the distance they are from the screen, and maybe even what they are looking at approximately (that information can all be used to enhance the 3D effect, though obviously with more than one viewer that's impossible). That's not to say that I personally think it is necessary. I think having head-tracking in general combined with any kind of 3D motion sensor is already going to greatly enhance the awareness of 3D space. Your brain doesn't actually need that much information to be able to fill in the blanks. It's incredibly good at filling in blanks, in fact, sometimes even a little too good.

 

[sebbbi]

Yes, this is what I was thinking too - the Sony demoes have shown that the eyes can be tracked. That means you can also track the distance they are apart and the distance they are from the screen, and maybe even what they are looking at approximately (that information can all be used to enhance the 3D effect, though obviously with more than one viewer that's impossible). That's not to say that I personally think it is necessary. I think having head-tracking in general combined with any kind of 3D motion sensor is already going to greatly enhance the awareness of 3D space. Your brain doesn't actually need that much information to be able to fill in the blanks. It's incredibly good at filling in blanks, in fact, sometimes even a little too good.

Natal seems really promising, as it constructs the skeleton of the user. With the 3d skeleton you have both the head orientation and rotation (in 3d space). With a simple web camera it's much harder to get good quality head position and rotation. Stereo 3d with properly calculated two eye positions with head tracking data should look stunning. If Natal gets as popular as Microsoft predicts, I am sure we will see some Xbox games supporting stereo 3d + head tracking in the future.