Rumor: body sensing motion control for 360 coming

To me, the most important thing about that video clip, is it was posted over a year ago. So where's the tech now? We've seen similar from Sony for several years now, showing it work fantastically. If it works that well, why isn't it out in the general public? I'm guessing there are issues that stop it working effectively. eg. With the original EyeToy, IIRC the idea was much older but the technology wasn't there to work in low-light (living-room) conditions, so the had to wait for a suitable camera to be developed. In development it worked perfectly in brightly lit, showcase situations, but failed in the home. Likewise 'You're in the Movies' was demo'd working in an unrealistic setting. Eurogamer's review had them struggling to get it to operate.

Because none of this tech is new, and yet still isn't mainstream, it must be very hard to achieve a robust response - the reason why the EyeToy crew didn't go with background subtraction. What is this system going to do differently to overcome these issues?

 

Sounds like the thing Sony was talking about with the PS3Eye. Turns out they needed a LED waggle to be able to use it properly.

 

The company/technology now belongs to Microsoft:

http://www.theregister.co.uk/2009/04/10/microsoft_3dv_mou/

IMHO you will probably get to know how far it has advanced at this years E3.

As far as I know the camera uses various shades of white to black, along with other indicators, to determine your movements as opposed to what ever is in your back ground. I cannot find the link, but it some how uses the colors to track depth as well.

EDIT:

Found this:
http://www.3dvsystems.com/news/ZCam Release FINAL 12 2 07 (3).pdf

"ZCam™ offers real-time depth imaging at 60 frames per second, eliminating any perceived latency. The low CPU consumption of the unit means that gamers will not drain their systems of processing power. ZCam™ boasts the highest resolution on the market and since the camera works independent of lighting conditions, it will work flawlessly in any room, night or day."

 

That´s not the company in the video clip.

Anyway I am pretty excited about how it will work, it would allow for some pretty advanced eye-toy like games. For any kind of excersise game it sounds like a perfect fit.

 

3D camera is a really new fascinating technologie with new possibility
eyeToy/Pseye is calssic 2D camera

 

I'm a little pessimistic on the whole motion-thing. I guess I lack the imagination to really see how I can be sold on this.

It will take a pretty impressive demo to really get me to change my mind on the whole thing.

I was thinking it would be pretty cool to control the dashboard via a Minority report interface, but in the end it would be a fad for me where I'd probably go back to the controller.

 

That's the depth buffer, the whiter objects being closer to the camera.

Sure, it could be very cool, but it still faces some of the same issues like knowing which movements to follow and which to ignore. The rumour article says it doesn't track spectators for example.

Do you know what are the 2 additional sensors are alongside the camera (in the rumoured design)? IR lightsources is a guess. How does depth detection work with a single camera?

 

3D camera use revolutionary new sensor, every pixel sensor can measure depth value with "Time-of-flight" method use on light (camera emitting short pulses of infra-red light for the depth measure)
i think it's really a revolution in the sensor industry like Mems/Nems

 

That's very clever! You can see from the z-buffer clip that there are some clear optimizations that can be made. A simple filter of all values beyond a distance will focus solely on an area in front of the screen, eliminating the issues of optical background visual removal. Motion into the screen would also be measurable, allowing for punches and swings. Existing rotation tracking could be coupled with the depth feed too. And object occlusion is possible considerably aiding Augmented Reality. There's certainly potential. I still don't rate it's chances in this implementation though

 

Didnt people said the same about Wii?

 

Say what about Wii?

 

People said that there still has problems with Wiimote like tech, namely knowing which movements to follow and which to ignore, adverse pratical conditions (homes)...

At the end Wiimote works very well, we need to know more of the tech.

 

Demo of Boxing from CES 2008.

It looks pretty accurate for a simple demo they where showing at CES, at least as accurate as anything we have seen from the WiiMote. With more time and bigger developer support this seems to have as much potential as Wii Motion+ in terms of providing 1 to 1 motion for sports games.

Flight Simulator with hands as controller.

This demo is even more impressive, as you can see the system recognizes the position of his thumbs to fire machine guns or bombs. It seems to be about as accurate as Sixaxis in this video, but again its just a simple demo prepared by 3DV Systems to demonstrate the technology.

IMO, if this is in fact the company that Microsoft has purchased and the technology they intended to implement it seems to have amazing potential.

 

Ok i am sold, just give me a few triggers/joysticks anyway, I will miss those.

 

Thanks for the vid links.
I could see this being an addition to classic gaming with a controller. I don't think it would substitute for it though.

I wonder if this would work for a game like Rock band and GHWT where it would replace the components of the drum. Essentially just a shell and drumsticks that pick up the movements and translate it to inputs. For the guitar part, it would have to be really accurate to detect finger presses and strums.

 

Yeah, there is defiantly a limitation to how accurate the system is going to be. For things that require lightning quick reactions like Guitar Hero an actual Guitar peripheral with buttons is going to be ideal. I imagine it would also be hard to detect the minute movements of someones fingers even for a system like that. It would be amazing if it did work though, you could even imagine a flying game where you still use the regular 360 controller for shooting and having access to the buttons while the camera would monitor controller so it could be used in a SixAxis like manner for flying the plane.

If you take a look at the YouTube link I posted you'll notice there are a hole range of demos related to this system, ranging from Minority Report style control of Second Life and Media Center to gaming applications. It seems to be a pretty versatile system, especially if you could couple it with a traditional controller or even a motion controller/IR pointer.

 

Wow, I read the new about Ms buying 3dv sometime ago but I didn't realize how cool this was, damned this goes way further than just gaming. No matter this find an application in game/360 this will bring a revolution in the way media center edition of famous OS work.
Basically for non bureautic /extensive forum posting applications keyboards and mices could be send doing dust collecting duties.
Actually it turn me on way more than possible game applications.

 

Picosecond accurate distribution of timing signals and if you don't just put a counter on each pixel you also need very low noise (to use integration for time measurement). With the amount of engineering to get that to work I don't think consoles would be the first to expect these in. Hell I wouldn't be surprised if you could buy 10 consoles for the price of such a sensor if it came to market now.

PS. oh they say they have a fast shutter, that takes care of the timing distribution I guess ... still it's strange to see something like this being targeted at low end commodity products straight at introduction.

 

I never thought of using a combination of both the controller and the camera in that manner. I wonder if you could something similar to the video Graham linked.

I'm still skeptical, but I guess there could be some cool apps. The other video with browsing through the videos selection and then using a gesture to play the video was pretty cool though. I thought it was even better than using a touch screen. I'm still not sold on using it in-game...yet.

I have noticed that a lot of networks are using the large touch screens for both news and sports. It would be even cooler not even having to touch the screen.

 

It's possible that MS could be considering using it as a loss leader for moving more consoles and thus more games and peripherals. Whether that's financially viable we have no idea at the moment as without more information on what the system can do and whatnot, it's hard to come up with a ballpark figure for how much something like that would be to implement.

I have to say however, that I'll be disappointed if this doesn't also make it to the PC at the same/similar timeframe.

It would be interesting for use with an RTS also. And could bring mouse like RTS controls to X360 owners.

If they wanted, they could get really radical. Martial arts instruction, where the console could determine if you were doing things correctly. Sword fighting using any stick you could pick up (careful not to destroy anything in your room ).

I'm still far more interesting in UI integration though. Especially with regards to Multi-touch and Media Center... I get excited just thinking about those.

Still getting ahead of myself. I'll really be pleasantly surprised if MS can pull this off.

Regards,
SB