Project Natal: MS Full Body 3D Motion Detection

[Silent_Buddha]

there are a substantial number or people (a huge untapped segment of the adult population) who think controllers are toys and are for kids and or are intimidated and would be open to "virtual reality" type experiences.

The fact that you think that they are only retirement home people explains why you are not in marketing for a huge software company on the verge of tapping that market with the expansion of a very good idea into another level of actuality.

Heck imagine a fishing game where someone can use their own fishing rod that they use when they go fishing.

Or a tennis game where you can use your own tennis raquet.

Or for driving games mounting your own custom racing steering wheel on something and using that.

I wonder if this system can also roughly measure acceleration and velocity?

Also, while it's great for casuals, losing stuff like force feeback and rumble might be a turn off for more traditional gamers.

And, geez, if they can somehow get Japanese devs interested in this, imagine the things they could come up with.

LoL, just imagine boob physics combined with 3 dimensional control (not just the boring 2D bouncing you could do with a regular camera). Not that MS in a million years would give approval for such a game to be sold.

Regards,
SB

 

[MfA]

What use is the multi array microphone over a single microphone? I'm assuming it means a bank of microphones, but what is the purpose of that?

It makes isolating signals easier, each signal has a different delay to a given microphone which together with statistical models of the signals at issue (voices) allows all kinds of signal processing wizardry.

 

[RobertR1]

I wonder if this system can also roughly measure acceleration and velocity?

Regards,
SB

I believe this is possible and discussed when the lady was doing the demo with the balls coming at her. The force with which she swung and kicked back was translated in the movements of the balls she came into contact with.

 

[TheChefO]

If done properly it brings so much possibilities for casual games/activities, attraction I think more and more that Ms should brand this on its own.

Exactly.

 

[AzBat]

3DV Systems has completed its sale to a third party in the neighborhood of $7million...

http://www.foxbusiness.com/story/ma.../elron-group-company-dv-completes-asset-sale/

Just doesn't say which one.

Tommy McClain

 

[Crossbar]

It makes isolating signals easier, each signal has a different delay to a given microphone which together with statistical models of the signals at issue (voices) allows all kinds of signal processing wizardry.

Yep it´s very useful for filtering out sound sources. They may also be used to identify the direction of a sound source.
The sony waggle patent uses ultra sonic sound sources as one part of it´s function to identify the position of the controller i 3d space.

The more I have been thinking about, I expect that Microsoft to add additional peripherals with buttons (waggles) to cover the game space that the Wii covers today (and possibly Sony, we'll know later tonight).

The main reason to why I expect that is the size of the camera bar, it wouldn´t need to be that wide for the kind of stuff it contains.

If you playing with multiple wands the identification (positioning of each controller) could work in a similiar way as the Game Trak Freedom.

What sets the Freedom apart from other motion controllers, however, is its use of ultrasonic 3D positioning. The Freedom system includes two sensor bars placed or mounted to the sides of a display, which communicate with an ultrasonic emitter built into the Freedom controller. The ultrasonic pulses emitted by the controller and identified by the sensors, along with the data fed by the accelerometers enable the system to calculate the distance, orientation, and speed of the device with extreme precision. Each Freedom remote (up to four can be used simultaneously on one system) emits three pulses to accurately triangulate distance, and a fourth pulse is emitted for accuracy.

Now you may say that is a load of redundant positioning stuff that can be taken care of by the depth sensor of Natal. I am not so sure about that. Let´s take a golf simulator game as an example. The pitch of the club is absolutely vital to where the ball will be going, the depth simulator cannot measure that with any precision. The wand described above is much better equipped for that. The Natal camera bar may very well already be prepared with sensors for that kind of wand, hence the size of it.

 

[fearsomepirate]

I'd agree with the lack of an actual public hand-on is due to there not being a demo-able game at this time.

Which suggests MS isn't really particularly serious about this. When Nintendo debuted the Wii Remote, they already had 3rd parties in the loop and were showing it on the E3 floor. MS has announced neither games nor a release date for this. The company has virtually mastered the vaporware (or the over-promise and under-deliver) counterattack, and I suspect that's what this is. Watch for this to get delayed several times before either quietly disappearing or getting released as a slightly fancier version of the eyetoy with a handful of gimmick games.

Also, I find it amusing that MS is promoting this with virtually the exact same language Reggie used at E3 2006,

 

[22psi]

Kinda too early to jump to conclusion right?

MS is showing this behind closed doors to audiences, where they can talk and interact with Milo.

 

[zed]

Their "on stage demo" was a bit...questionable.

a bit!!
download the video + step through it
theres parts where the detection fails to pick up a kick or a whole arm movement etc
and yet ppl are asking 'will it register finger movements'

 

[obonicus]

Kinda too early to jump to conclusion right?

MS is showing this behind closed doors to audiences, where they can talk and interact with Milo.

Well, not really. Kudo did let out that studios were getting their Natal dev kits yesterday (or were being shipped them yesterday, anyway). Clearly MS' internal studios have been working on it longer, but we don't have any software announcements from them, either, which of course doesn't mean there's nothing. But at the same time, Keighley said sources have pointed to Fall 2010 at the earliest.

 

[obonicus]

a bit!!
download the video + step through it
theres parts where the detection fails to pick up a kick or a whole arm movement etc
and yet ppl are asking 'will it register finger movements'

Well, considering everything else was essentially concept renders, it's quite possible that we didn't really see the real thing -- E3's all about the spectacle above the truth(look back to the KZ2 and Motorstorm 'in-game footage'). It's not like your typical viewer can try to superimpose (and rotate etc.) the images and time them out.

 

[Acert93]

The ball demo looked cool and like a good workout. IMO this really needs some sort of wand so it can be utilized in exhisting genres and allow the best of both worlds. Minimally I look forward to a FM3 patch in 2010 that offers cabin view head tracking. A very simple feature but a much needed one!

 

[Brad Grenz]

Yeah, a PSMC wand would make it much more appealing. But then MS would have to eat their "invisible technology" rhetoric.

 

[patsu]

A "no controller" solution has its charm. If MS can make the detection precise (e.g., tracking finger movement), it will have its own application areas !

My sense is MS is keen to explore music games (e.g., using bare hands to play regular instruments), FPS (Halo, Gears), casual games (revolutionalize Wii), media applications, and educational titles at least.

 

[Crossbar]

A "no controller" solution has its charm. If MS can make the detection precise (e.g., tracking finger movement), it will have its own application areas !

It sure has it´s charm just like the current Eyetoy games, but of course more advanced since there is a depth component.

But precise detection of finger movement seems like a pipe dream (crazy high resolution, crazy amount of computing) and even if they could, how much would you need to pull your finger to actually trigger the "button", how could they with any precision sense how much you twist your hand when smacking your virtual racket to give the virtual ball the intended virtual top spin?

I standby with my previous post where i predict MS has prepared wand-compatibility from the start. I agree with Joshuas point about a needed wand as well.

From a business point it probably makes sense, the camera is likely quite expensive while the wands are dirt cheap to make. Sold separetly the wand sales may help subsidise the camera.

(notices that the Sony dick-wand has turned real, making a compatible wand even more likely for the 360)

 

[V3]

Eyetoy detect hands movement alright I thought.

Can Natal detect fingers movement ? You can do guitar heroes without the controller ? That would be cool, kinda like air guitar that people do.

Sony solution can utilised rings to detect fingers, assuming that bulb can be made small enough.

 

[Crossbar]

Eyetoy detect hands movement alright I thought.

Can Natal detect fingers movement ? You can do guitar heroes without the controller ? That would be cool, kinda like air guitar that people do.

Sony solution can utilised rings to detect fingers, assuming that bulb can be made small enough.

The resolution of the PlaystationEye will be the limiting factor. The Eye resolution is 640x480 at 60 fps, I think the USB-connection is a limiting factor since it also works at 120 fps at 320x240.

I doubt very much that Natal has a higher resolution than 640x480 of the depth detector. Possibly 320x240 or even lower detecting just the rough shape of the body and limbs, still with a very high accuracy of the Z component. Extrapolarization of a complete body shold be pretty easy from that.

The RGB camera of Natal can of course be of a higher resolution. As the raw data is processed by a local CPU in the camera there is no need to transport the raw data to the 360 cpu, instead the video stream can be transfered in a compressed format without satuarating the bandwidth of the USB-channel.

 

[MfA]

If they use 3DVs technology then the image sensor is bog standard (and thus cheap even at higher resolutions). Using a higher resolution one only means you need brighter IR leds.

 

[Crossbar]

If they use 3DVs technology then the image sensor is bog standard (and thus cheap even at higher resolutions). Using a higher resolution one only means you need brighter IR leds.

Really? The nano-second shutter doesn´t really sound like that.

3DV's core technologies are in the 3D system level, in nano-second imaging-quality shutters based on GaAs or Silicon, as well as in extremely fast and tightly controlled illumination. The technology performs superior depth imaging (depth resolution of millimeters) in real-time (60 fps or more), using little or no CPU.

 

[MfA]

The shutter isn't ... but that simply sits in front of the image sensor, it doesn't put a limit on resolution.