Project Natal: MS Full Body 3D Motion Detection

[Shifty Geezer]

Professional talk is indeed a load easier to parse than the common vernacular tongue from people who can't readily express themselves. If you really listen to speech, you notice how much people repeat themselves, umm and err, get the wrong word etc., we us humans can work with, but which will slay an AI parser. Heck, we still have trouble matching spoken sounds to known keywords with absolute certainty (but then humans have that trouble too with some accents!).

That isn't what Natal's offering though, so it's a tangential discussion to this one. Voice recognition as a tech can have it's own thread if people want, in the general algorithms department.

 

[patsu]

I think MS is lumping voice/speech recognition and image recognition into Project Natal. Otherwise, there wouldn't be any context in showing the Milo experiment.

 

[Scott_Arm]

"Compatible with any Xbox 360 system, the "Project Natal" sensor is the world's first to combine an RGB camera, depth sensor, multi-array microphone and custom processor running proprietary software all in one device."

http://gizmodo.com/5274319/xbox-360...tion-control-one+ups-the-wii?skyline=true&s=x

 

[patsu]

Is it still sub $100 ? I think the original price from 3DV camera was a simpler config.

 

[Arwin]

I'd imagine so, but the state of the art speech recognition won't be able to do what Milo did in the concept video. The automated help desk software works because their context focuses on a limited vocabulary.

I can't really agree with that. Voice recognition is a constantly evolving technology and keeps on improving. Besides, in the case of Milo, there's different things at play as well - Molyneux discussed how Milo actually does a lot of 'faking', and listens to intonation (which is much easier to recognise than words) for a lot of stuff. So if you make a joke, for instance (an example demoed live to and by Eurogamer), Milo will chuckle even if it doesn't understand the joke, just because it recognises an intonation suitable to telling a joke. It's a clever fake, and while simplistic, the effect is nonetheless pretty unnerving according to Eurogamer.

By the way, is the depth-sensor infrared? I'm just wondering how it will deal with low-light circumstances, and the depth-sensor could help there.

 

[patsu]

I can't really agree with that. Voice recognition is a constantly evolving technology and keeps on improving. Besides, in the case of Milo, there's different things at play as well - Molyneux discussed how Milo actually does a lot of 'faking', and listens to intonation (which is much easier to recognise than words) for a lot of stuff. So if you make a joke, for instance (an example demoed live to and by Eurogamer), Milo will chuckle even if it doesn't understand the joke, just because it recognises an intonation suitable to telling a joke. It's a clever fake, and while simplistic, the effect is nonetheless pretty unnerving according to Eurogamer.

Yes, I mentioned that above. But I was refering to the concept video where he carry a full conversation with a woman and ask appropriate questions based on what the woman said. He still sounds a little like Eliza (for the speech forming part).

If it's based on grunts and tones, then yes it's similar to EyePet, and it's not "speech" recognition per se.

 

[Laa-Yosh]

Do you think that this may also help to down the development costs as it could be used to do mo-cap for cheap?

Don't think so. We're having to clean up and sometimes fully reanimate data we get from a relatively good and expensive Vicon system... Natal would produce very jerky motion, particularly if you'd like to do stuff like turning away from the camera.

 

[Scott_Arm]

I'm curious to know how much work the processor does to offload the 360 CPU, and how much memory there might be on the device rather than using the 360s memory.

What use is the multi array microphone over a single microphone? I'm assuming it means a bank of microphones, but what is the purpose of that?

 

[liolio]

Don't think so. We're having to clean up and sometimes fully reanimate data we get from a relatively good and expensive Vicon system... Natal would produce very jerky motion, particularly if you'd like to do stuff like turning away from the camera.

Thanks for the insight, do you think that Ms could dare to implement this in its xna tools to helps amateur/indies developper pushing better xbla/community games? I mean it might not be good enough for big budget games but still could be better than nothing for low budget products.

 

[patsu]

What use is the multi array microphone over a single microphone? I'm assuming it means a bank of microphones, but what is the purpose of that?

Probably similar to the competition: "multi-directional voice location tracking, echo cancellation, and background noise suppression."
The software part needs to be complete though. e.g., PS Eye lacks echo cancellation for voice chat.

 

[tha_con]

They did ...

Their "on stage demo" was a bit...questionable.

Especially the Milo stuff. I'd be willing to wager a large majority of everything happening in the Milo video (which was not live) was staged and / or controlled somehow.

I think it was Eurogamer who said the back stage event with Milo was awkward, because a developer was "manipulating milo" while people talked to it. So I wonder how much of it is strings, and how much is smoke and mirorrs.

 

[Silent_Buddha]

I wouldn't rule out a Christmas 09 release for this considering how well it appears to be working. The kicker will be if there's a good enough app for it. And in that case there's at least one or two devs that have multiple months of experience with it. Lionhead and it's probably a given that Rare has had access to it if for no other reason than UI integration.

Timing would be good for a release at the end of this year also as it could tie in well with the Fall NXE update.

So it's quite possible that either (or both) Lionhead or Rare have a game in the works already. Considering the close relations they have with MS, it's quite possible they would have had notice quite a while back of what MS's intentions were with regards to this and had time to start developement even if they didn't have hardware until 3 months ago.

It's more likely to hit in 2010 I think, but it's certainly a possibility that it will hit this Christmas. As next Christmas is a looong ways away.

And I'll just have to keep my fingers crosses that they'll allow this to work with Windows 7 at the very least. I just get all giddy inside when I think of using multi-touch with Windows 7 from my couch.

Regards,
SB

 

[DrJay24]

I wouldn't rule out a Christmas 09 release for this considering how well it appears to be working.

How well is that? The video with the family playing various games and interacting with the UI seemed entirely a proof of concept mock up, nothing looked real time or interactive.

Are you referring to the chick on stage spazzing around to the dodge ball game? They could have done that with their old camera, it looked like Totem Ball or whatever it was called.

 

[Shifty Geezer]

I wouldn't rule out a Christmas 09 release for this considering how well it appears to be working.

Appears being the operative word If if was a robust system, surely MS would have it out in the wild for everyone to try. After all that proved very successful with EyeToy. The only reason to show it by invite behind closed doors is to demo it in a more controlled environment. I don't think the software is reliable yet. They may be aiming for a Christmas release and may get things up and running by then (that'd be the ideal time to launch after all!) but I'm pretty sure the current build is ropey. Plus they are only demoing concepts, not actually products. There's no actual full game to try with it. My guess is this is an earlier than usual showing from MS to 'win' E3, teal the limelight, draw attention to the their platform and away from the competition, and perhaps also to frighten the others saying 'we own this tech, and you guys aren't going to be able to match us!' Leaving the unveil to E3'10 wouldn't be advantageous if that'd be more accurate. Especially when rumours were flying thick and fast!

 

[Tap In]

But you're missing the point. .

no i'm not. I think a driving game with Project Natal would not be appealing to me ..... but to some people who would never pick up a controller guess what? it is.

Also I'm more excited that IF it works that well (sensing subtle foot movements for acceleration and braking and subtle hand movements to allow control of a car.... the possibilities are endless.

 

[DrJay24]

no i'm not. I think a driving game with Project Natal would not be appealing to me ..... but to some people who would never pick up a controller guess what? it is.

Are we talking about old people playing game in their retirment homes? How do I change gears, hit the brakes, hand brake, move the camera? I guess we are talking about incredibly over simplified games, otherwise you need a controller.

 

[Tap In]

How do I change gears, hit the brakes, hand brake, move the camera? I guess we are talking about incredibly over simplified games, otherwise you need a controller.

from the hands on I posted which you can read yourself above.

instead of using a joystick, you steer by holding your hands up in the air like you're gripping a steering wheel. To hit the gas, you move your foot forward along the floor. To brake, you move it back. To trigger the turbo boost, you do a gear-shifting, fist-pumping movement with your right arm. Awesome.

Are we talking about old people playing game in their retirment homes? .

there are a substantial number or people (a huge untapped segment of the adult population) who think controllers are toys and are for kids and or are intimidated and would be open to "virtual reality" type experiences.

The fact that you think that they are only retirement home people explains why you are not in marketing for a huge software company on the verge of tapping that market with the expansion of a very good idea into another level of actuality.

 

[liolio]

OT
I wonder if this thread should stay in the tech section as we are speaking about random stuff from technic to howx to market the thing.
/OT
More on topic I think it will prove impossible to have this ready for this fall Shifty has a point on this.
The system should launch during 2010, fall 2010 sound like a good idea along with a slim system, but Ms may want to ship this asap and fill the slow months of the year (end of winter early spring?).
Actually if the hardware is closed to be finalized pushing games for it should not take that much time.

 

[Silent_Buddha]

How well is that? The video with the family playing various games and interacting with the UI seemed entirely a proof of concept mock up, nothing looked real time or interactive.

Are you referring to the chick on stage spazzing around to the dodge ball game? They could have done that with their old camera, it looked like Totem Ball or whatever it was called.

No I'm talking about all the hands on reports that we're getting from various journalists.

Regards,
SB

 

[Silent_Buddha]

Appears being the operative word If if was a robust system, surely MS would have it out in the wild for everyone to try. After all that proved very successful with EyeToy. The only reason to show it by invite behind closed doors is to demo it in a more controlled environment. I don't think the software is reliable yet. They may be aiming for a Christmas release and may get things up and running by then (that'd be the ideal time to launch after all!) but I'm pretty sure the current build is ropey. Plus they are only demoing concepts, not actually products. There's no actual full game to try with it. My guess is this is an earlier than usual showing from MS to 'win' E3, teal the limelight, draw attention to the their platform and away from the competition, and perhaps also to frighten the others saying 'we own this tech, and you guys aren't going to be able to match us!' Leaving the unveil to E3'10 wouldn't be advantageous if that'd be more accurate. Especially when rumours were flying thick and fast!

I'd agree with the lack of an actual public hand-on is due to there not being a demo-able game at this time. If you announce something this big, do you really want to demo it to the public at large on a game that's been out for a while? Much better to demo it to journalists behind closed doors where you can better explain the potential. And then show them the potential with apparently flawless and more interestingly--precise controls patched into an already existing game. A game type that is both friendly to analog control and can benefit from increased precision.

Likewise no hard announcement of a Christmas launch (other than the clever name) is probably due to questions about whether a game that may (or may not) be in developement will be finished in time for a launch. IE - better to surprise launch than to announce something this big and then postpone it.

That's why I said it's most likely a 2010 launch but wouldn't be surprised if it was launched this Christmas.

Regards,
SB