Project Natal: MS Full Body 3D Motion Detection

[Scott_Arm]

I just can't believe that there's even a discussion of problems with ethnicity or body types. You've got prototype hardware that's still running in debug mode with crashing and glitching, and we're trying to speculate whether an unknown time of flight camera running unknown software has issues with dark skin tones because one guy had trouble using it.

Oh yeah, and the whole issue was started by an IGN forum user. He suggested skin tone may be an issue because of a journal about using near infrared to monitor oxygen delivered to tissue via blood. Is that the basis of credible technical discussion?

 

[ShadowRunner]

Its no different than discussing any other product that is not final.

 

[AzBat]

According to Engadget & Jimmy Fallon's producer the jumpsuits were just for fun...

http://www.engadget.com/2009/06/11/jimmy-fallon-rocks-out-with-project-natal/

Tommy McClain

 

[Scott_Arm]

Its no different than discussing any other product that is not final.

If people want to talk about how the camera might work, and what the issues might be based on real information from people in industry, PR people or general information about 3D and RGB cameras, that's one thing. I don't think we need to make random guessing from IGN forum members a talking point.

 

[ShadowRunner]

Oh yeah, and the whole issue was started by an IGN forum user. He suggested skin tone may be an issue because of a journal about using near infrared to monitor oxygen delivered to tissue via blood. Is that the basis of credible technical discussion?

If the idea that skin colour could potentialy cause issues is not credible then we should be able to discuss the technical reasons for it being credible or not.

Ill say it once again, i dont believe it is an issue, but from a tehnical standpoint i wouldnt totaly dissmiss the idea without reason.

 

[Shifty Geezer]

You'd think being on a tech site, someone wouldn't have to hold you down to hammer the concept of "improvements during the development cycle" but for you and quite a hanful of others that seems to be the case.

Nowhere have I said they won't manage improvements. Of course they will. However as this is a tech site, we want to know why the problems exist and what solutions are available! And looking at the issues, there is a question here whether the issues lie with the software or hardware, and how much can be improved within the price limits of an affordable peripheral. Unless you are happy to take at face value that non-finalised technology exhibiting problems will be perfected within the year, when you can play Natal on your SED TV, you should be willing to question what can and can't be accomplished.

In case you forgot, MS claimed 'In The Movies' had working background removal. The end result didn't work, despite MS's claims for the technology. The EyeToy people expected it'd fail and gave their reasons for it. If you understood the technology before release, you'd know the steep challenge MS faced and slim likelihood for success. The same can be said of Wii, which didn't deliver what many hoped for, but a dissection of the underlying tech shows us it never could. Some of us are wanting the same here - to understand how this new tech works and how it will cope in real-world uses, instead of just taking at face-value the PR promises.

This is not to say MS will fail, or won't. It's a new technology to be treated as such, possibly amazing but also possibly buggy as is normal for new and unproven technologies.

 

[ShadowRunner]

If people want to talk about how the camera might work, and what the issues might be based on real information from people in industry, PR people or general information about 3D and RGB cameras, that's one thing. I don't think we need to make random guessing from IGN forum members a talking point.

A claim was made that it may not work in the dark. Technical reasons why it should work in the dark came along.

A claim was made that it could have issues with dark skin. With discussion technical reasons for why thats bullshit may come.

There is reason do discuss almost everything and if you feel like its a discussion you dont want to have then dont partake in it, but there is no reason to request others not to.

 

[patsu]

...The EyeToy people expected it'd fail and gave their reasons for it. ...

Ha Ha, on the contrary ! I didn't expect Natal to fail in Ngai's case. The IR solution should work in the dark. Period.

It's the boundary conditions I'm interested in (e.g., how far can the IR work ? how many people ? how fast ? etc.). The skin color is just something that came out of the blue. It may or may not change some of the parameters, but it's an interesting consideration nonetheless.

It's like saying an ultrasonic controller may be vulnerable to echo/reflection and interference. Doesn't mean the final product will be. Someone has to come up with a suitable countermeasure. That's engineering.... and this is Console Tech forum.



EDIT: In fact, if it's not the skin color, then I'd be curious what Ngai did (or did not do) to throw the controller off.

 

[Silent_Buddha]

EDIT: In fact, if it's not the skin color, then I'd be curious what Ngai did (or did not do) to throw the controller off.

There's one big HUGE glob of information that would help to clarify what may or may not be going on that we're missing.

Every demonstration of Natal is accompanied by the debug machines. One part of which is a display (that everyone at the demonstration can see but is off limits to cameras) that shows exactly what the motion tracker is tracking.

There has been no mention from any parties in the case of Natal failing to respond in Burnout (I haven't seen claims that it has failed in the Breakout demo) as to whether the debug machine showed that Natal was itself still tracking the person or not.

So at this point it's hard to know whether it's Natal that is messing up or something wonky with the Natal - Burnout interface bugging out.

Also if it's Natal borking. That debug display tracking the skeleton would display just what exactly isn't getting tracked properly.

Regards,
SB

 

[ShadowRunner]

There's one big HUGE glob of information that would help to clarify what may or may not be going on that we're missing.

Every demonstration of Natal is accompanied by the debug machines. One part of which is a display (that everyone at the demonstration can see but is off limits to cameras) that shows exactly what the motion tracker is tracking.

There has been no mention from any parties in the case of Natal failing to respond in Burnout (I haven't seen claims that it has failed in the Breakout demo) as to whether the debug machine showed that Natal was itself still tracking the person or not.

So at this point it's hard to know whether it's Natal that is messing up or something wonky with the Natal - Burnout interface bugging out.

Also if it's Natal borking. That debug display tracking the skeleton would display just what exactly isn't getting tracked properly.

Regards,
SB

Hmmm. It was stated that ngai and another person were swaping around and it worked flawlessly for the other guy but not ngai each time. It would suggest a problem with the tracking as natal should pass the data in a consistent manner as long as it is able to generate that data, regardless of who it is being tracked.

 

[MfA]

A quick google finds that dark skin both scatters near infrared more and absorbs it more, absorbtion only really starts below the epidermis though ... I would guess scattering in deeper layers isn't really relevant to the measurement, as such dark skin should actually increase near IR reflection. So it shouldn't make detection impossible.

If such small differences in reflection matter though it might have some problem with clothed parts of the body with the "wrong" clothes.

 

[MfA]

In case you forgot, MS claimed 'In The Movies' had working background removal. The end result didn't work, despite MS's claims for the technology. The EyeToy people expected it'd fail and gave their reasons for it.

Well at least this should work reliably with their new camera, it's early marketing was as a reliable alternative to green screen after all (you don't need a lot of depth precision for this either).

 

[ShadowRunner]

Have been investigating a little too. Accordin to this site:

"...the amount of light reflected from caucasian and negroid skin was very similar and that the nipples of a caucasian appeared pale while those of a Negro appeared very dark. Concluded that the melanin pigment in negroid nipples was so abundant and near to the surface that it absorbed infrared."

So yea if the camera uses the reflection of infrared light to determine depth then the difference in white and black skin shouldnt cause an issue.

 

[obonicus]

Have been investigating a little too. Accordin to this site:

"...the amount of light reflected from caucasian and negroid skin was very similar and that the nipples of a caucasian appeared pale while those of a Negro appeared very dark. Concluded that the melanin pigment in negroid nipples was so abundant and near to the surface that it absorbed infrared."

So yea if the camera uses the reflection of infrared light to determine depth then the difference in white and black skin shouldnt cause an issue.

Not unless it somehow involves nipples.

 

[ShadowRunner]

Not unless it somehow involves nipples.

Someone in japan just cried with dissapointment

 

[AzBat]

Can we abandon this inappropriate skin color discussion & get back to more important matters?

Tommy McClain

 

[Silent_Buddha]

Hmmm. It was stated that ngai and another person were swaping around and it worked flawlessly for the other guy but not ngai each time. It would suggest a problem with the tracking as natal should pass the data in a consistent manner as long as it is able to generate that data, regardless of who it is being tracked.

The point being that in none of this have I seen anyone mention...

1. Whether the debug display was still tracking things as it should.

2. Breakout not tracking.

It's only with regards to burnout that it isn't tracking properly. And apparently only steering.

So this would lead to questioning what about Natal in burnout would cause it not to track steering correctly or at all?

Perhaps something do with hand shapes? Hand sizes? How the person is holding their hands?

Perhaps the burnout implmentation is extremely sensitive to hand placement for steering? After all one article mentioned that Burnout would stop steering immediately as soon as one hand was removed from the driving position (IE - couldn't use one hand to steer for the demo).

All indications so far is that it has nothing to do with ethnicity, skin color, pigment color, sex, whatever. As it occasionally fails on caucasians as well as people of other skin color. Male or female doesn't appear to matter.

As Azbat said, can we drop the whole thing about skin color as that has nothing to do with what may be happening here. (BTW - this isn't related to me responding to ShadowRunner. )

Regards,
SB

 

[catisfit]

Well at least this should work reliably with their new camera, it's early marketing was as a reliable alternative to green screen after all (you don't need a lot of depth precision for this either).

No, but silhouettes are a big issue here.

 

[DudeMiester]

I remember researching about the ZCam technology a few years ago, which MS bought for this. I wonder if they will release this as a stand alone product for web cams and video conferencing. It would be a marvel for augmented reality versions of those, and other areas of research. It would seriously be a shame if they locked this technology up purely for the xbox.

 

[Scott_Arm]

I remember researching about the ZCam technology a few years ago, which MS bought for this. I wonder if they will release this as a stand alone product for web cams and video conferencing. It would be a marvel for augmented reality versions of those, and other areas of research. It would seriously be a shame if they locked this technology up purely for the xbox.

Agreed. I'd like to see it on Windows. Make it available for the XNA, so people could potentially make casual games for both 360 and Windows.