Old Discussion Thread for all 3 motion controllers

[Shifty Geezer]

There's no real information on the capabilities or requirements of Natal that can be gleaned from this announcement, sadly. The PrimeSense technology link of Alucardx23 states

The PrimeSense SoC contains a highly parallel computational logic designed by PrimeSense that calculates the depth value of every pixel, based on the signal from a standard CMOS sensor. In addition, the PrimeSense SoC contains interfaces for a color CMOS sensor, integrated ADCs, USB with integrated USB PHY and a controller for running the firmware, thus providing flexible and upgradable system control. NITE translates raw visual data, such as depth and color, into meaningful application commands.

Thus the SoC doesn't perform any skeleton tracking; that's left for their NITE middleware, which will no doubt be replaced with MS's own middleware (otherwise they've wasted their time getting all that user data! ).

In fact if anything, this supports the view that processing is on XB360. The original design was a two chip solution according to these reports, which would include the PS1080 for image and depth data, and the second processor for turning that data into skeletal information. The reduction by half would be the removal of the processor to derive skeletal information while keeping the essential image capture chip.

 

[patsu]

I think providing depth data to the host processor has always been "free". It's the skeleton processing and motion tracking algorithm that takes CPU resources.

It's a little like saying sending audio data to the host processor is "free", but it's the speech recognition that consumes CPU and memory resources.

 

[Alucardx23]

There's no real information on the capabilities or requirements of Natal that can be gleaned from this announcement, sadly. The PrimeSense technology link of Alucardx23 states
Thus the SoC doesn't perform any skeleton tracking; that's left for their NITE middleware, which will no doubt be replaced with MS's own middleware (otherwise they've wasted their time getting all that user data! ).

In fact if anything, this supports the view that processing is on XB360. The original design was a two chip solution according to these reports, which would include the PS1080 for image and depth data, and the second processor for turning that data into skeletal information. The reduction by half would be the removal of the processor to derive skeletal information while keeping the essential image capture chip.

I see, too bad, but why make the announcement of the same thing then?

 

[Alucardx23]

Can it be that the original two chip design was now replaced by an all in one chip? So its cheaper to make. Tell me what you think.

Traditionally, gaming has been about player control agility. Gaming control has evolved over the years from the antiquated joystick to the novel Nintendo Wii remote, which uses accelerometers to capture player controls. The next step for gaming control is inevitable – full-body control with no wearable or hand-held items. Such natural interaction enables complete player immersion in the game – thus providing an unparalleled degree of freedom and creating a new gaming control paradigm.


Full-body-based gaming is based on a gaming console with a three-dimensional view of the playing grounds. While traditionally such a 3D view has been computationally expensive and therefore prohibitive to the consumer market, PrimeSense is now offering a consumer mass-market depth sensor that can provide full-body motion capturing on a significantly lower-resource host.


An interesting point of comparison between a depth-based gaming experience and a remote control based gaming experience is illustrated by a simple comparison to the Wii. The Wii remote includes two accelerometers, which can provide an accurate relational 3D location of two points along the Wii remote axis. In contrast, the PrimeSensor provides a VGA-sized depth image (640x480 pixels), where each pixel in the depth map is an independent 3D location. Comparing the two control paradigms, the PrimeSensor provides 307,200 points in space, whereas the Wii remote provides only two. The unbounded emerging gaming experience is straightforward from here.


A comparison of gaming control technologies is illustrated in the following figure. Until recently, games were controlled using a standard remote unit, which has existed for many years. The Wii introduced an accelerometer-based technology that enabled hand motion tracking. Today, PrimeSense’s low-cost depth acquisition technology enables full-body motion tracking.

http://www.primesense.com/category/gaming

 

[patsu]

While traditionally such a 3D view has been computationally expensive and therefore prohibitive to the consumer market, PrimeSense is now offering a consumer mass-market depth sensor that can provide full-body motion capturing on a significantly lower-resource host.

The keyword there is highlighted. Even with an EyeToy, the challenge is motion tracking and recognition, not capturing. Motion capturing is more of an input device's challenge, rather than the host processor's -- like a video camera capturing your movement.

What the host processor does with the captured data is the intensive part.

Would be best to wait for more technical reports.

 

[Alucardx23]

The keyword there is highlighted. Even with an EyeToy, the challenge is motion tracking and recognition, not capturing. Motion capturing is more of an input device's challenge, rather than the host processor's -- like a video camera capturing your movement.

What the host processor does with the captured data is the intensive part.

Would be best to wait for more technical reports.

PrimeSense’s low-cost depth acquisition technology enables full-body motion tracking.

http://www.primesense.com/category/gaming

 

[patsu]

PrimeSense’s low-cost depth acquisition technology enables full-body motion tracking.
http://www.primesense.com/category/gaming

Giving us depth info enables better tracking, but it does not mean the input device will do the tracking for us. PrimeSense's original dual-chip proposal does do all the work for us, but Natal's accessory may or may not.

It's futile to throw all those marketing material around. They don't have enough details. More technical reports please !

It would be accurate to say EyeToy enables full-body motion tracking too.

 

[Alucardx23]

Giving us depth info enables better tracking, but it does not mean the input device will do the tracking for us.

It's futile to throw all those marketing material around. They don't have enough details. More technical reports please !

You can say EyeToy enables full-body tracking too, and it's accurate.

The PrimeSensor is an end-to-end solution that enables a device to perceive the world in three dimensions and to translate these perceptions into a synchronized image, in the same way that humans do. The solution includes a sensor component, which observes the scene (users and their surroundings), and a perception component, or brain, which comprehends the user interaction within these surroundings.

The PrimeSensor is able to see and track user movements within the scene and provide the application layer with control widgets – a simple, clear API that translates user gestures or postures into known, deterministic application inputs. All activity is performed without any assumptions about the host, the user or the environment. No wearable equipment is required, making the solution practical, convenient, intuitive and easy to use. The sensor provides a natural interface to living-room devices (such as game consoles and set-top boxes), mobile devices and more.

PrimeSense is dedicated to bringing a novel and natural interaction solution to the mass consumer market. PrimeSense provides both a thin-host PrimeSensor depth acquisition device, as well as embedded middleware components that perform depth processing. To support a fast-growing market, PrimeSense has teamed up with depth-processing middleware partners and application providers to co develop a unique holistic solution. The benefits of this novel, collaborative API are clearly visible – enabling companies that adopt the natural interface to quickly capitalize on their investment by means of new applications, new experiences and new target audiences.

http://www.primesense.com/category/solution

 

[patsu]

PrimeSensor is not Natal though. The sensor component and the perception component could be the dual-chip set up. *If* the rumors are correct, it would mean MS dropped the perception component hardware and use 360 CPU instead.

 

[Alucardx23]

PrimeSensor is not Natal though. The sensor component and the perception component could be the dual-chip set up. *If* the rumors are correct, it would mean MS dropped the perception component and use 360 CPU instead.

That’s what I’m saying, the original Natal design included two chips, PrimeSense is promoting their tech as an all in one chip, but lets wait and see.

 

[Shifty Geezer]

In contrast, the PrimeSensor provides a VGA-sized depth image (640x480 pixels), where each pixel in the depth map is an independent 3D location. Comparing the two control paradigms, the PrimeSensor provides 307,200 points in space, whereas the Wii remote provides only two.

Marketing speak goes doolally. Out of those 307,200 3D points, in almost all games around about 307,000 will be irrelevant! A game only really cares for a few key points. The basis of Natal is deriving a 40ish-point skeleton from all those 3D points. And depending on the game, it may well be that only the hand positions and orientations are required, needing as much info as two positional controllers. eg. Table Tennis, your skeletal position doesn't really matter although it's nice to accurately map an avatar; all you really need if the bat position and orientation.

Of course, what the Prime Sense and MS skeleton tracking is doing is way awesome! It provides multiple 3D point tracking without needing controllers, meaning things liie head-tracking and body position can be integrated into a game. However, it shouldn't be sold on such silly figures as 'number of 3D samples.'

 

[patsu]

That’s what I’m saying, the original Natal design included two chips, PrimeSense is promoting their tech as an all in one chip, but lets wait and see.

Not sure where you're heading, but the PrimeSensor says nothing about skeleton processing, speech recognition, facial recognition, etc. Those are MS's licensed tech or original innovation, and will be done by the 360 CPU -- according to recent unofficial reports.

User gestures and what not can be done on the CPU based on the sensor data too.

 

[ShadowRunner]

MS themselves have said that the processing will be done on the CPU. Natal is based on the PrimeSense reference design, this does not mean it is the same thing. For instance the Specification of then reference design does not exactly match what we have heard of Natal.

Cost is only one reason they may have decided against an inbuilt chip for skeletal tracking. Performance may not have been up to where MS would want it, possibly only allowing them to track much fewer points for instance.

 

[Shifty Geezer]

That’s what I’m saying, the original Natal design included two chips, PrimeSense is promoting their tech as an all in one chip, but lets wait and see.

The clue's in your own links! Look at the diagram on the right, and the bullet-point description of their solution :

• PS1080 SoC - depth image, color image and audio acquisition engine (provided by PrimeSense) - Nothing about skeleton/point tracking
• PrimeSensor Reference Design - a consumer-market low-cost three-dimensional depth acquisition device (provided by PrimeSense)
• PrimeSensor NITE / Perception Middleware - a set of small-footprint embedded software libraries providing depth, color and audio processing and analysis capabilities, extracting and tracking users, gestures, features – resulting in a clear and deterministic application level API (provided by PrimeSense and Partners)
• Platform - a consumer market device hardware (provided by PrimeSense Partners)
• Application - a unique application that take advantage of the low computational cost depth acquisition and processing capabilities in order to bring novel user experience previously unseen (provided by PrimeSense Partners)

They provide a page about their middleware : http://www.primesense.com/category/nite_middleware

 

[Alucardx23]

Not sure where you're heading, but the PrimeSensor says nothing about skeleton processing, speech recognition, etc. Those are MS's licensed tech or original innovation, and will be done by the 360 CPU -- according to recent unofficial reports.

User gestures and what not can be done on the CPU based on the sensor data.

We where talking if the PrimeSensor would take care or not of motion tracking, which you where saying was the challenging part.

 

[patsu]

Nope. We are talking about Natal, not PrimeSensor. Why would I care about PrimeSensor ?

 

[Alucardx23]

The clue's in your own links! Look at the diagram on the right, and the bullet-point description of their solution :



[/LIST]
They provide a page about their middleware : http://www.primesense.com/category/nite_middleware

Ok so is just software based motion tracking.

 

[Alucardx23]

Nope. We are talking about Natal, not PrimeSensor. Why would I care about PrimeSensor ?

I suppose because Natal is based on their technology? But anyways it seems that the 360 CPU is going to do all the work.

 

[ShadowRunner]

Skeletal tracking is part of the PrimeSense platform, but not part of the PrimeSensor. The Skeletal tracking is done in software using the NITE middleware API which is what the skeletal tracking in natal is most likely based on. Now this could be done on the device itself if it had a processor for it, which was MS initial idea, but it is not part of the PrimeSence reference hardware design.

 

[Alucardx23]

MS themselves have said that the processing will be done on the CPU. Natal is based on the PrimeSense reference design, this does not mean it is the same thing. For instance the Specification of then reference design does not exactly match what we have heard of Natal.

Cost is only one reason they may have decided against an inbuilt chip for skeletal tracking. Performance may not have been up to where MS would want it, possibly only allowing them to track much fewer points for instance.

Yeah but remember that the design its still not final, I just would like to be sure if the Natal hardware that lets the 360 CPU do all the work is the final one or if the final hardware will help in some way.

This news shows that the Natal hardware will be better when it launches, but why I would like to know, I mean in what way will the final hardware will be improved, because it doesn’t seem like its just the software part that is improving.

"I did some boxing moves. I did a few little jabs and honestly it works. It picked them up. You could see my character in silhouette mimicking my moves," he said. "That's where it's going to be exciting."

Interestingly, Bradbury revealed that Microsoft had told him an improved version of the prototype is now ready.

"[Microsoft] did make a point that the next generation of prototype, which is now ready, which is not what we were using, we were using the E3 hardware. The new stuff is even more precise, the movements are more precise," he said.

http://www.videogamer.com/news/next-gen_natal_prototype_is_ready.html