Kinect technology thread

[Silent_Buddha]

Well, that's kinda cheating. The skeleton tracking has lag. Ubisoft have abandoned MS's Kinect skeleton tracking in favour of their own low-latency solution, which defeats in part the wonderfulness of Kinect. how much of their low-latency solution is dependent on the depth information, and how much could be achieved with a basic camera? If the later, Kinect ends up being overkill for this title.

Well, if they are ignoring/not using the depth camera then surely we'll see a PS3 version (with PS3 eye) that is identical to the X360 version with regards to tracking and control. After all UBIsoft isn't shy about releasing multi-platform titles.

I have a feeling the depth camera is playing its part in this whether or not they are using the standard Microsoft tracking doodads. At the most basic, you can use it to quickly isolate what needs to be tracked which probably cuts down on computation required if you were to attempt to isolate what needs to be tracked from a full color image of everything the camera can see. And then there's always the lighting problem if you were to go purely with the full color video stream.

Regards,
SB

 

[Shifty Geezer]

I have a feeling the depth camera is playing its part in this whether or not they are using the standard Microsoft tracking doodads. At the most basic, you can use it to quickly isolate what needs to be tracked which probably cuts down on computation required if you were to attempt to isolate what needs to be tracked from a full color image of everything the camera can see. And then there's always the lighting problem if you were to go purely with the full color video stream.

Yes, that's my thinking, as if the depth becomes used just for object isolation. Which is a good idea in part, but it also suffers from 'lighting' issues! Kung-Fu Live is showing some impressive background removal producing a silhouette of similar quality, although it's going to be much more susceptible to lighting issues. Still, chances are Move could achieve very similar, bringing a broader range of titles to the platform, and being one string to Kinect's bow that doesn't remain unique. I had thought such fitness titles would be simplified on Move without depth-based skeleton tracking.

 

[scently]

The ubisoft title does not have to use the skeletal tracking but could use just the depth information alone.

From my understanding this cannot be done with the pseye or the eyetoy, although in union with the move it should be able to judge the player's depth but I don't think that that capability can be compared to the depth information from kinect.

 

[AzBat]

A nice video about natal on engadget show.

http://www.engadget.com/2010/06/24/the-engadget-show-010-jimmy-fallon-kudo-tsunoda-microsoft-k/

Here's a snip from my post about that show...

32:50 Kinect Adventures - Obstacle Course - 2 player
35:15 Kinect Adventures - River Rush - 2 player
38:47 Kinectimals - 1 player
44:44 During Kinectimals demo the full body/sitting down question is answered
46:40 Kinect Joy Ride - 2 player
49:07 Skeleton tracking, 3D scanning, living room size determination
52:40 end

Tommy McClain

 

[AzBat]

Personally I don't think 1 title not using the skeletal libs is any indication of anything. Especially for a 1st generation launch title. The software libs will get better and devs will have better tools to make better games. Now had all launch titles dumped the skeletal libs and went the Your Shape route, then you would have a legitimate gripe. However for now, it's just one title. Also realize that Kinect is more than just a depth sensor, it's also an RGB camera and a microphone array. So not every launch title is going to use every technical feature in a compelling way. The question you need to ask do all the launch titles as a whole bring a unique experience afforded by its technical capabilities? For the initial target audience this holiday, I think that's a yes. We'll see how it progresses as the developers improve the tools.

Tommy McClain

 

[Alucardx23]

How kinect looks inside.

Article
http://www.t3.com/feature/exclusive-how-does-microsoft-xbox-kinect-work?

More pictures
http://www.t3.com/features-gallery?articleId=17763

 

[AzBat]

Great find! Love the pics.

That's a lot of components in there. Explains the higher price of the add-on. Here's one thing that explains why it was so wide...

The problem facing the microphone subsystem is that it needs to be sensitive to voices up to 10 feet away, while being able to ignore ambient noises and any sounds other than your voice. To solve this problem, the Microsoft lab went to 250 homes with 16 microphones and took a host of recordings from different setups, determining in the end the very best mic positioning.


The end result is an array of four downward-facing (so that the front of Kinect stays clean and grill-free) mics, spaced one on the left and three on the right. In fact, this specific microphone placement is the only reason why Kinect is as wide as it is.

Tommy McClain

 

[Alucardx23]

Great find! Love the pics.

That's a lot of components in there. Explains the higher price of the add-on. Here's one thing that explains why it was so wide...

Tommy McClain

Yeah it doesn’t look cheap, some new stuff from the article.

"What about that promo trailer where Kinect signs in players just by looking at them? We saw that work in real life. The reality is that you’ll need to go through an ‘enrolment’ process in for that to happen. It’s a short one, but works by mixing your skeletal measurements with some basic facial recognition software. Microsoft says that if you drastically change your appearance you’ll need to reenrol."
​

"The onboard processing unit cancels out noise that it determines is coming from your beefy 5.1 surround system, while a software system called ‘Beam Forming’ works with the camera to work out where you are to create an envelope of sound around you. This hammers in the sound of your voice and ignores your friends or family members either side of you."

"The motor also operates the camera’s zoom function, which allows it to expand the play space. The example we were shown was somebody joining a video chat by walking in behind the sofa, at which point the picture zoomed out to compensate and allow them into the frame. There is also a fan that only kicks in when needed, so as not to interfere with the microphones."

 

[AzBat]

Here are pics of them testing the camera in the lab...

Tommy McClain

 

[AzBat]

More pics, video & details from Gizmodo...

http://gizmodo.com/5604308/deep-inside-xbox-360-kinect-the-interface-of-microsofts-dreams

 

[Alucardx23]

Kinect looks bigger than usual in that picture.

 

[AzBat]

I'm more focused on the two Kinect prototypes we aren't allowed to photograph, one that looks like the head of EVA from Wall-E, a palm-sized bean shape with two antennae shooting out of the side. It was probably rejected for being too personable. The second looks a lot like the current Kinect, but more Apple-like, a glossy black center wrapped in a kind of brushed aluminum.

I would have loved to seen those.

Tommy McClain

 

[Silent_Buddha]

Interesting it sounds like the depth camera is in control of the automatic zoom function of the color camera. So if the depth camera detects another person in it's FOV it'll zoom out the color camera in order to include them.

That also means the color camera isn't used in anyway for skeletal tracking. I had been wondering if it played any part in it.

Going back to the above discussion on the UBIsoft games, it would then appear that only the depth camera is used. Well other than the cutout of the player from the video feed.

Kind of cool how they use the depth camera to control the noise cancellation algorithms allowing for a more targetted region of allowed sound.

Regards,
SB

 

[scently]

Interesting articles. Hopefully this will stop all the silly comparison with the Eyetoy or PSEye because this is soo much more.

 

[Rotmm]

Hopefully this will stop all the silly comparison with the eyetoy...

No chance

Interesting reading though.

 

[Rangers]

Wow Kinect looks almost low tech on the inside. Not sure that's good or bad, but after reading the article about all the expense developing it and seeing the pics, $150 seems a lot more reasonable now.

So it has three cameras inside? I see 3.

 

[Graham]

Ubisoft have abandoned MS's Kinect skeleton tracking in favour of their own low-latency solution, which defeats in part the wonderfulness of Kinect. how much of their low-latency solution is dependent on the depth information, and how much could be achieved with a basic camera? If the later, Kinect ends up being overkill for this title.

If you watch the ubi videos, the tracking only seems to kick in when the character is mostly stationary, and seems to deal with specific limbs/joints. Also, given they know exactly how the player should be standing (matching the on screen avatar) I imagine they simply don't need to the fully dynamic solution Microsoft provides.

In my view, it's a good example of the systems' flexibility. With a large set of optional secondary controls, it's nice to see companies already using it in their own way.

 

[goonergaz]

Interesting articles. Hopefully this will stop all the silly comparison with the Eyetoy or PSEye because this is soo much more.

I must confess to being guilty - clearly there is a lot more tech (altho I still think the price is way too high) - the problem is, until it grabs me and says 'see I'm not a slightly better eyetoy!' then I'm still not 'getting it'.

 

[Arwin]

Wow Kinect looks almost low tech on the inside. Not sure that's good or bad, but after reading the article about all the expense developing it and seeing the pics, $150 seems a lot more reasonable now.

So it has three cameras inside? I see 3.

I thought the reverse - with the tilt motor that is active a lot more than I thought, the three boards, the two cameras and the infrared grid projector (this is your third 'camera'), the cooling fan, as well as the primesense licences, SDK development costs, and the various other preventive IP purchases, I now don't believe we'll see a price for the standalone unit below $99 anytime soon.

 

[Alucardx23]

Another interview with Blitz Games Studios, chief technical officer, Andrew Oliver. He talks about using the GPU for Kinect image processing.

What kinds of resources are these additional software algorithms taking up on the Xbox 360 hardware?

Well that's interesting, because obviously if you're trying to run your game and look at these huge depth buffers and colour buffers, that's a lot of processing. And it's actually processing that a general CPU is not very good at. So you can seriously loses half your processing if you were to do it that way. We've found that it's all down to shaders, but turning a depth buffer into a skeleton is pretty hardcore shader programming. What you tend to do is write all your algorithms, get it all working in C++ code, and then work out how to now write that in shaders.

By shaders you mean that it's running on the GPU?

Exactly. The GPU on the Xbox is very powerful but we've all only been using it for glossy special effects. A really good example of this is Kinectimals, as the most intensive thing that you can do on a GPU is fur rendering. So that GPU is doing all the fur rendering, and I can guarantee that it's also doing a lot of image processing too. It's brilliant that the Xbox has a really good GPU and can handle both these things, but actually writing that shader code to do image analysis is hardcore coding at its extreme!

http://www.next-gen.biz/features/interview-andrew-oliver