Kinect technology thread

[bkilian]

Um, that's how it currently works, it's not an "improvement". We do update the models based on upcomiong games and data we get back from the field, but that video is describing the process we currently use.

 

[dragonelite]

So any words on when microsoft is releasing SDK for kinect.
Over two weeks i have to make a demonstration app for kinect for school.
Would be nice if they could release it fast and in C#.

 

[scently]

Um, that's how it currently works, it's not an "improvement". We do update the models based on upcomiong games and data we get back from the field, but that video is describing the process we currently use.

So the software runs on Xenos? Always thought that it runs on the CPU.

 

[Silent_Buddha]

So the software runs on Xenos? Always thought that it runs on the CPU.

It's been known for quite a while that it runs on Xenos. The massively parallel nature of GPU compute is an advantage for something like this.

Regards,
SB

 

[bkilian]

So any words on when microsoft is releasing SDK for kinect.
Over two weeks i have to make a demonstration app for kinect for school.
Would be nice if they could release it fast and in C#.

That's not owned by the Kinect team, it's owned by MSR, so I have no idea on their timetable, or even the featureset. I don't think it'll make your deadline though. I suspect you could find a reasonable substitute, and it wouldn't be a huge amount of work to create a .NET compatible API.

 

[LightHeaven]

Looks like MS have improved the body part recognition algorithms...



http://www.gamasutra.com/view/news/...d_For_Identifying_Player_Bodies_On_Kinect.php

Gamasutra reference this PDF & this website & video...

http://www.i-programmer.info/news/1...e/2176-kinects-ai-breakthrough-explained.html

Tommy McClain

Good now maybe the lag will be reduced further. Really good news.

I don`t know... Looks about the same of what is now to me XD

Ms should allow all users to use kinect viewer, i'm sure that would erradicate a lot of disbelieves and misconceptions about the accuracy and response of the tech. Plus, the tool was made to collect data, so having everyone as a potential tester could help improving it even more.

 

[dragonelite]

Kinda do want something like this.

 

[Nesh]

I want that

 

[Silent_Buddha]

Kinda do want something like this.

Haha, very cute and clever. Amazing what people can do with Kinect.

Regards,
SB

 

[dragonelite]

Wow got my hands on a kinect for this school period project.
You can do a lot of cool things with it.

Tried doing some of the sample stuff in OpenNi and some cool drawing programs where you can change stuff on the fly.
Changed some code in the drawing program you had to draw with your head and to reset your canvas to draw on you had to do a high kick. The faces you could see when people looked at us was just pure gold some students and teachers even ask if they could try. Options are as good as limitless for none gaming stuff if you ask me only downsize is that for Core gamers the input latency is just to high for a Fps or other twitch shooter but for story telling games like Heavy with a this could be really immersive experience. Instead of press X for jason you can just shout jason.

Stupid person in our group doesn't want to make a Air guitar app not fair.

 

[Shifty Geezer]

That'd be amazingly immersive. I'd have thought VR would be making a comeback in arcades, as these sorts of headsets are still too costly for home use.

 

[Silent_Buddha]

Wow got my hands on a kinect for this school period project.
You can do a lot of cool things with it.

Tried doing some of the sample stuff in OpenNi and some cool drawing programs where you can change stuff on the fly.
Changed some code in the drawing program you had to draw with your head and to reset your canvas to draw on you had to do a high kick. The faces you could see when people looked at us was just pure gold some students and teachers even ask if they could try. Options are as good as limitless for none gaming stuff if you ask me only downsize is that for Core gamers the input latency is just to high for a Fps or other twitch shooter but for story telling games like Heavy with a this could be really immersive experience. Instead of press X for jason you can just shout jason.

Stupid person in our group doesn't want to make a Air guitar app not fair.

Pretty cool that it's starting to be used in school classes for learning. You'll have to keep us updated on what you guys come up with.

Regards,
SB

 

[(((interference)))]

If it only takes 5 ms of GPU time, why is there so much lag? Shouldn't there only be 5-10 ms more lag than an equivalent 30 fps title? ie. if the game is just doing 1;1 limb tracking and not gesture recognition.

 

[warb]

Some games have more lag than others. Part of it could be to do with how lag is represented. With Kinect you can track it through the whole motion, even if it's minimal.

 

[ShadowRunner]

If it only takes 5 ms of GPU time, why is there so much lag? Shouldn't there only be 5-10 ms more lag than an equivalent 30 fps title? ie. if the game is just doing 1;1 limb tracking and not gesture recognition.

5ms GPU time for processing the image is not the only area where latency can occur. For example it must take an amount of time for the camera to capture and package the picture and be sent down the wire and stored in RAM etc before the data is even ready to be processed by the GPU. If i remember correctly even the latency of a single button press on DualShock 3 is around 20ms and kinect is doing a whole lot more.

 

[bkilian]

If it only takes 5 ms of GPU time, why is there so much lag? Shouldn't there only be 5-10 ms more lag than an equivalent 30 fps title? ie. if the game is just doing 1;1 limb tracking and not gesture recognition.

2ms I believe (according to the docs posted here recently). But that is just classifying the image. From there, you have to match a skeleton to the identified parts, and then, in some games, map an avatar to the skeleton.
Other causes of latency are the buffer transfer, the depth image creation on the camera, and the rendering method you're using.

 

[dragonelite]

Pretty cool that it's starting to be used in school classes for learning. You'll have to keep us updated on what you guys come up with.

Regards,
SB

As it now stand we are probably gonna make a 3D rubiks cube program which will use head and hand tracking. I actually wanted to make something like Forza 4 showroom but then for a every kind of model something let say a architect could use. Maybe i will make it myself in my spare time working in xna trying to figure out 3D graphics.

 

[(((interference)))]

2ms I believe (according to the docs posted here recently). But that is just classifying the image. From there, you have to match a skeleton to the identified parts, and then, in some games, map an avatar to the skeleton.
Other causes of latency are the buffer transfer, the depth image creation on the camera, and the rendering method you're using.

Oh, so the GPU body part identification time is down to 2 ms now, that's very quick.

5ms GPU time for processing the image is not the only area where latency can occur. For example it must take an amount of time for the camera to capture and package the picture and be sent down the wire and stored in RAM etc before the data is even ready to be processed by the GPU. If i remember correctly even the latency of a single button press on DualShock 3 is around 20ms and kinect is doing a whole lot more.

Yes, but even with the 20 ms of controller lag the best responding 30fps games have 100 ms of input lag, for Kinect titles the best we've seen is something like 150 ms. That's a huge difference, especially if it only takes 2ms to identify the body parts (which you'd think would be the most difficult task to do).

 

[LightHeaven]

Oh, so the GPU body part identification time is down to 2 ms now, that's very quick.



Yes, but even with the 20 ms of controller lag the best responding 30fps games have 100 ms of input lag, for Kinect titles the best we've seen is something like 150 ms. That's a huge difference, especially if it only takes 2ms to identify the body parts (which you'd think would be the most difficult task to do).

The increased lag are probably because of two things:

- To smooth the animation;
- They need to make sure you actually make the gesture they are looking for... It would be very error prone if they estimate that based on how you are moving at some point . They need to see you moving first to react after, that also induces lag. Some games, even though they use body tracking seems to add extra lag to smooth out the animation data... The measures i tried taking for Kinect Adventures for instance, showed that the delay is longer to your avatar start a movement than it is for it to stop (Let's say, they start moving at 5 frames or so behind, but stop the motion at fewer frames behind.

Also, if you look at the kinect viewer, which shows the raw data from kinect (including the body tracking of course) you'll notice that its very close if not a perfect match to the target 100ms, but games seems to add a few more ms of delay due to some processing that has to be done on top of that data. Some games like dance central and your shape do feel extremely close to the raw output from kinect, thought... So i guess that not having to map an avatar frees up some ms.

 

[Arwin]

Yes, but even with the 20 ms of controller lag the best responding 30fps games have 100 ms of input lag, for Kinect titles the best we've seen is something like 150 ms. That's a huge difference, especially if it only takes 2ms to identify the body parts (which you'd think would be the most difficult task to do).

Controller lag is typically 4ms (at least that is what it is for the DualShock 3 on PS3, but I reckon the 360 controller will be identical). It is definitely not 20ms, because I remember this from the Move lag discussion, which is typically also 4ms, but with extreme circumstances and tracking 4 Move controllers at once it could go up to 20ms.

One reason for a difference is that afaik the Kinect camera looks at the input data with 30fps, so that is more of a delay right from the start. From there on everything depends on how much the application wants to do and in what way it wants to do it for how much lag there will be. DF did some good testing to show the differences between various games and applications back in the day. There definitely is room for improvement even for the best performing titles, and it will probably be time for Richard to pull out his kit to retest some stuff if we get confirmation of improvements in the drivers.

It definitely speaks to the 360s GPU that it can do this stuff, no question about it - impressive job.