Kinect technology thread

From what i can see it is only using the x/y position of the users hand rather than the orientation of the users arm. Its a subtle difference but i would imagine it would have a significant impact, i cant imagine using move in MAG for instance if it was only using the x/y possition of the ball for pointing. Kinect seems to use a similar control method to the menu selection in Beat Sketchers, works well in that setting.

 

Seems, but apparently it's 3D still for Beat Sketcher (e.g. the spray can will have a wider range with lower density if you move forward or backward).

Anyway, yeah, the hand tracking for pointing purposes seems decent. However, I really hate the hold to select - if they could replace that with an option to snap your fingers or something instead, that would be much better for me personally, I'd be able to go through the menus in a snap (or two/three) literally!

 

Oh, yes, you're quite possibly right and that's not terribly exciting. Although it seems to be working across a small area of the screen, which would be a very low resolution for positioning. Can anyone test this, maybe by keeping their hand in the same place and moving their arm left and right behind it, changing the pointing angle but not the displacement?

 

This sounds like he's saying a highly probability based, branch heavy piece of code is better suited to GPU than general purpose CPU?

 

It sounds like he's saying it's "a highly, highly parallelisable operation" and that "GPUs are machines designed for these types of operations".

 

Thinking about it, i dont think tracking trajectory from the whole arm would be much different actually. If the elbow is in a fixed position you will be moving your hand in x/y anyway to control the pointer, so they would just use the x/y of the hand and apply some form of acceleration for the same effect. For laser style pointing like Move/Wii it is all in the wrist. Child of Eden is a good example, looking at that it could very well be using just x/y of hand or the orientation of the whole forearm but the result would be much the same to the point that you cant tell what its doing as its so similar.

 

XD The heroes are all back stage !

 

That's not how I understand it. Creating those probability values is massively parallel number crunching, comparing large dataset for similarities, but the decision making, the final branches, can be handled on CPU. I would like to know what exactly they're doing on GPU, but I doubt we'll ever hear that! I also want to know how much local-machine or 'cloud' learning is actually going on. The YouTube vid a few posts above shows Kinect throwing a wobbly over someone's legs occluded by a short camera tripod. Is this something it'll later adapt to? How can it without some degree of correction? Experience based learning needs to have decisions as marked as successful or not, otherwise the system doesn't know to change. In this case, Kinect is placing the leg bent to the left when the user's legs are straight down. What is going to tell Kinect that was the wrong choice, and next time keep the guy's two legs below him?

One area his comparison with a human brain falls down, and a pet hate of mine with computer people speaking up their AI, is the computer lacks any comprehension and can only do comparisons with data. It's still fundamentally a conventional data analysis system, only with more computations and more reference data. Unless they have modelled self-evaluation and self-correction, its capacity to learn and adapt won't be as great as Kipman has described.

 

Well, keep in mind that article is also with regards to children. Depending on the child some jump fairly high (a few inches) while quite a few barely get one inch off the ground when jumping in place. I'd imagine that could get tricky, especially if the camera is mounted above a large screen TV.

Regards,
SB

 

In today's kinect session I attended, they showed some dev tech demos where they had a kind of picture-in-picture app showing what the various cameras saw and you could select more. The sensor was on the table but this was a classroom table, not living room so it was waist-high I'd guess. The cameras have a reasonably wide FOV, wider than I expected at least and do pickup the floor. One thing they did have to do was lower the window blinds because it wasn't facing the (strong) sunlight coming through the windows.

But I don't think you really need the floor. Just track the body's displacement relative to the joints. If the body shifts upwards without the knees being bent before you can assume the person jumped. I'm only guessing, they didn't go into much detail on how exactly the software works but the various cameras and algorithmic passes I saw tell me they have plenty of processing involved. It's not just a simple contrast-based, silhuette, edge-finder. The real-time skeletal tracking was particularly impressive.

 

I'm not sure about the floor. I think it was mentioned somewhere that Kinect has to see the floor for perspective or positioning or something.

 

So could you make a kinect with two ps eyes (one with the IR filter removed) and an IR projector, since the rest is in the software?

 

The physical hardware is the least complex part of Kinect. So, in theory anyone could easily replicate the hardware. Getting that built would, IMO, get you about 5% of the way to a working Kinect module. In contrast, Move is probably quite a bit more complex on the hardware front, but I'd be willing to bet the software still makes up greater than 50% of the complexity of the whole thing.

Regards,
SB

 

Is writing software for the IR camera much more complicated than image recognition software many companies (incl. Sony) already have? It seems to be an easier problem since both the color space and the resolution are smaller.

 

The 3d skeleton tracking is. Unlike face recognition with it's easy targets, finding the hands and legs when crossed or in front of another bit of body is very hard. Kinect represents several years work on this very task alone, and it still has issues. No-one but a boy-wonder genius in a Hollywood film, who interfaces with comptuers solely by typing and never using a mouse, will be able to get a home-made depth camera up and running with full skeleton tracking without considerable amount of effort or time; unless MS are plain lying about the effort they went to!

 

Heh, it may be possible for someone to do as a university research study for their PHD if you constrain it to a set height, limb size, weight, etc. Things I believe have been done to limited extent prior to MS embarking on this process, which was mentioned in the big Kinect writeup. But again the key there was that it was extremely limted to either a set plane (distance), set area, set body type, set range of motions, etc. And that tracking was easily lost and could only be re-aquired by standing in a certain spot with a certain body stance for a set amount of time.

The beauty of MS's solution isn't that it can do skeleton tracking (although that in itself is pretty significant) but that it can do it with a variety of body shapes, heights, etc all at varying distances. All the while easily tracking multiple skeletons and seemlessly adding and subtracting people as they step into and out of the camera FOV. And never once do you have to stand in a certain place, or with a certain body position in order for Kinect to start tracking you. Well, except when it glitches out with certain people. It'll be interesting to see how rarely or how often it glitches once millions of people have access to it.

As was mentioned in one of those Kinect pieces. It must have been absolutely amazing the first time they had some random Joe step in front of Kinect and it automatically started tracking their skeleton.

Regards,
SB

 

Yep, the software seems to the be the key to this. Being able to tell the position in a 3D volume without using an accessory, like the wiimote or move, and having only a single stationary "sensor" is a hard problem to solve. Do that for more than one person, allowing them to "switch places", bump into one another, gracefully handling exiting and re-entering line of sight, other people walking in front, all that in real-time, etc. It's mighty impressive.

What the cameras themselves showed didn't look out of the ordinary but I was very intrigued when they begun showing the same cameras after doing algorithmic passes and the new information they were extracting.

 

after seeing this http://kotaku.com/5681314/see-kinects-thousands-of-points-of-invisible-light
can someone provide an ir image from the location roughly similar to the ir cam on kinect? preferably in 640x480 + HD/Full HD res. It would be nice to see a raw image from the ir cam + whether a higher res cam can get a lot more usefull point cloud or not.

 

http://www.gamasutra.com/view/news/...l_Launch_Amid_Celebration_Questions_Alike.php

Looks like it performs Exactly like I predicted when the device was announced, but this is not to say it wont succeed.
fitness + party games are enuf (look at the wii, apart from a few first party games thats all it had!) as these dont really need to read your movements accurately.
$500million marketing at launch guarantees a huge launch, But at this stage its too hard to see how it will pan out in 6 months time.

 

That's a very cool video!