Kinect technology thread

[(((interference)))]

Sitting with Kinect works with Forza 3...



http://www.neogaf.com/forum/showpost.php?p=22030301&postcount=586

Kudo Tsunoda confirmed as much on the Engadget Show. Also great behind the scenes tech preview. I'm sure a lot of it will hit YouTube.

Tommy McClain

Well, that's good, but clearly they're still ironing out some bugs - perhaps it doesn't work well with certain body types/ages or couches.

I wonder if MS will allow the implementation if it will not work for certain users, or will they gimp the experience for everybody?

And what about my earlier questions:

Do we know what kind of resolving power a 640x480 res camera can give at the minimum distance (1.8 metres).

Also, can Natal extract a person's image from the background and use it in games, ie. instead of the skinned avatars or other characters you currently see in the launch titles? EyeToy did this right? And that Xbox Vision Camera "You're in the movies" game (albeit badly)

 

[Graham]

And do we know what kind of resolving power a 640x480 res camera can give at the minimum distance (1.8 metres).

Also, can Natal extract a person's image from the background and use it in games, ie. instead of the skinned avatars or other characters you currently see in the launch titles? EyeToy did this right? And that Xbox Vision Camera "You're in the movies" game (albeit badly)

It will depend a lot on the field of view (which I expect is fairly high). However 640x480 should be adequate for most needs. For example, the PSEye is 640x480 and is used primarily for tracking the Move orbs.
If the FOV was 90 degrees horizontal, then 1.8m the pixels would be ~ 0.6cm. (I think )

And yes, a lot of games have shown some form of background subtraction using kinect. Your Shape and Konami's dance game comes to mind as being the most obvious (Although in this case it's still using a flat background).

 

[Silent_Buddha]

I was thinking about the Child of Eden demo that has been shown, and it struck me that the game uses hands/arms for aiming and shooting, and it almost seems as if the hands are opened/closed. You can see quite a bit of detail in the point cloud representation that is shown in 3D in the fitness game, and this got me wondering whether Child of Eden is perhaps using more detailed information from this point cloud to focus just on the hands, rather than using the default full body frame information that most games use? It looked interesting and natural for such a huge screen, but if you'd do the same thing in front of a small screen that would invite the user automatically to use much smaller movements, which could be a problem.

The point cloud may be splodgy, but it can be quite detailed, if you looked at that scene in the fitness game demonstration, where you could see the garment that was taken off and was pretty faithfully replicated all the way until it hit the ground.

It could be that since you can get an image from Natal, that either Natal has some form of image processing to isolate and remove players from the background, or they did it themselves. That would allow it to see a relatively easy open hand/closed hand.

If you notice his hands are almost always away from his body to the sides, IE - never overlaps that body that I can remember. It's also white gloves on a dark background making it even easier to do basic image processing/recognition. I wonder if the game will come with white gloves and suggest you play in the dark?

Natals ability to at least reproduce detail is pretty good. In the Your Shape demo, reporters have mentioned that it showed a clearly defined press card on their chest for example. I think the "blobiness" is deliberate on the part of UBIsoft rather than Natal.

Sitting with Kinect works with Forza 3...

http://www.neogaf.com/forum/showpost.php?p=22030301&postcount=586

Kudo Tsunoda confirmed as much on the Engadget Show. Also great behind the scenes tech preview. I'm sure a lot of it will hit YouTube.

Tommy McClain

I could swear that some media and celebrities tried Burnout sitting also way back when.

Anyway, Turn 10 seems to be experimenting with Natal a lot more than what has been shown by others so far. IE - moving around out of the "optimal" area. Head inches from camera. Sitting while playing... Or at least showing more.

As MS has stated though, I think they are still tweaking it, trying to catch all corner cases that might not work. And not approving it as a guaranteed feature until they nail down all or almost all corner cases.

Regards,
SB

 

[scently]

It could be that since you can get an image from Natal, that either Natal has some form of image processing to isolate and remove players from the background, or they did it themselves. That would allow it to see a relatively easy open hand/closed hand.

If you notice his hands are almost always away from his body to the sides, IE - never overlaps that body that I can remember. It's also white gloves on a dark background making it even easier to do basic image processing/recognition. I wonder if the game will come with white gloves and suggest you play in the dark?

Natals ability to at least reproduce detail is pretty good. In the Your Shape demo, reporters have mentioned that it showed a clearly defined press card on their chest for example. I think the "blobiness" is deliberate on the part of UBIsoft rather than Natal.



I could swear that some media and celebrities tried Burnout sitting also way back when.

Anyway, Turn 10 seems to be experimenting with Natal a lot more than what has been shown by others so far. IE - moving around out of the "optimal" area. Head inches from camera. Sitting while playing... Or at least showing more.

As MS has stated though, I think they are still tweaking it, trying to catch all corner cases that might not work. And not approving it as a guaranteed feature until they nail down all or almost all corner cases.

Regards,
SB

In an interview, they did state that they were using their own engine.

It seems most devs are using the standard sdk provided by MSFT but you can write your own engine it seems.

 

[(((interference)))]

It will depend a lot on the field of view (which I expect is fairly high). However 640x480 should be adequate for most needs. For example, the PSEye is primarily 640x480 and is used primarily for tracking the Move orbs.
If the FOV was 90 degrees horizontal, then 1.8m the pixels would be ~ 0.6cm. (I think )

And yes, a lot of games have shown some form of background subtraction using kinect. Your Shape and Konami's dance game comes to mind as being the most obvious (Although in this case it's still using a flat background).

Well, that would allow some form of finger tracking then, at least for big fingers. So I suppose you could have games where you stand closer to the TV and use your fingers, like that cancelled Finger Heroes game (probably the best Kinect concept yet)
http://www.joystiq.com/2010/01/20/arkedos-canceled-natal-game-2-finger-heroes/

At the very least it should allow the fingers to be tracked separately from the palm, allowing you to grab etc.

I'm not sure, does Kinect bowling track the opening and closing of your hands? or only the hand itself and not its configuration?

And with that dance game, he's just controlling some premade character right? it's not actually replicating a 3D version of him within the game.

That was my question, can Kinect do that? In the same way that Eyetoy does, except possibly 3D.

 

[onQ]

Why is the main motion control thread closed for no apparent reason?

And do we know what kind of resolving power a 640x480 res camera can give at the minimum distance (1.8 metres).

Also, can Natal extract a person's image from the background and use it in games, ie. instead of the skinned avatars or other characters you currently see in the launch titles? EyeToy did this right? And that Xbox Vision Camera "You're in the movies" game (albeit badly)

Dance Master does it
Mod: use 'yt' tags to with the Youtube code to embed...{yt}mUz3G8W7fxQ{/yt} only square brackets instead of braces.

 

[Shifty Geezer]

Background removal is a doddle with Kinect as you just remove anything beyond a certain distance. Although as we see, it isn't terrible accurate, but nothing is at this level. That kung-fu game on PSEye also has background removal done visually, and you can see a lot more noise in its results.

 

[Silent_Buddha]

I'm not sure, does Kinect bowling track the opening and closing of your hands? or only the hand itself and not its configuration?

It only tracks the endpoint of the arm going off to the side (simulating your arm moving in order to grab a ball). You automatically get a ball once that endpoint has gone a certain distance.

So far the vast majority of early Natal games are only using the skeleton tracking and ignoring the other capabilities of Natal. Presumably in order to get a game out in ~1 years time. 1 year of game developement doesn't leave a lot of time for experimenting.

Only a few of the games shown have used more than just the skeletal tracking. Your Shape with the character blob. Dance Central with the player image (wish it was more like the much better Harmonix one though). Perhaps Child of Eden with possible image processing to recognize open hand/close hand.

Can't recall any others. I'm expecting anything truly experimental is going to come in another year assuming Kinect does well and MS remains commited to it. As well any game with anything but basic levels and basic art design is going to take while to make for all the art and level assets. Kinectimals has some nice art design but it's rather limited in scope. Other Kinect games with more expansive games, and consequently more basic art design. Really 1 year just isn't a lot of time to do anything but fairly basic stuff.

An adventure game would be perhaps the most straightfoward type of game to not only include the image of the actual player, but also motion controls.

Regards,
SB

 

[AzBat]

Couldn't find the Engadget Show on YouTube. So, I went with the source...

http://www.engadget.com/2010/06/24/the-engadget-show-010-jimmy-fallon-kudo-tsunoda-microsoft-k/

Kinect Demo
32:50 Kinect Adventures - Obstacle Course - 2 player
35:15 Kinect Adventures - River Rush - 2 player
38:47 Kinectimals - 1 player
44:44 During Kinectimals demo the full body/sitting down question is answered
46:40 Kinect Joy Ride - 2 player
49:07 Skeleton tracking, 3D scanning, living room size determination
52:40 end

Enjoy. Not bad for 20mins.

BTW, the whole show was pretty good. Didn't realize I would like Jimmy Fallon or the host Josh Topolsky. Both are pretty funny for geeks. Anyway, I'll definitely watch future episodes of the Engadget Show.

Tommy McClain

EDIT: Found some skeleton tracking & 3D scanning pics at GAF...

 

[Silent_Buddha]

Interesting. And that picture shows that while finger tracking is going to be too much for the system, it "should" be able to easily tell open hand versus closed hand. Which would be better than hovering over a button to "press" it IMO. And I still think that's what is being used in Child of Eden with the hands being held such that they don't overlap with the body.

The caveat being that the software developer will probably be the ones having to implement it currently rather than Kinect telling the program a hand is closed or open.

Regards,
SB

 

[zed]

it "should" be able to easily tell open hand versus closed hand.

No, its not easy in fact its bloody hard.
Theres been a lot of study into this and whilst a kid of 4 years old and pick out eg pictures of houses with 100% accuracy, the state of the art software cant do it with 100% accuracy.
Time and time again in many threads Silent_Buddha (and others) youre making the same mistakes WRT kinect, by servery overestimating what todays state of the art AI is capable of.

 

[corduroygt]

Dance Master does it

Night of Fire = win

 

[Shifty Geezer]

No, its not easy in fact its bloody hard.

Open hand versus closed hand doesn't need AI image recogition. It only needs to see if there's a cloud of forward points around the end of the arm, or not, and that's straight-forward image analysis.

 

[scently]

No, its not easy in fact its bloody hard.
Theres been a lot of study into this and whilst a kid of 4 years old and pick out eg pictures of houses with 100% accuracy, the state of the art software cant do it with 100% accuracy.
Time and time again in many threads Silent_Buddha (and others) youre making the same mistakes WRT kinect, by servery overestimating what todays state of the art AI is capable of.

You do know that Child of Eden does exactly that.

 

[Silent_Buddha]

Not so sure anymore, it's hard to tell exactly what is triggering the shots...

From this video (http://xbox360.ign.com/dor/objects/14354640/eden/videos/e310_childofeden_spc_061610.html?show=hi ) we learn a bit about the controls but still not much.

Clapping your hands or bringing them together (it's the proximity of the end points not the sound or force of clapping) changes weapons.

Raising your arms (he does it with hands open, no clue if that means anything) releases a smartbomb.

Using either hand allows you to target items. It would have been cool if you could use both hands to target as the UI live demo's have shown no problem selecting with both hands at the same time. That would be awesome to be able to shoot two weapons at the same time completely independant on the other. That would be something I don't think has ever been done before...

To shoot (the "missle" weapon) he always pulls his hand to his chest with fist closed, then flings his arm out and simultaneously opens his hand.

At this point I'm not sure whether his opening and closing his hands is just a personal affectation, or if the open and closed hand is the trigger while the arm motion is for something else.

Man, I'm really wishing you could do two weapons independant of each other. But being multiplatform, I don't think Sony would approve a game that would require 2 move wands rather than just having an option for 2 move wands. Maybe they'll put in a challenge mode or something with that. Hell, that would be great for co-op with multiple people shooting.

Regards,
SB

 

[Arwin]

Man, I'm really wishing you could do two weapons independant of each other. But being multiplatform, I don't think Sony would approve a game that would require 2 move wands rather than just having an option for 2 move wands. Maybe they'll put in a challenge mode or something with that. Hell, that would be great for co-op with multiple people shooting.

Regards,
SB

Time Crisis 2 or 3 has a dual gun mode that works like this, it's pretty cool, but your field of view in a game isn't really big enough to make this as useful as you'd think. Co-op worked great though of course, though Time Crisis 3 had the two players sometimes go on different paths which required a split-screen mode (or linking up two PS2s of course ).

Anyway, I think it's great there is a game like this for Kinect. I do hope that if they have something like leaderboards (this game very well may not ) there are going to be separate leaderboards for Kinect and Analog stick ... that's definitely going to be more fun. It's great to have the option for games that require a lot of physical exercise. Of course you have to be careful (injuries, etc.) but burning adrenaline in your body is a great way to get yourself to feel not only fitter, but happier and more confident (adrenaline/dopamine is burnt in your muscles and changed into serotonine, the feel-good drug).

 

[zed]

Open hand versus closed hand doesn't need AI image recogition. It only needs to see if there's a cloud of forward points around the end of the arm, or not, and that's straight-forward image analysis.

Hold your hand out in front of you, make fists. (dont do it too long or youll strain yourself :smile
now imagine the camera looking at you with a res of 640x480 i.e. perhaps the hand is a block of 10x10pixels, i.e. not many. the problem is if you tilt your open hand slightly the amount of pixels is gonna go down, the AI will pick that up as making a fist. A 4 year old child would not make that mistake

 

[AlphaWolf]

Hold your hand out in front of you, make fists. (dont do it too long or youll strain yourself :smile
now imagine the camera looking at you with a res of 640x480 i.e. perhaps the hand is a block of 10x10pixels, i.e. not many. the problem is if you tilt your open hand slightly the amount of pixels is gonna go down, the AI will pick that up as making a fist. A 4 year old child would not make that mistake

10 pixels is plenty, it seems more than adequate for a font. You might actually want to try your own little experiment with a web camera, I just did. The appearance of an open hand to a closed one is markedly different even at a 10' distance.

There's also the option of simple props to improve recognition. If you had to wear something like a normal glove with orange dots on it or something.

I don't really expect this tech to be a replacement for the standard controllers anytime soon, but they've got to get it out there for people to see what is actually possible.

 

[onQ]

Kinect Technical Specifications & Xbox 360 Requirements Revealed ??

In anticipation of the upcoming release of Microsoft’s Xbox 360 full-body motion control system, Kinect, the publisher has revealed the technical specifications of the device. Speaking to a number of UK retailers, Electronic Theatre has managed to obtain the official console requirements and specifications of the hardware.

The most notable reveal would perhaps by the requirements of the Xbox 360 console itself. While Microsoft has insisted since the original unveiling of Kinect at the Electronic Entertainment Expo (E3) in 2009, then known as Project NATAL, that the device would be compatible with all Xbox 360 consoles, the revelation of a dedicated port included on the newly redesigned Xbox 360 model has sparked rumours of incompatibility, which were quickly debunked. However, Electronic Theatre can officially reveal today that the only requirement placed upon the console when using Kinect is the need for 175MB of storage for drivers and other software, and a single USB port.

The full technical specifications for the Kinect peripheral follow below, and Electronic Theatre will keep you updated with all the latest details on the forthcoming 360 full-body motion control system for Xbox 360.

Kinect Technical Specifications

Sensor
Colour and depth-sensing lenses
Voice microphone array
Tilt motor for sensor adjustment

Field of View
Horizontal field of view: 57 degrees
Vertical field of view: 43 degrees
Physical tilt range: ± 27 degrees
Depth sensor range: 1.2m - 3.5m

Data Streams
320x240 16-bit depth @ 30 frames/sec
640x480 32-bit colour@ 30 frames/sec
16-bit audio @ 16 kHz

Skeletal Tracking System
Tracks up to 6 people, including 2 active players
Tracks 20 joints per active player
Ability to map active players to Xbox LIVE Avatars

Audio System
Xbox LIVE party chat and in-game voice chat (requires Xbox LIVE Gold Membership)
Echo cancellation system enhances voice input
Speech recognition in multiple languages

NOTE: This information is based on specifications supplied by manufacturers and should be used for guidance only.

-END-

http://www.electronictheatre.co.uk/...ecifications-a-xbox-360-requirements-revealed

 

[Acert93]

If it is just a camera with no processing chip, why are there player limits on the camera?