Video Motion Recognition

[Arwin]

It's more and more obvious that this is coming to next-gen consoles big-time, and that this is to be the main answer from both Microsoft and Sony to the success of the Wii.

Here's some new comments from Bill Gates:

http://www.gamesindustry.biz/content_page.php?aid=25524

Which are almost identical to what Loco Roco's designer Tsutomu Kouno mentioned in an interview with the same site earlier:

http://www.gamesindustry.biz/content_page.php?aid=25196

Will be interesting to see if E3 is going to show more on this from both parties.

 

[Shifty Geezer]

I did think Gate's comments rather playing catchup. He denies Wii let's people pick up and play with a raquet or bat, I guess because it's emulating a bat instead of being a real one? Yet in the video control game department they haven't done anything with Live! Camera and aren't talking about doing anything with Live! Camera, whereas Sony have a number of developments. I guess that is MS's modus operandum though!

 

[Asher]

To be fair, there is already a game on the Xbox 360 that uses the video camera as a control mechanism.

 

[Todd33]

To be fair, there is already a game on the Xbox 360 that uses the video camera as a control mechanism.

The Tiki ones? That's realllly bad, almost impossible to play. I think it's a function of the poor camera, so something like this might work better with the new Eyetoy thing which has nice specs.

 

[pc999]

So I would need to have a raquet and a basebol bat and a whatever in order to play a game. That dont seems very pratical to say the least.

Althought it could be cool for games like the boxe, dancing and such where you need no equipament.

 

[one]

So I would need to have a raquet and a basebol bat and a whatever in order to play a game. That dont seems very pratical to say the least.

Althought it could be cool for games like the boxe, dancing and such where you need no equipament.

You should have read the second link in the OP, the interviewed guy says EyeToy is better than Wii remote since it's required to hold nothing while playing.

 

[pc999]

You should have read the second link in the OP, the interviewed guy says EyeToy is better than Wii remote since it's required to hold nothing while playing.

I didnt noticed it , anyway really cool things for the future it seems, althought to raw at the moment.

Do you think that this can indeed be a reallity for the XB3/Ps4/Wii2 ?

 

[Arwin]

I didnt noticed it , anyway really cool things for the future it seems, althought to raw at the moment.

Do you think that this can indeed be a reallity for the XB3/Ps4/Wii2 ?

Who knows? I guess it will depend on how much of a success this will be in this generation.

 

[rekator]

To be fair, there is already a game on the Xbox 360 that uses the video camera as a control mechanism.

Yes, for some mini-games Rayman raving rabbids used the camera for motion recognition.

But I'm always thinking that for a real 3D motion recognition you need two cameras?

 

[deepbrown]

*YAWN* PS3 eye is designed to so this with a refresh rate of 120fps to catch all movement. And it can do more than understand movement, it knows the shape of your face...

 

[Arwin]

*YAWN* PS3 eye is designed to so this with a refresh rate of 120fps to catch all movement. And it can do more than understand movement, it knows the shape of your face...

There's more nuance in that - they mentioned 120fps / 320x240 or 60fps / 640x480, depeding on the application. Still, it's going to be interesting to see how far they can get. They certainly have the processing power, though how much of it they'll have to sacrifice to get good results is anyone's guess. Unless of course the little thing has some processing power of its own. But I doubt that? Though apparently the Eyetoy did (or else why would they mention how the Eyetoy detected movement vs how the PS3 eye detects it?

 

[Shifty Geezer]

*YAWN* PS3 eye is designed to so this with a refresh rate of 120fps to catch all movement. And it can do more than understand movement, it knows the shape of your face...

You understand the Eye knows nothing, right, and it's the software that can work out the face or anything else? The PS3ye is well thought out in providing a proper video interface, allowing both 120Hz fast update (less motion blur) and uncompressed footage (no compression artefacts to mess things up with). I can understand Bill's sentiments, but he doesn't seem to have backed them with any effort. Maybe there's more going on in the RnD side than we know of? There was that vid of a camera based interface using a finger-and-thumb guesture to create a control structure the camera could recognise. They might be putting in effort towards games. If they are though, they're being very coy about it. The Live! Vision was released wtih no noise at all about motion gameplay, and shipped with a by-all-accounts very weak title to showcase this gameplay. MS are definitely 3rd out of 3 in terms of moving controls forwards (or sideways, depending on how you look at it!)

 

[deepbrown]

You understand the Eye knows nothing, right, and it's the software that can work out the face or anything else? The PS3ye is well thought out in providing a proper video interface, allowing both 120Hz fast update (less motion blur) and uncompressed footage (no compression artefacts to mess things up with). I can understand Bill's sentiments, but he doesn't seem to have backed them with any effort. Maybe there's more going on in the RnD side than we know of? There was that vid of a camera based interface using a finger-and-thumb guesture to create a control structure the camera could recognise. They might be putting in effort towards games. If they are though, they're being very coy about it. The Live! Vision was released wtih no noise at all about motion gameplay, and shipped with a by-all-accounts very weak title to showcase this gameplay. MS are definitely 3rd out of 3 in terms of moving controls forwards (or sideways, depending on how you look at it!)

I know the PS3eye doesn't do it. But its a very good tool to get the info which the software will then be doing in the PS3. Eyedentify will be a mark of how accurate it is... Also you'll be controlling characters in new ways with the eye. Just watch...should go down to your fingers...

 

[Crossbar]

I wonder how hard it would be to detect some lip movements with the Playstation Eye or 360 camera and let the game map those movements to your avatar/character in your favourite MMO/FPS/etc. It would be pretty awesome when you are using voice chat. :smile:

Detection of movements of the eye brows and the jaw would be nice as well.

 

[betan]

I wonder how hard it would be to detect some lip movements with the Playstation Eye or 360 camera and let the game map those movements to your avatar/character in your favourite MMO/FPS/etc. It would be pretty awesome when you are using voice chat. :smile:

Detection of movements of the eye brows and the jaw would be nice as well.

Those are old tech already.
See Quickcam Orbit MP(Youtube video from Make)

 

[Crossbar]

Thanks!
That was exactly what I had in mind. :smile:

And that is achievable with with (link):
Technical Specifications

• Live video: up to 640 x 480 pixels
• Up to 30 frames per second live video with recommended system

System requirements

• Windows® 2000 or XP
• Pentium® 4 1.4 GHz or AMD Athlon® processor 1 GHz (Pentium® 4 2.4 GHz recommended**)
• 128 MB RAM (256 MB RAM recommended**)

Considering that IBM thinks the computational power of one single SPU is comparable to a 3.2 GHz Pentium 4 this should be achievable on the PS3 together with the PS Eye without the Cell breaking a sweat in addition to the normal game play.

 

[Titanio]

It's more and more obvious that this is coming to next-gen consoles big-time, and that this is to be the main answer from both Microsoft and Sony to the success of the Wii.

I'm not sure how much of an answer it is to anything given that work has been done in this area, in videogames, long before Wii emerged..

An answer to Wii might be making it standard in every system, likely as a complement to a physical controller. But that seems like a rather obvious path to tread, rather than something that would spawn as a reaction to Wii. Long before Wii was announced or shown there was talk of eyetoy being standard in PS3, for example, even to the point where Phil Harrison suggested that would be very likely in an interview, years and years ago.

In short, what I'm trying to say is, cameras as standard in the system of at least one of the players (Sony) seems to have been on the cards for a long time now, and that probably isn't something prompted by Wii. It's work that was already well underway, with a long roadmap seemingly planned out (e.g. Richard Mark's work with infrared demoed years ago, which still isn't even in a commercial Playstation product, but no doubt will be one day).

Having got that off my chest ( ), one thing I'm sort of interested in is how well something like Playstation eye might perform eye-tracking..specifically the latency between the eye moving in a particular direction, and the camera feeding that data to the application. I think eye-tracking might have some interesting potential in non-obvious ways..

 

[Shifty Geezer]

Eye tracking if done well could solve an age old problem in Soccer where you never get to swap to the player you want! If the game determines which player I'm looking at when I push the swap player button, it'll solve a host of problems.

 

[Titanio]

Eye tracking if done well could solve an age old problem in Soccer where you never get to swap to the player you want! If the game determines which player I'm looking at when I push the swap player button, it'll solve a host of problems.

Yeah, to generalise the problem, I'm interested in the feasibility of determining the area of focus on the screen. There'd be various parameters here presumably..distance from the screen (could be determined by the appearance of the eye's size?), size/dimensions of the screen etc. Perhaps it would only be possible with screens of a certain size also..

But if that could be done, it could open up some interesting possibilities, and not just in terms of game control.

It'd be interesting to see the state-of-the-art currently.

 

[Shifty Geezer]

It would be straight forward with the eye filling most of the capture frame. The problem here is the eye only being so many pixels long and wide. Here's a typical EyeToy scene

Even with PS3ye's higher resolution, you're talking 10 x 5 pixels for the eye. You could determine which of, say, 16 quadrants the eye is looking at, but nothing more accurate than that, unless they have some uber-tech I'm not aware of! Eye tracking would be a lot better in a PC setup, with a webcam set to fill the frame with your head.