Sony's New Motion Controller

[patsu]

Quaz51, link doesn't work.

 

[Quaz51]

sorry

 

[Brad Grenz]

I think as silent budha, Ms states that Natal could works with some "pre natal" games, the games can be patched to handle the new input but it's unlikely have the cpu ressource do deal with the amount of data generated by Natal.

a photo of Roku Player:

If Ms want to make the most of their take they have to free as much as they can the 360 cpu.
Natal may end with quiet some work on its hand, all together:
generate 3d squeleton and track its motion
2D shape recognition
sound processing
I would be surprised if the system pack an ARM/MIPS tiny/cheap tied to some kind of DSP + tiny amount of ram and rom.

Yeah, it sounds to me like the hardware will either be very expensive for them to produce, or the performance hit on the 360's CPU when using natal could be significant. The PSMS will certainly use host based processing, but it will be stuff the SPEs excel at and far less complex given known quantities provided by the wand.

 

[Love_In_Rio]

Something is moving...

Patent from 05-14-2009 :

http://www.faqs.org/patents/app/20090122146

 

[patsu]

I took a very quick glance. The image capture device is a 3D camera, like the technology in Quaz51's post ?

 

[Love_In_Rio]

After all they knew what was Microsoft working on. They will try to add the 3d object tracking.

 

[patsu]

Well... they had a tech demo as early as 2004.

 

[Love_In_Rio]

One interesting note: they are using 4 leds arranged in a square/rectangular shape to measure the tilt depending on the modification of the square/rectangular shape.
In the example the leds would be between L1 and R1 buttons ( suppossing a breakable controller design i suppose ).

 

[Shifty Geezer]

Well as to CPU power who's to say. But conventional speech recognition still requires a fair bit of CPU power to recognize and track just one person speaking.

Mobile phones have word recognition for years. You need to do some funky maths to get numerical data for waveforms that lets you do comparisons, but it's DSP type stuff which Cell is very well suited for.

The Natal goes one bit further in that it can recognize and keep track of multiple speakers presumably speaking at the same time. And not only that adds in background noise removal and presumably echo cancellation so that you can use it instead of a headset for voice chat while gaming. I wish the PC had something similar.

A quick Google.
http://www.andreaelectronics.com/Buy/ProductDesc/Superbeam_Array.htm

This was released in 2002.

Then additionally it has to record and process 2 distinctly different video streams and then integrate them into something that can represent what it see's in 3 dimensions.

The video stream thing is a webcam's abilities. It just sends a honking great bitmap to the console. (note:more on this later...)

And not only that the claim is that it can recognize and differentiate between multiple people. As well tracking items/people/apendages that may pass out of sight of the camera due to being occluded due the movement of other people or the objects.

When you've mapped the skeleton, following out-of-sight limbs comes with the system with no extra overhead.

And on top of all of the above it also has to be able to recognize facial features (enough to distinguish one person from another) as well as track changes to those facials features (smiling/frowning). And likewise do it for each and every person in the field of view.

Again, mobile phones and cheap cameras have this tech.

I dunno, maybe all of that takes virtually no memory or CPU power. But it at least appears as if it would take some non-trivial amount of processing power and memory. Memory BTW - that both consoles are fairly short on. Considering to do something similar with the Eye Toy + Wands the PS3 would have to do all the work as well as use system memory.

Well, there'll definitely be some processing overhead on PS3, but it may well have enough capacity to swallow it up without any impact on the game. A lot depends on what exactly Natal is doing. Having seen the report on the box size, there's more in there than I guessed, and there may well be more processing going on then I expected, and onboard RAM etc. We don't know what data is being passed to the console. Is it just a couple of framebuffers? Is it accompanied with skeleton data? Is it multiple audio streams or just one feed of Fourier tranformed audio data? At this point we're guessing!

So there will be an pverhead on PS3. However, the more stuff inside Natal, the more expensive it's going to be. If it's a little technology marvel, it could cost too much to be mainstream. PS3's advantage over that is that, though the tech is simpler and will take more assets from the PS3, it'll be cheaper and more readily adoptable. Also the PS3 will only need use as many resources as needed. If the camera is being used for voice detection, there's no video overhead, whereas Natal has to be designed to supply the peak use at all times.

Certainly regards the PS3 solution, it should be able to comfortably handle all the processing requirements. At least, I'll be very surprised if it isn't. Worst case, resources are used meaning comparable titles are weaker looking on PS3 with less stuff on screen etc. but it'll still provide a good experience and at a low cost. For Natal, we can't being to guess what overhead there is or isn't on XB360. Actually we can guess - none, because Burnout was working with it

 

[Nesh]

Well as to CPU power who's to say. But conventional speech recognition still requires a fair bit of CPU power to recognize and track just one person speaking.

The Natal goes one bit further in that it can recognize and keep track of multiple speakers presumably speaking at the same time. And not only that adds in background noise removal and presumably echo cancellation so that you can use it instead of a headset for voice chat while gaming. I wish the PC had something similar. Using Teamspeak without a headset is begging for some horrendous background bleed that's enough to make your ears bleed as well as everyone else in the Teamspeak channel.

Then additionally it has to record and process 2 distinctly different video streams and then integrate them into something that can represent what it see's in 3 dimensions. And not only that the claim is that it can recognize and differentiate between multiple people. As well tracking items/people/apendages that may pass out of sight of the camera due to being occluded due the movement of other people or the objects.

And on top of all of the above it also has to be able to recognize facial features (enough to distinguish one person from another) as well as track changes to those facials features (smiling/frowning). And likewise do it for each and every person in the field of view.

I dunno, maybe all of that takes virtually no memory or CPU power. But it at least appears as if it would take some non-trivial amount of processing power and memory. Memory BTW - that both consoles are fairly short on.

Considering to do something similar with the Eye Toy + Wands the PS3 would have to do all the work as well as use system memory.

Perhaps that's why it hasn't been demo'd with a game that could possibly stress the system. IE - retrofitting it for use in a conventional game. After all neither a painting application nor the Eye Pet appear to be graphically challenging or memory intensive.

Regards,
SB

If I got it correctly, the NATAL device itself should include a powerful enough processing unit and memory separate from the console in order for the 360 to run satisfactory demanding games with all the features promised. Right?

If this is the case I expect NATAL to be a costy hardware.

It would defeat it's own purpose which is to attract casuals. Unless MS manages to market the 360 like the way Sony tried to market the PS3 but not that effectively: An all in 1 home entertainment device for every person in the family, necessary for the best digital entertainment experience in your living room. Necessary is a keyword in order to excuse a pricy product and make the consumer feel that he NEEDS it and therefore pay the price

 

[liolio]

I think Natal will at first cost Ms some money, that's it. And as others said the easiest way to market this properly is to go for a new brand with proper relaunch. That's why I think they will try to launch in the end Q3 or early Q4 along with a well polished slim model. Ms could keep only two SKU:
the slim xbox360 around the same price as the arcade now include 16/32 GB of ram, 180 euros would sound reasonable if regard to competitor offering. More than one year away with 45 and 40 nm parts, they could even make few money on the hardware.
The natal 360 ( say they come with a cleverer name like.... Muve just kidding ) @300 euros, that let 120 euros to cover the Natal cost.
As for natal alone, well as far as it's not more expansive than two games and come with a game I'm sure it would do nice. Not too mention that the price will go down with volume and there will be volume I fail to see how Ms could not bring this to the "pc verse". I think Ms start on the 360 because it's easier to make it work (in regard to navigation within the interface) in a controled environment. It may take more time to make this work in a special edition of 7.

 

[patsu]

http://e3.gamespot.com/story/6211559/ps3-motion-control-already-in-devs-hands

it appears Sony's motion-sensing efforts may be farther along than many thought. Speaking with GameSpot, Sony Computer Entertainment America senior vice president of marketing Peter Dille divulged that the technology--first revealed in a US Patent filing last fall--is already in developers' hands.

"We're a little bit past the research phase," he told GameSpot. "We're having conversations with the third-party community. The dev kits have started to go out to the third parties as well. They're working on the tech. They couldn't be more excited about it."

Link has a video interview with Peter Dille.

 

[Brad Grenz]

My presumption, based on joker454's comments, is that EA Sports is already working on PSMC stuff. I think it will be very easy for them to have PSMC integrated into their PS3 versions of Tiger Woods and Grand Slam Tennis by next summer.

 

[whome0]

and look at 59mn. Richard Marks have also made some demo with Depth camera like project natal few years ago (2004)
http://stanford-online.stanford.edu/courses/ee380/040121-ee380-100.asx
maybe Sony have a project like natal in their box for PS4

Richard Marks at Siggraph 2000
http://www.research.scea.com/research/pdfs/SIGGRAPHetech2000.pdf

not true 1:1 and good accurate but near concept than controller shown at sony conference

Almost similar controller as shown in E3 conference, but bigger lightbulp. Then interesting in a video how EyeToy team and Phil Harrison took the bite creating a shipping products.
29m30s = Wand with a lightbulp
35m00s = create a magic effect such as fireballs, same Richard Marks mentioned in E3 show "imagine you could cast a spell and throw a fireball"
59m00s -> = 3d depth camera, tech borrowed from some Israel compary. Skeletal mapping etc demoes.

Sony have had everything for years, did they want to wait for PS4 generation make everything cost effective and out-of-the-box device. Now they was forced to move faster and create addon controller.

 

[_phil_]

tech borrowed from some Israel compary

3DV is in Israel.

 

[obonicus]

My presumption, based on joker454's comments, is that EA Sports is already working on PSMC stuff. I think it will be very easy for them to have PSMC integrated into their PS3 versions of Tiger Woods and Grand Slam Tennis by next summer.

Are there any Sony studios we haven't heard from lately?

 

[Graham]

I don't see how that holds, Microsoft's solution has the least compatibility with the standard (Wii-mote). Natal games will have to be very specifically designed for it, control schemas for Wii-mote/Waggle targeted games will simply not fit it ... unless they compromise and include a wand.

If anything, I see the opposite.
There is one crucial point that is often missed, and I alluded to it earlier. Natal is a complementary technology. It can be used in conjunction with all their existing control systems, and from everything microsoft has said, it will not significantly impact performance on the console. (From what I can tell, the demo they were showing behind closed doors was *entirely* processed externally).

It's this combination which I think will be the critical point. A set of well developed systems (motion tracking, facial/image recognition, voice tracking and recognition). With potentially little to no overhead and the ability to complement an existing control scheme. For developers, it becomes a question of what they can add to improve instead of what they can replace to improve, which I feel is important. If this process is as easy as they have implied, then the success of the system is far more likely (in my view). The cost/benefit ratio is very good.

If anything, I'd compare it to the innovations the DS introduced. Touch screen, microphone and a second display. It didn't limit any existing style of game (after all, all the controls were still there) however it did open the door for all sorts of unique and unexpected games - but importantly practically every style of game could take advantage of at least one to improve the game somehow.

It will be interesting to see what odd ball uses developers come up with. I would have never even considered the milo/tamagotchi angle. Honestly, simply having a character look at you is a huge thing in my opinion.
Two different things that come to my mind, are the old Wii hack point-of-view demos are now possible (potentially TrackIR style interaction too), also in theory it would be possible to mix the video stream and face tracking to put your face in a multiplayer game - in real time. There are loads more, but these two things are good examples that don't put huge demands or requirements on the game's design, and would be a real benefit to immersion.

To bring it back on topic my concern is Sony's scheme just doesn't fit this model. To me, it puts a lot of requirements on the game's design. Something few developers want to mess/experiment with.

 

[Love_In_Rio]

Are there any Sony studios we haven't heard from lately?

Sony London-Cambridge ?. Have been making Home and EyePet "only". Sony London has always been working with Richard I think. Cambridge wasn´t also reestructured integrating people from Ninja theory ?. The Getaway and 8 days to Vegas were canned, i wonder if it´s something related. Although knowing Sony... I don´t think. They will start everything from now probably.

 

[Love_In_Rio]

If anything, I see the opposite.
There is one crucial point that is often missed though, and I alluded to it earlier. Natal is a complementary technology. It can be used in conjunction with all their existing control systems, and from everything microsoft has said, it will not significantly impact performance on the console. (From what I can tell, the demo they were showing behind closed doors was *entirely* processed externally).

It's this combination which I think will be the critical point. A set of well developed systems (motion tracking, facial/image recognition, voice tracking and recognition). With potentially little to no overhead and the ability to complement an existing control scheme. For developers, it becomes a question of what they can add to improve instead of what they can replace to improve, which I feel is important. If this process is as easy as they have implied, then the success of the system is far more likely (in my view).

If anything, I'd compare it to the innovations the DS introduced. Touch screen, microphone and a second display. It didn't limit any existing style of game (after all, all the controls were still there) however it did open the door for all sorts of unique and unexpected games - but importantly practically every style of game could take advantage of at least one to improve the game some how.

It will be interesting to see what odd ball uses developers come up with. I would have never even considered the milo/tamagotchi angle.
Two things that come to my mind, are the old Wii hack point-of-view demos are now possible (also TrackIR style interaction), also in theory it would be possible to mix the video stream and face tracking to put your face in a multiplayer game - in real time. There are loads more, but these two things are good examples of things that don't put huge demands or requirements on the game design.

Yes. Theorically Microsoft could make a wand also and Sony a 3d cam also.

But the point here is:

Do the buy from Microsoft of 3DV and the Z-cam patents make inviable for Sony to make a 3d cam ?. That´s the master movement i think Microsoft made here. In fact they finished the buy the same day of their conference in the morning. I could imagine Sony trying to buy it the same day in the evening...

 

[TheWretched]

The Europeans studios have been very quite imho... but that can also be only my perception^^