Sony's New Motion Controller

[Crossbar]

I still can't fathom something like that replacing a Dual Shock 3. That was the reasoning my question: how can they make a break-apart controller that replaces the Dual Shock 3 AND supports the Playstation Motion Controller technology? So far I haven't seen anything that fits the bill.

We don´t know if they will make a break-apart controller yet. There are a lot of patents that never get implemented. But I can see it would fit a cheap PS3 SKU aimed at casual gamers.

I would assume they want to make a neutral wand that fits both the left and right hand, but I wonder what will happen to the PS-button in that case? It will be really interesting to see what the design turns out to look like.

 

[patsu]

We don´t know if they will make a break-apart controller yet. There are a lot of patents that never get implemented. But I can see it would fit a cheap PS3 SKU aimed at casual gamers.

They should try to make it happen,

I would assume they want to make a neutral wand that fits both the left and right hand, but I wonder what will happen to the PS-button in that case? It will be really interesting to see what the design turns out to look like.

The left and right part don't really have to be identical (One has PS button while the other does not). The red and blue SingStar mics are not equivalent even though they look alike. The blue one can use speech recognition (decided by software).

If they want, Sony can implement the PS button on both left and right halves.

 

[Nesh]

CamSpace seems to "recognize" the object in your hand to control the application. The demoes I saw didn't really control the app with bare hands. Nonetheless, the concept is indeed similar to the PS Eye and Project Natal.

I think the latter two are more advanced though. e.g., PS Eye can track up to 120Hz @ 320 x 240 resolution (or 60Hz @ 640 x 480 like regular webcams). It comes with a mic array for voice applications. The PS Eye software is also more versatile (Tracks human head, color, motion, pattern, speech and sketches/drawing).

Natal has z-camera, mic array and very sophisticated skeleton tracking system to boot. Speech, face and other forms of recognition are also in the works.


EDIT: Bah, forgot about the new controller totally. PS Eye also helps to handle precise and very fast tracking of the traditional game controller. So currently, Sony seems to cover all the basics. Hopefully they package everything together in a seamless manner. Already described what I wanted in the new controller (i.e., make it behave like a mouse when laid on a table, playable like a Dual Shock, and as precise as a Wiimote Plus when flailing )

This isnt about how well Camspace fairs next to Natal and EyToy.

My question is how can Sony incorporate a similar functionality with the Eye Toy in a more advanced form to approach Natal

 

[patsu]

Yeah I get that, but I am pretty sure PS Eye can already do what CamSpace does (assuming no patent violation). They just need to find an application for the technology.

All the EyeToy mini-games require you to control the game UI using bare hands anyway. So adding a colored object to track should be relatively easy.

 

[patsu]

Let me clarify what I said above. The "magic" is in the CPU as much as the PS Eye input device.

Sony takes a different approach from MS. They are Cell-centric, at least for this gen. Problems such as natural interface can presumably tap on Cell since there are already 2-3 years worth of porting/optimization by companies and universities worldwide. I have seen Cell projects covering voice, image, video, motion, head, face, pattern, writing, drawing recognition and tracking so far.

The key missing piece is the assorted input devices, and how to use them intelligently to solve a real-life problem.

CrossBar's earlier post sounds the most interesting to me (when we were talking about HD camera). If someone wants to introduce a sophisticated, high bandwidth peripherals, they could use SPURSEngine and the Gigabit Ethernet connection to extend the PS3. Because of the existing Cell programming model, the same SPUlet can be moved between the PS3 and the SPURSEngine peripheral (depending on application setup). That way, they can grow the wealth of knowledge and software on Cell programming, making them useful for other CE applications over time.

Not sure if Sony is still on Kutaragai's unified Cell vision though. There are still holes in this approach. e.g., At this stage, I think "one-off" features like smile recognition can be done vastly cheaper by a small camera chip. I wonder what the sweet spot is.

 

[one]

Sony Spills More PS3 Motion Controller Details To Devs
http://kotaku.com/5297265/sony-spills-more-ps3-motion-controller-details-to-devs

The PlayStation 3's second motion control push is slated to launch in Spring 2010, Sony Computer Entertainment reiterated to developers today, arriving with a "range of first-party and third-party content." SCE pitched the new controller as capable of functioning "with any games across all genres from casual to core."

On the technical side, SCE says that the PlayStation Eye accessory will come equipped with force feedback rumble. Sony boasts that the "face-tracking and head-tracking capability already in the PS3 SDK" will offer developers "powerful tools for new types of user interaction."

Sony says its new controller will also work in tandem with the Eye's built-in microphone array for voice input and voice recognition. Up to four of the new motion controllers can be tracked at once by the PlayStation Eye.

Adding the wand controller to games comes at a horsepower cost, Sony notes, writing that the "motion-tracking library has some SPU overhead and has an impact on memory as well, although these will be minimized."

 

[AntShaw]

What exactly is "force feedback rumble" on the PS Eye and exactly how would one make any use of that?

 

[one]

What exactly is "force feedback rumble" on the PS Eye and exactly how would one make any use of that?

PlayStation Eye accessory. Probably referring to the PSMC itself.

 

[tirminyl]

What exactly is "force feedback rumble" on the PS Eye and exactly how would one make any use of that?

PlayStation Eye accessory will come equipped with force feedback rumble.

I read that as the controller will come with force feedback (rumble) as the motion controller is an accessory to the PlayStation Eye camera.

Maybe that boomerang controller will make a comeback as the breakaway controller? I really wanted that controller

 

[Nesh]

Sony Spills More PS3 Motion Controller Details To Devs
http://kotaku.com/5297265/sony-spills-more-ps3-motion-controller-details-to-devs

Sony boasts that the "face-tracking and head-tracking capability already in the PS3 SDK" will offer developers "powerful tools for new types of user interaction."

Sony says its new controller will also work in tandem with the Eye's built-in microphone array for voice input and voice recognition. Up to four of the new motion controllers can be tracked at once by the PlayStation Eye.

Adding the wand controller to games comes at a horsepower cost, Sony notes, writing that the "motion-tracking library has some SPU overhead and has an impact on memory as well, although these will be minimized."

I guess this is one issue MS has solved by having extra memory and processing power inside Natal.

 

[TheWretched]

I doubt that the memory and cpu overhead is that much of a deal... Especially if they also go after the Wii crowd (Wii ports to PS3^^).

Videoprocessing is one of the things CELL does best, and a 640x480 picture isn't that huge either.

What I find more interesting is, that we basically go back to the 90s, where every console had its own games. I don't expect to see any real synergie between 360 and PS3 in that regard and the Wii is in a totally different league.

Natal is set out to do something completely different than what the PSMS is doing (input device vs. body motion, buttons and gestures vs. gestures only...)

 

[Brad Grenz]

Also I bet voice processing will happen on the host CPU for Natal as well.

And it does seem like there's a lot of spare SPU time to go around even in demanding first party games. I do wonder what the memory overhead is for something like voice recognition, though.

 

[TheWretched]

The Voice thingamagic in Vista and 7 uses about 50 megs... and since it can do QUITE a bit more, I guess they could shave off siginificant amounts of RAM.

 

[Arwin]

What I find more interesting is, that we basically go back to the 90s, where every console had its own games. I don't expect to see any real synergie between 360 and PS3 in that regard and the Wii is in a totally different league.

I don't think so, actually. I mean of course the consoles will have their own games, but there's going to be a significant amount of overlap as well. We'll still see the same tennis games and golf games on both platforms for instance, and I'm willing to bet they'll have significant amount of controller overlap. For instance the 360 and PS3 will typically support and share regular controller style controls as well as implementations of motion control - perhaps the 360 version will come with a golf-club or just allow you to use any stick or just pretend you hold a stick, etc., but other than that it won't be that different. And PS3 and Wii controls will be like-for-like for all intents and purposes.

In fact I think that we'll see a lot of third party Wii exclusives be released for the other two platforms now as well. I think we'll first see some Wii titles appear on PS3 and many of them will also or even exclusively release on PSN.

The exclusives are going to be interesting though. There's going to be an interesting new battleground where the differences between platforms are significant enough to make for an interesting race where it's often not possible to match the exact controls of a competitor, or in the case of the Wii and PS3, significant differences in terms of graphics and physics and the like. Each platform has its unique strengths and weakness and it's going to be interesting to see how they play out. And if Nintendo should decide to actually make a Wii HD, that may bring the PS3 and Wii even closer together.

 

[patsu]

I guess this is one issue MS has solved by having extra memory and processing power inside Natal.

If they need more h/w resources to run natural interfaces in a big way, I think it'd be an interesting solution to use a SPURSEngine extender via the gigabit Ethernet port. Then have PS Eye (plus any other extra accessories) connect to the SPURSEngine extender directly. That way, they can offload some or all of the same SPU code to the extender "without much change".

Presumably Toshiba would be interested to carry this extender to their high-end laptops and regular PC too.

 

[patsu]

http://www.joystiq.com/2009/06/25/sony-to-talk-a-lot-more-about-motion-controller-at-develop-con/

According to the official programme for this year's Develop Conference (a UK soiree for game devs), the recently totally confirmed PlayStation Motion Controller will be a highlight of a keynote titled "PlayStation: Cutting Edge Technologies." Specifically, attendees are being told to "expect to hear a lot more about the new PS3 controller shown at E3, together with vision processing and advanced graphics techniques in the PS3 libraries" during a presentation by SCEE staff.

 

[patsu]

I have recently came across a site of a software company where you can download an application that makes your camera an enabled motion detection device to play your PC games with.

The application is called CamSpace

http://www.camspace.com/

No controllers needed. It is the exact same concept used by Natal. I havent used it yet with any game but it is an interesting concept that makes me wonder how can Sony use a similar concept with the Eye Toy. I mean...that thing can make my own PC with a standard camera play games without controllers.

The only thing it may be lacking is depth detection and some more advanced technology to decrease the affection of different light conditions. 2D detection should be fine. Based on what I saw, many of the experiences may not necessarily require Z axis movement

Lets say that Sony wants to incorporate a Z-axis detection of course they might need a peripheral. Probably tiny bulbs using the technology we have seen on their demonstration that can be attached on body parts?

Youtube is filled with ton of videos of people using it. Even Goggle Earth can be used with it.

edit: Heh even draw with some moderate accuracy?

http://www.youtube.com/watch?v=E4BY-1bbp5M

Nesh, Sony filed a patent for using any real world object as the controller:
http://www.siliconera.com/2009/07/0...em-that-uses-ordinary-objects-as-controllers/

A recently published patent from Sony details a system where a camera can dynamically map an object — any real world object — for use in a video game. The illustration has a U shape block, but the patent outlines other example objects “include items such as coffee mugs, drinking glasses, books, bottles, etc.” While these are given as examples the object mapping system is not limited to those objects, it can identify any three dimensional object.

EDIT: It appeared 2 weeks ago in this thread first:
http://www.psu.com/forums/showthread.php?t=209417

The post also has a related video:

 

[Nesh]

Can this be applied to the Eye Camera?

 

[Arwin]

Can this be applied to the Eye Camera?

I would assume yes. Basically the same algorithm that's used for tracking the ball on the PSMC should be usable for object tracking. Sure, the ball is going to be more precise and reliable under most lighting conditions, and also less CPU intensive, but I think it should be able to do pretty well with a lot of different basic objects, especially if you have a predefinition of what shape it has in 3D.

 

[patsu]

Live Blog of yesterday's Develop session:
http://www.vg247.com/2009/07/16/liveblog-sony-motion-controller-session-develop-2009/

No photography was allowed. The site staff promised to post an audio recording of the session later.

EDIT:
More on PS Eye "Facial Recognition":
http://www.develop-online.net/news/32417/Sony-developing-PSEye-facial-recognition-software

The facial tech can see the position and direction of a player's head and can work out the gender and age of the face, and detects different parts of the face including nose, mouth, eyes, eyebrows, shape of the mouth - and even glasses.

The facial recognition tech can also detect smile according to the live blog above.

Some more on the waggle controller:
http://www.develop-online.net/news/32415/Sony-motion-controller-is-true-interaction

The mix of camera tracking and hardware means that the device is 'aware' even when it isn't being seen by the camera.
"It tracks the controller based on acceleromoter - so you can use it really freely, even behind your back," said Hirani. "You have great accuracy."

...

"It picks up all the pitch and movement. It's precise and responsive - the sphere on [the front] is what the controleller is tracking - it uses the full RGB spectrum for the colours."

EDIT 2:

Developers waning to make a game for the motion controller need a PSPEye - and prototypes of the actual hardware is available from SCEE account managers, but are in limited quantities at the moment. Studios will have to get whitelisted by Sony and discuss their ideas with them before the prototypes become more freely available.

And in terms of external middleware the device uses AiLive - which as Hirani pointed out "other devices using motion controllers use" (i.e. Wii - and it's free to all PS3 developers.