PS3 to do real face morphing?

[mckmas8808]

This new processor is supposed to bring all kinds of good mojo to the PS3, due to its ability to spread work out over as many as nine cores on a single chip. Most of us still haven’t really grasped the tangible aspects of this, until now. Reader Alexqd3 sent us a link to a couple of Japanese videos that show these amazing face-morphing programs, supposedly capitalizing on CELL technology. The first video here shows a Japanese correspondent playing around with facial makeup and hair which seems fairly normal, but the second video shows actual 3D face manipulation based simply on a photograph of the correspondent’s face.

If this is evidence of the CELL processor’s abilities, I’m definitely looking forward to what this will mean for the PS3, as well as the kinds of games that will be developed to take advantage of it.

Video http://www.tv-tokyo.co.jp/wbs/2005/08/26/movie/tt.ram

Link http://us.gizmodo.com/gadgets/home-entertainment/ps3s-cell-chip-can-help-morph-your-face-131545.php

The first video we've seen before, but the second I've never seen personally. It is morphing her face from a digital picture that they took of her! With the HD Eyetoy this application is very possible in the PS3. That about innovation. Hopefully some devs will pick something like this up in PS3 exclusive games. Please put my face of a character and I would buy the game pronto.

And one please please tell us what the hell they are saying. I'm begging you.

 

[Shifty Geezer]

There's already at least one PS2 game that lets you stick your own fizzog on the characters.

 

[mckmas8808]

There's already at least one PS2 game that lets you stick your own fizzog on the characters.

Correct I think it was MLB 2006. But look how well the CELL is doing it though. The quality of her digital face looks so good. Perhaps Sony could allow you to save people's face to their online name, then when that person gives you an email their face would come up like the video and the digital face could read it for you.

It would be a nice little enhancement instead of reading the email you know.

 

[Carl B]

Correct I think it was MLB 2006. But look how well the CELL is doing it though. The quality of her digital face looks so good. Perhaps Sony could allow you to save people's face to their online name, then when that person gives you an email their face would come up like the video and the digital face could read it for you.

It would be a nice little enhancement instead of reading the email you know.

Mckmas I like that email idea!

 

[Vince]

There's already at least one PS2 game that lets you stick your own fizzog on the characters.

If this is the Toshiba demo then not only is the article old, (heh), but it's a bit beyond anything seen before. They were manipulating the face itself in 3D, applying different haircuts and make-up styles to the face via texture layering in reatime. It was pretty neat to see.

This would be a bit more akin to scanning your face into a next generation FPS in which you can see you're become dirty and bloody during battle, have your hair burned off, face deformed, or hell, tattoo yourself.

 

[mckmas8808]

This would be a bit more akin to scanning your face into a next generation FPS in which you can see you're become dirty and bloody during battle, have your hair burned off, face deformed, or hell, tattoo yourself.

Correct Vince. And can you imagine scanning your girlfriend or insert your favorite movie star or singer(i.e. Halle Berry) into a game like GTA 4 and the girlfriend in the game looks just like your girl. To me that's innovation if done correctly.

 

[Laa-Yosh]

Could someone tell me how the PS3 will extract a normal map from a photo? A specular map? How it will remove the lighting to get a clear color texture?

These automated kind of things have existed for years, but haven't been much more than a gimmick. They morph a generic face model to fit the photo to a degree, but it won't be a good fit and chances are that the more different your face from the average is, the less this technique can be used with you.
This can not replace a talented art team, so don't expect much use of such features, especially in games with high quality graphics (the 'scanned' face would look far too inferior compared to the rest of the assets).

 

[Inane_Dork]

Stop being smart...

I can do that for him if he refuses.

 

[mistan]

you know what?
isnt this suppose to show what the CELL can do?
well ummmm, what OS is running in that 1st video?

 

[Npl]

Could someone tell me how the PS3 will extract a normal map from a photo? A specular map? How it will remove the lighting to get a clear color texture?

These automated kind of things have existed for years, but haven't been much more than a gimmick. They morph a generic face model to fit the photo to a degree, but it won't be a good fit and chances are that the more different your face from the average is, the less this technique can be used with you.
This can not replace a talented art team, so don't expect much use of such features, especially in games with high quality graphics (the 'scanned' face would look far too inferior compared to the rest of the assets).

suppose you aint just using a photo but have to rotate your head. Should be quite possible to extract (non moving) lightsources and a quite good "shape". If the new eyetoy additionally captures depth this would be even more exact. I guess the hair could give serious trouble though.

Noones talking about a game which consists entirely about "home-scanned" models between and replacing art-teams. Singstar dint replaced regular Popstars either

 

[Shifty Geezer]

If this is the Toshiba demo then not only is the article old, (heh), but it's a bit beyond anything seen before.

Absolutely. I was just pointing out that if having one's own face was enough to sell Mckmass a game, he ought to have bought the PS2 game(s) that allow this already

 

[one]

Stop being smart, this is totally different than that. This is possible. You are looking right at it. Why question it now? And "one" where are you with the translation man. I can't understand what they are saying.

Hmm I think I remember them... but where? Here

http://www.beyond3d.com/forum/showthread.php?t=23239
http://www.beyond3d.com/forum/showthread.php?t=24203

Now that's amazing if these things traveled to Gizmodo from here. A nice internet meme experiment...

 

[Vince]

Toshiba brought 2 demos for this Cell devkit, well-known 48 MPEG2 streams playback and the new "Digital Kagami Type F" (digital mirror type F) demo. This new image-based rendering demo converts user's face into polygons and applies makeup to it by texture-mapping, template-matching and motion prediction by matrix calculation, and makes it into 720x480@30fps in realtime, on a half-mirror.

EDIT: Nevermind, sorry One... I missed your link. Fashionably late yet again....

 

[one]

well ummmm, what OS is running in that 1st video?

It's on Linux according to this report though ITRON is also available. The bit rate of 1 MP@ML stream in 48 MPEG2 streams decoding demo is about 4Mbps.

http://www.ednjapan.com/content/l_news/2005/10/05_07.html

Oh, and I have to make it clear that Sony's demo that generates 3D facial animation from a 2D picture is not using Cell.

Toshiba does R&D on pattern recognition and biometrics so the Cell "digital mirror" demo seems to be a derivative of these research.
http://www.toshiba.co.jp/rdc/mmlab/tech/w30e.htm

 

[randycat99]

Wrt getting a 3D model from a 2D image- it is possible and has been under research for a few years by now. It's not perfect, sure, but it is quite amazing the kind of data that can be synthesized with image processing on a mere 2D image. I think it uses lighting and perspective cues to infer depth data. Probably since year 2000, they have been playing around with algorithms that can take 2D images of buildings and auto-transform them into 3D CAD geometry.

 

[PC-Engine]

http://www.beyond3d.com/forum/showthread.php?t=22891

 

[pakpassion]

mckmass. yes real face morphing can be done with the eye toy but the G70 architecture is not powerful enough to emulate the 3d imaging power of the Cell in games. so this will only have eyetoy implications, not Game implications.

 

[Shifty Geezer]

Huh? Why not have the Cell do the 'morphing' needed for gameplay then? Not that I can think of any amount of gameplay associated with messing with one virtual features (except for girles, and a 'Barbie's Makeover' game). The only real contribution is including your own 3D likeness in a game but that's not gameplay related.

 

[SynapticSignal]

Cell is a demo-hyped processor IMO

don't misunferstand me, I believe that cell can do
1) morphing from a cam
2) sound processing
3) phisycs simulations
4) rendering without a gpu
5) hundreds of ducks
6) AI processing

but in a demo, Cell have to process only one at time of this tasks
what I don't believe is that Cell will manage all this task togheter in a game
not with 512 KB of cache, not with only 7 spe that can't reach directly the main memory, not with a single PPE core in order

IMO
so I consider all this demo, very useless if you aim to misure the real performance of the processor

this is only marketing

 

[Shifty Geezer]

Demo's illustrate availabe power. And in that respect if Cell can render 100 ducks with full rigid body and fluid physics etc at 60fps, and a P4 can't, then it shows what it's supposed to - Cell is more powerful in those tasks. And demo's do so in an entertaining way. And as there's no cross-platform software to run and compare with new hardware, what else are you going to show off? Honestly, how on earth are Sony/Toshiba supposed to demonstrate their new chip?

"Yep, we've created Cell. It can do stuff other processors can't but we can't really fairly show you with demos which are written specifically to showcase these extra capabilities. It'd be nice to compare it to other processors but there's no cross-platform software we can run on different machines to compare. It's not like we can write our own software to demo on P4, AMD64 and Cell because everyone would accuse us of deliberately implementing weak code on the other platforms to make Cell look better. And we won't give you any peak-spec numbers because everyone always complains they're meaningless and it's only real-world number that count.

"So here we are. We've created a great new processor, but can't tell you about it without being accused of hyping numbers, and can't showcase it as our demos are seen as fictious, worthless applications, yet somehow we need to convince people to buy it and write software for it. To that end we're releasing the world's most extensive range of chip customizations. Cell's package has several face-plate configurations, and we're adding an extensive range of light and sound audiovisual enhancements. No other chip in history has had the same aesthetic possibilities as Cell, and we cater for female owners too with a series of cutesy faceplates."