How good is the PS3 or Xbox 360 at handling something like this?

["Nerve-Damage"]

1) Are these systems (PS3 or Xbox 360) even capable of performing this type of animation in a real-time application (i.e. game).

2) Which system is better suited at performing this level of animation in real-time?

Note: I’m not talking about the graphic detail per-se; I’m talking about the level of animation in the video in a real-time application.

Videos:

Credit: thabesttodoit

Presented as a technology preview at SIGGRAPH 2005, Face Robot™ provides artists with the tools to produce lifelike facial animation and digital acting with emotion and depth.

SOFTIMAGE®|FACE ROBOT™ allows 3-D artists to achieve realistic, lifelike facial animation for high-end film, post and games projects. Based on extensive anatomical research, Face Robot™ uses a groundbreaking new computer model of the soft tissue of the face to mimic the full range of human emotion. Designed with input from leading animation experts, Face Robot™ gives artists an intuitive way to interact with their characters while providing control over details like wrinkles, frowns, flaring nostrils and bulging neck muscles.

Face Robot™ supports both keyframe animation and motion capture, the primary techniques used for digital acting today. The soft tissue model at the core of the Face Robot™ technology removes the need to manually create dozens or even hundreds of 3-D shapes for different facial expressions and allows animators to work with an optimal number of control points. Keyframe animators can gain very direct, intuitive access to facial expressions, while motion capture animators can work with fewer markers to reduce setup and cleanup time.

With dedicated facial animation technology, Softimage enters a new category that complements the advanced SOFTIMAGE®|XSI® animation system. Face Robot™ removes the barriers to believable digital acting, allowing the 3-D community to fulfill their creative imagination.

 

[Bad_Boy]

omg that seriously creeped me the fuck out lol. I really didnt expect that image.

ok continue thread.

edit:
OK lemme get serious. Those videos are pretty damn impressive. Id jump for joy if we could get something like that this generation. But I always thought animationss were only as good as the animator could make them. (or software I guess in this case) And to have animations like that wouldnt it require for the faces to have much more poly's? CELL is supposed to be a great number cruncher so wouldnt that help in these specific animation situations?

Im not to tech savy on animation, so sorry if i may sound like a idiot on the subject.

 

[Laa-Yosh]

That animation is based on facial motion capture. I have no idea how you could separate the detail from the animation; especially with a human face, you need a certain level of detail in the geometry in order to show a reasonably complete range of facial expressions. The face used here is ~5000 (quad) polygons IMHO, comparable to the detail in Gollum's face. Games would probably have to work with 1000-2000 triangles on nextgen consoles.

The quality of mocap facial animation depends heavily on the number of marker points tracked on the face; these each drive a bone or cluster on the 3D model, that will deform the underlying vertices with a transformation matrix.
The actual software demoed here, Facerobot, automatically adds a lot of secondary animation to the geometry as well, based on a sort of heavily simplified muscle and tissue simulation; the result IMHO is far too cartoon-like, instead of realistic. However, this is why they are only using ~25-30 markers; to reach this quality with bones/clusters only, you'd need at least two times as many.

So, in one way, the quality of ingame animation will depend on the number of bones and vertices in the 3D model of the face, which is a question of pure geometry processing. However, in my opinion it's far more dependent on the quality of the artists' work: how good is the model, the placement and weighting of the bones, and of course the animation itself.

It is also quite unlikely to use fully motion captured facial animation in a game, at least for every conversation. It's far more efficient - memory wise - to work with a limited set of facial expressions (25-50 pre-stored positions for the bones and/or vertices) and use a higher level of animation, driving these expressions and not the bones themselves. This can also allow easy automatic lipsync, as seen in Half-life 2.

So the short answer: both systems are quite capable of this, but due to memory constraints, they will probably end up using simplified approaches.

 

[!eVo!-X Ant UK]

I should imagine that it depends on how many polygons and animation points a next gen face will have. The less you have the less facial expressiveness you'll have to play around with.

You could possibly get that kind of facial detial, expressiveness on a fighter perhaps.

 

[Bad_Boy]

Yeah, I beleive fight night has many impressive facial animations for their game. But definately not up to that caliber, from what ive seen. Maybe another title will match that one day, who knows.

 

[Laa-Yosh]

Mocap animation data takes a lot of storage space. Continously animating a face with mocap means at least one, if not two, orders of magnitude more data per seconds of animation, then animating with pre-recorded expressions.
Actually, most developers even avoid using blendshapes because they consider them to be too memory-consuming as well - and a good set of shapes is just a few hundred kilobytes for a face...

 

[MrWibble]

Looking quickly at this, I don't think the mesh complexity would be a problem, nor do I really think the animation process itself looks particularly tricky (I'm assuming it could be achieved with blend shapes and/or skinning).

What might be an issue, is the amount of data required - both in terms of production and storage.

Assuming enough animation could be created, and it can be stored or streamed in, then yeah I don't see why next-gen consoles won't have stuff like this in certain circumstances. I wouldn't expect it everywhere though.

 

[Laa-Yosh]

Looking quickly at this, I don't think the mesh complexity would be a problem

Are you sure? Fight Night 3 seemed to have far less detailed faces than this (and the wires you see are for the control cage only, the rendered model is subdivided at least once, but more likely twice).

 

[dukmahsik]

safe to say that'll be ps4 or 720 before we see that realtime

 

[_phil_]

Actually, most developers even avoid using blendshapes because they consider them to be too memory-consuming as well - and a good set of shapes is just a few hundred kilobytes for a face...

who told you that ? that is not true.
Blendshape targets are just stored vertex coordinates .Only moving vertices are stored relative to a base shape.with say 16 targets of 600-800 moving vertices you can do a lot without hiting any memory problems.

 

[Laa-Yosh]

who told you that ? that is not true.
Blendshape targets are just stored vertex coordinates .Only moving vertices are stored relative to a base shape.with say 16 targets of 600-800 moving vertices you can do a lot without hiting any memory problems.

16 targets aren't enough for nextgen expectations IMHO... 30-40 are the bare minimum, but a fully articulated face may require even more.
But anyway, I've read a 2005 Maya Masterclass by Shinichiro Hara, character technical director at EA, and he has mentioned memory consumption (and workflow issues) as a reason for using bones. By the way, the example face he's using is built from ~3000 triangles, using 40 bones, and about 1K of texture resolution (with two extra 512*512 normal maps to layer on extra wrinkle deformations).

 

[rounin]

Someone put a warning in the title of this thread...

 

[MrWibble]

Are you sure? Fight Night 3 seemed to have far less detailed faces than this (and the wires you see are for the control cage only, the rendered model is subdivided at least once, but more likely twice).

Yes, I would think that's quite doable.

Even at 60Hz a single scene on a next-gen console might well have a polygon budget in the millions. I would say that's probably enough for this. Also, the question was about real-time reproduction of the animation rather than the rendering - the control mesh would be enough for the motion, the subdivision could be skipped and replaced with normal mapping without affecting the animation.

Note, I'm not suggesting you'd stick that kind of detail on a character 50 pixels high or during gameplay - but for a cut-scene... yeah.

 

["Nerve-Damage"]

Someone put a warning in the title of this thread...

Did the bad man scare you?

 

[wco81]

I understand having a detailed face on screen rendered and animated on screen is of technical interest to many.

But where is the aesthetics in having a screen filled by a giant face rendered in 3D and in RT?

Why not just use video or a prerendered sequence for such a scene?

It's not like you're going to do anything interactive with it.

The Molina demo was a tech demo but in the context of a game, it wouldn't be terribly interesting to move the light source or make his expressions change by manipulating the controls.

For such a detailed closeup, the gamer would be mostly watching. So no need for real-time. Might as well use high-def video from film sources. They hire actors to do the voice acting anyways. Throw a bit more money to have them act on camera. Can't be worse than doing some complex art and development just to get the animation and details life-like.

 

[Bad_Boy]

It's not like you're going to do anything interactive with it.

I'm sure Kojima could think of something.

I just think its so much more believable when a game doesnt have to cut to fmv cutscene just to extend a story, I think this is one reason why Kojima really has the ability to immerse the player, and the fact that most of his cut scenes are interactive even for just a small portion of controling a camera makes the game feel more complete. But thats just me. This next generation should have more real time cutscenes than cgi/fmv ones imho, the power is obviously there to make them look good.

 

[MrWibble]

I understand having a detailed face on screen rendered and animated on screen is of technical interest to many.

But where is the aesthetics in having a screen filled by a giant face rendered in 3D and in RT?

Why not just use video or a prerendered sequence for such a scene?

It's not like you're going to do anything interactive with it.

Maybe yes, maybe no.

In HL2 they had cutscenes that just consisted of characters standing around talking, but with the player still in complete control - so you can walk up to the NPCs and watch them, or wander off and explore the room.

Meanwhile a lot of other games have cutscenes that could be triggered in subtly different circumstances depending on the players actions. Maybe he's in a different room when he does something, or maybe he's got different characters in his team, etc.

All this is aside from the other reason already cited, which is that it can be an awful lot more immersive to have the cut-scenes using the in-game engine rather than cutting away to a video. As a gamer I'd rather have in-game scenes, and as a programmer I'm quite prepared to put in the extra leg-work required to make them possible.

The only issue is convincing the art-team to put aside the ambitions of working for Pixar and put down the renderer...

 

[London Geezer]

Another reason is cost...

Having real-time in-engine cutscenes is much cheaper than getting a studio to make the scene in CGI for you. Not even arguable. Prerenderes are (almost?) always outsourced to studios that sepcialise in CGI and CGI costs lots of money, obviously depending on the level and the render time.
Having your in-house animator construct the scene and let the engine you use for the game render the scene will be so much cheaper it's not even funny.

I'm still in awe from the real-time cutscenes in Jak3 (and also Jak2). That really must be the best real-time animation i've seen.

 

[_phil_]

16 targets aren't enough for nextgen expectations IMHO... 30-40 are the bare minimum, but a fully articulated face may require even more.
But anyway, I've read a 2005 Maya Masterclass by Shinichiro Hara, character technical director at EA, and he has mentioned memory consumption (and workflow issues) as a reason for using bones. By the way, the example face he's using is built from ~3000 triangles, using 40 bones, and about 1K of texture resolution (with two extra 512*512 normal maps to layer on extra wrinkle deformations).

i said 16 targets for current gen.Farenheit (indigo prophecy) for example is a ps2 game where the main characters facials are fully blendshape based.Streams of dialogues are animated with a glove controling blends by a specialist pupeteer.

Concerning animatable facial reflectance field:
http://gl.ict.usc.edu/research/afrf/
BTW ,not how the detail is very low on that face.

 

[McFly]

If I don't see something like this on the PS3, in at least one game, then I'm going to change my name to Bill Gates!

Fredi