L.A. Noire from Rockstar

Shifty Geezer said:
The lip syncing is superb! Best real-time virtual acting I've ever seen by a long chalk. Do we know if this is hard-baked from the English capture, or can they use alternative soundtracks and match up to those?
Click to expand...

Looking at the research on the Debevec site, I can't really decide - maybe it is a flexible facial rig that can be animated manually... but it's just too perfectly synced and so much is baked into the animated textures that I can't really decide. It'd take a huge effort to combine phonemes and emotional expressions. Also note how the eye reflections are completely static because they're also baked into the textures!

After a bit of digesting it, I think the way it works is to use two layers of deformation:
- the first is relatively standard image based mocap, using markers drawn on the actor's face, recorded together with the audio, to drive the lip sync and the expressions on the base polygonal model
- the second is to animate (blend between various versions of) the color + normal maps that have a few dozen pre-sets for expressions and phonemes to add finer things like wrinkles and such; these have been recorded in the "training" phase of the capture session, using stereo photography and the light stage and are driven by certain poses of the geometry or the marker based mocap
Also, normal maps are probably 1/4 resolution, which is why we don't see any skin pore detail. The only thing they really need to change the color map for is the blinking, but they need new normals for a lot of the expressions and have to blend the maps together when various expressions and phonemes are combined together. That probably takes a lot of processing, hence the simple lighting and shading.


So it is very unlikely that they could just use a sound file and have automatic lip sync. But they don't have to record the entire performance in full detail, only a set of predetermined expressions and then use "standard" mocap for the actual performance.


As for the rest, my first impression was that they were going for a stylised look.
Click to expand...

I'd say it's more about lacking the hw resources to use more than one light... The animated faces have to take up an insane amount of memory! One of the reasons that such tech wasn't really used before is that it comes with a lot of compromises.
Actually I wouldn't be surprised if the game would have to enter into "conversation" mode from normal gameplay and maybe even load for a while.
 
you guys are right, facial animation is the best I have seen in a game so far...but unfortunately...the rest is not, which kind of is a jarring difference?!?
 
Also, it really is what they've been talking about here:
http://forum.beyond3d.com/showthread.php?t=56574
Notice how I've been very sceptical ;) but it really did turn out to be a lot better then I expected.

It has been scaled down from the original research to fit the realtime resources. The other problem is that you need real living actors for every single role, you need to capture every second of all the facial animation for the entire game, and it takes a lot of processing and expensive equipment. You can't even change facial hair, age, or make-up as it's all recorded with the performance.

The good news is that even today's PC hardware should be able to significantly scale up the results for even better looking digital humans. Higher resolution textures with more detail, better skin shading, better lighting... they could probably also record the reflectance data for each actor's face and use some kind of HDR image based lighting to re-light it for any environment, using the same existing equipment and effort. Now that Rockstar has this system in place, they'll have a huge lead on everyone else on the new generation of consoles.

Is this game coming for X360 by the way? I seriously doubt that they could fit all this data onto a single DVD...
 
Also, just so you know, this is all very, very different from how stuff like Avatar's or Tron Legacy's CG characters are made, and it can't be used for any fantasy creatures that couldn't be created with make-up and prosthetics. So a game like Mass Effect 4 is probably not going to be able to use the tech ;)
 
PhilB said:
http://xbox360.ign.com/articles/113/1135354p2.html

New preview. Wow I'm loving the sound of this game. I just hope in the course of trying to make it accessible it doesn't become too easy to solve cases and such.
Click to expand...

I'm getting pretty excited about this game. My only concern, like yours, is that the investigation will be too easy, either because they lead you to the clues, give you too many hints, or make it too easy to read a person and tell if they're lying or telling the truth.
 
The very best thing about Rockstar is the way they don't worry needlessly about pushing the graphics envelope. Surely, sometimes the texture mismatches and shadow weirdnesses could be improved upon, but they deliver* the gameplay goods time and again. I'd prefer they keep the emphasis on those LoD formulae and seamless loading and keep my draw distance as high as possible. Yes, the lighting often seems like a hodgepodge of ideas,are and the characters rather cartoony even when they have high res textures (which is especially jarring at first). However, their approach to using what they've got is industry-leading IMO. Sacrifice, sacrifice, sacrifice, and keep the draw distance high and the camera angles interesting and the character acting phenomenal...

...bear in mind I hold the opinions of Laa-Yosh and some others around here, on graphical details, in the highest regard. But I think as a gameplayer you have to rejoice that GTA 3 looked so bland, for example, and GTA 4 so mismatched, and this looks very similar in that regard. Sure, maybe with a few more coders thrown at it, perhaps the lighting would look coherent,or maybe they'd wind up cutting corners somewhere else, or become obsessed with the graphics holy grail, and the game would just be pretty and suck. So, sure, you can certainly complain; their games always look like could have done better, but nobody really does it -- better looking, better playing games, that is.

*big exception has to be gta IV launch for PC. wtf happened to QA!?
 
Last edited by a moderator:
poopypoo said:
The very best thing about Rockstar is the way they don't worry needlessly about pushing the graphics envelope. Surely, sometimes the texture mismatches and shadow weirdnesses could be improved upon, but they deliver* the gameplay goods time and again. I'd prefer they keep the emphasis on those LoD formulae and seamless loading and keep my draw distance as high as possible. Yes, the lighting often seems like a hodgepodge of ideas,are and the characters rather cartoony even when they have high res textures (which is especially jarring at first). However, their approach to using what they've got is industry-leading IMO. Sacrifice, sacrifice, sacrifice, and keep the draw distance high and the camera angles interesting and the character acting phenomenal...
Click to expand...

This game is only being published by Rockstar, it's being developed by Team Bondi and is using an proprietary engine. Supposedly it features realtime GI.
 
N_B said:
This game is only being published by Rockstar, it's being developed by Team Bondi and is using an proprietary engine. Supposedly it features realtime GI.
Click to expand...

omg whoops! Somehow I missed Rockstar becoming a publisher o_O;;

...and here I thought the characters bore some resemblance to GTA 4 characters, but obviously that's just my brain seeing what it wants to see! ^^;;

thx for the clarification.
 
N_B said:
This game is only being published by Rockstar, it's being developed by Team Bondi and is using an proprietary engine. Supposedly it features realtime GI.
Click to expand...

Never heard of them but it seems the founder is from The Getaway group.

Team Bondi was founded by Brendan McNamara, the former Director of Development for Sony Computer Entertainment’s Team Soho Studio in London and the writer and director of The Getaway which has gone on to sell over 4 million units on the PlayStation®2 console.
Click to expand...
 
Honestly, I would take this amazing new facial technology of awesome facial textures because the only thing that puts me off a game more than horrible looking bushes is poor lip-syncing. Hopefully this pushes other developers to make a more "cinematic" experience in terms of how characters look and talk.
 
Not every game can utilize this technology (you need a real actor, complete with hair and make-up, for every single role, so fantasy and SF creatures are already out) and besides, it's pretty damn expensive, there's only one studio to do the recording anyway and it's located in Australia as far as I know.

Enslaved is said to have some nice facial animation too, by the way.
 
kancerkid said:
Honestly, I would take this amazing new facial technology of awesome facial textures because the only thing that puts me off a game more than horrible looking bushes is poor lip-syncing. Hopefully this pushes other developers to make a more "cinematic" experience in terms of how characters look and talk.
Click to expand...

While it's not bad, I don't think it looks very good yet either. But it's definitely a promising technology that I'm sure will gain ground over time and as the tech improves.
 
Laa-Yosh said:
Not every game can utilize this technology (you need a real actor, complete with hair and make-up, for every single role, so fantasy and SF creatures are already out) and besides, it's pretty damn expensive, there's only one studio to do the recording anyway and it's located in Australia as far as I know.

Enslaved is said to have some nice facial animation too, by the way.
Click to expand...

They don't have to use it for every character just like there's differences between poly counts for main characters, enemies, NPCs you can talk to and just background extras in most games. They could animate an alien in a traditional way in pre-rendered CG and then take results from that and put them through the animation streaming system.
 
If the characters aren't using the same tech for facial animation, the results are going to be far too inconsistent. Creating and animating a CG head and converting it into the system would be even more expensive then using a real actor, by the way.
 
Laa-Yosh said:
If the characters aren't using the same tech for facial animation, the results are going to be far too inconsistent. Creating and animating a CG head and converting it into the system would be even more expensive then using a real actor, by the way.
Click to expand...

Aliens don't necessarily fall into the uncanny valley so they could use different animation. For those that are humanoid enough to fall into the uncanny valley you could do digital make-up. You could reverse engineer muscle movement and model it on a real time character. There are loads of options to make aliens/animals/robots/cartoons possible. Bigger budget gives the best results tho.

As for inconsistency, most of the time you don't even see who of the background extras said something.
 
It's not a question of uncanny valley, which I still believe to be an oversimplification of a more complex problem. I'm too tired to get into specifics today, but please trust me on this a little bit, I've been working on facial animation for CG for like 7 years by now.
 
I think realism comes from the physics of the skin based on how it reflects light at an angle, How the skin stretches thinning the skin and its relative locality to other thicknesses and angles effects subsurface scattering, all in conjunction with the material properties and even the properties of multiple layers. Then there's movement based on force, resistance, weight and natural timing. Lightstage like systems covers all these things for real world stuff it records but the same animation system can take calculated animated things that fake these attributes well enough. So I figure I have a good understanding of the problems involved just from an understanding of the factors involved. Knowing how difficult some of my ideas are to implement in real world situations is some thing I don't have the experience for.
 
Last edited by a moderator:
Now there are two separate things mentioned there, shading and animation. The strength of the system used by this studio lies mainly in the animation, especially because the current game's engine does not use any complex shading at all, it seems to be a simple phong specular highlight and some low res normal maps. But there are more of those, specific to elemental expressions (sneer, wrinkled brows etc), that are blended together based on the recorded facial animation to create the final normal map used in the rendering.

The system also relies on the quality of the captured data, which is obviously 100% realistic with a real living person. Everything will behave well and consistently, no exploding mouth, flexible skull and jaw bones, moving eyes etc etc. On the other hand, if they try to build a character manually, from the ground up, with no capture, then it must be very close to the quality of a real human being in order to belong into the same world.

I understand that every single game out there has quality differences between the main and background characters. But the very point of using such a complex and hideously expensive system that LA Noire has implemented is to immerse the player in the characters, so creating an uneven cast will break the illusion and ruin the mood immediately. So if you're paying this much, you'd better go all the way, otherwise all the high tech efforts will be in vain. And if you're going to have fantasy creatures as a significant part of the cast then it's not worth investing in this technology, because you'd have to develop lots of high quality assets and tech with a different workflow - but it's silly to keep two completely different systems running in parallel.
 
You must log in or register to reply here.
Back
Top