Mocap/facial animation methods and design decisions *spawned from LA Noire thread
- Thread starter Laa-Yosh
- Start date
Al-right, I shall not talk to strangers too.Oh, and the reason they went with blendshapes was because Cameron wanted 100% control over how each expression looked, down to the arc of lips and so on, and the best way is really just to move the vertices around manually. A full blown muscle system can't give you that, it's very indirect and slow on top of that all.
Recreating someone's face on the computer is another issue though, but the faces in those shots at GI are quite obviously not 100% accurate replicas anyway.
Recreating someone's face on the computer is another issue though, but the faces in those shots at GI are quite obviously not 100% accurate replicas anyway.
If the rendering isn't going for photorealistic, surely Uncanny Valley is a non-issue, as it only rears its head as we close in on realistic. Animation may not fit and look jarring, but that isn't Uncanny Valley where it creeps people out.
Sure. As I understand it, the guy who coined the phrase was talking about an emotional response to visuals simulations, where viewers are comfortable with unrealistic representations of people, and perfectly realistic, but when we get close to realistic, the expectations of realism collide with little inaccuracies and the viewer is unable to resolve what they are seeing as either artifical or natural, causing a discomfort. That's different to lip-sync failing, which is jarring but isn't related to trying to understand what one is seeing; it's obvious you're looking at a cartoon, as it were, but the lips don't match the words and it's annoying. In this case, L.A. Noire is clearly artifical, so there's no chance of confusing players as to whether they are seeing something real or not. Any advances in motion capture will benefit the quality of the virtual acting, but not alleviate Uncanny Vallye which won't actually be present.
In fact, it could even contribute to UV! Visually the game isn't realistic, but if the performances are, including subtle eye movements, the game will look like real people made out of dubious waxy-plasticy stuff. There needs to be sufficient distance from reality to avoid a negative emotional to the confusion of a nearly-real world.
In fact, it could even contribute to UV! Visually the game isn't realistic, but if the performances are, including subtle eye movements, the game will look like real people made out of dubious waxy-plasticy stuff. There needs to be sufficient distance from reality to avoid a negative emotional to the confusion of a nearly-real world.
The guy was a japanese robotics scientist and he originally discussed humanoid robots.
That's not true IMHO... I'm not gonna get into it, but just think about how scary some of the AC2 characters are, despite them being obviously not 100% realistic. The oversized and exploding mouth on the Kristen Bell character is almost scary and there's a lot of that kind of stuff going on in the game.
Even a cartoon character can become creepy with the 'right' kind of problems. So it's not as simple as "don't make it realistic" to avoid problems with facial animation and negative emotional responses.
That's not true IMHO... I'm not gonna get into it, but just think about how scary some of the AC2 characters are, despite them being obviously not 100% realistic. The oversized and exploding mouth on the Kristen Bell character is almost scary and there's a lot of that kind of stuff going on in the game.
Even a cartoon character can become creepy with the 'right' kind of problems. So it's not as simple as "don't make it realistic" to avoid problems with facial animation and negative emotional responses.
Okay, so what you're saying is the Uncanny Valley model is a poor one, and it's not just a case of distance from approximation of reality that can lead to a negative response, as described in the theoretical plot? I can agree to that, though I don't suppose anyone has nailed down research that properly quantifies human response to design decisions. There's also a degree of subjectivity, and responses that aren't based on inaccuracy but actual designs. I suppose at this juncture Uncanny Valley is a term being hijacked to me an unwanted dislike of a character caused by a limit of its implementation. In the case of L.A. Noire, they are targeting the animation limits that affect other titles like, say, Heavy Rain, which can feel disjointed in its animation (to me at least!). Rockstar's intention is to capture and apply directly real-world acting to make the characters empathic.
Yeah, well, it could be something like this but I don't know how much memory it could use and how flexible it'd be. It's near impossible to change anything about the captured performance, it has to be the same actor with the same facial construction and so on.
This is sort of the complete opposite of Avatar's artistic approach, trying to measure and digitize everything without knowing what's really going on. It has the chance of missing the emotions, and it's also pretty hard to manually tweak if there's any need for it. I am usually biased against this approach on a principal basis, even though I have to admit that it has its uses.
Yeah, sort of... The way I'd put it is that based on our experiences with real life, we have an instinctive expectation of how certain things should be recreated. This can include facial animation, but also mass and gravity and dynamics of movement (for complete characters, vehicles, smoke, water etc). It also makes sense to apply it to puzzles in video games, as Naughty Dog seems to think.
And robotics with realistic looking androids is just one possible appearance of this sort of phenomenon.
My take on this is that if something's supposed to be a human being and it's not as stylized as, say, a Garfield comic strip, then there are several very important characteristics that it just can't miss.
Stuff like keeping the volumes of the lips intact during both stretching and compressing motion, sliding skin instead of stretching it whenever possible, maintaining the bony forms of the skull, and so on.
If LA Noire wants to succeed in believable facial animation, then they have to work really hard, based on the balance of realism vs stylization that the screenshots suggest. The characters are still very far from cartoons and our brain will still expect them to move realistically.
Some of these issues can be pretty complicated with motion capture, especially when there's not a full match between the actor's face and that of the CG character. Bones and skinning in itself is notoriously bad at maintaining volumes, and on top of that many games tend to re-use the same geometry for very different faces (although Heavy Rain isn't like that). Blend shapes can solve some of these problems but can't usually be driven by mocap, and the movement of individual vertices on their own is always linear (although it's possible to use in-between blendshapes and/or nonlinear transformations).
The Lightstage-based capture system can solve many problems here but I don't have any idea how it could be scaled to a realtime environment. It also only really works if its combined with animated normal maps at the least, which are also captured from the actor's face. So they don't have much freedom here, once they go with this method they'll have to follow it through all the way. Then again it's also just a possibility at this point, although it fits with the GI article's description.