Could you go into more detail on FACs if you know anything?
Yeah, sure.
FACS is for facial action coding system, developed by Dr. Paul Ekman and his team in the `60s to get a way of describing facial expressions of patients in mental institutes, so that the doctors could compare exactly what's going on instead of trying to describe it or draw it.
I think it can also be used to detect subconscious ticks and such that the face does when lying, and there's a TV series with Tim Roth about it as well?
Action units are based on what the brain can tell the facial muscles to do, not on the individual muscles of the face or what we call expressions like anger or sadness. Those larger expressions can almost always get decomposed into a set of AUs with various weights for each.
For example a nice smile usually involves the Lip Corner Puller but also the Cheek Raiser to form the crow's feet around the eyes and perhaps stuff like Inner and Outer Brow Raiser and Upper Lip Raiser and so on. But leave out the Cheek Raiser and it's suddenly a fake smile.
There are detailed descriptions on what muscles each AU fires, I think they even did some research with electrodes and such. Lots of facial expressions use AUs unintentionally although getting into the evolutionary and behavioural aspects would be too much for me
They also included head turns and gaze directions but this is irrelevant to facial rigging, btw.
The big advantage of FACS is that since the system uses small elements, there's an almost limitless number of expressions that can be built, and the face can move into an extreme position and still act from there (like look pretty angry with all muscles compressed and then start to talk or shout). Lip sync is also not about dialing phonemes like E and F and M, but using the various AUs to shape the lips.
The idea to use this approach in CG facial animation is kinda old by now, but the first big example was Gollum. The main reason it took such a character is that there was an inherent problem with combining to many AUs and getting a result that looks unrealistic and disturbing and kinda messed up altogether. Weta solved this by creating 'fix' shapes for practically all the thousands of combined expressions and mixing them on top of the base shapes. This involved a LOT of work but the results were amazing. Also the animators had to learn how AUs work and it was somewhat against traditional 2D animation approaches where realism was secondary, as stylization helped with readability. We all recognize a smiley face, after all (I'm told even babies do).
Today FACS is sort of an industry standard, especially after Avatar, Apes and Hobbit have taken it to a new level. But FACS was also used for Benjamin Button, Dobby in Potter 7, the CG Guy Pearce in Iron Man 3 and so on. So it was inevitable for games to implement it as well.
Last of Us is using pre-set values for bone rotations and translations to create AUs, and adds blendshapes and wrinkle maps on top. Halo 4 is not using a direct implementation of FACS but it's very similar, and it's completely blendshape based but also using wrinkle maps (and not just normal, but color as well).
I think Beyond is still not FACS but a direct transform based approach, as they're capturing from the same actor that was used for the CG model. They'll probably have to switch for PS4 projects though.