Accurate human rendering in game [2014-2016]

Status
Not open for further replies.
What??? Quantum Break's faces are looking fare more realistic to me. The facial animation system definitely works much better.

The game has a highly graded look though, similar to some movies of this past decade, but that has nothing to do with the realism of the faces.
 
iegq3y9XZb8EQ.png



IIRC remedy uses a special technique for the teeth. Denture mold or something.
 
I had a google, quantum break looks perhpas the best so far, though here the woman looks a lot better than the man
Quantum-Break-E3-2013-Xbox-One-Trailer_8.jpg

also this screenshot is quite old E3 2013. I think we will have to wait for some 'real' ingame screenshots, before truly judging
 
That woman conveys the feeling of fear as good as a puppet. Tough thing to model I imagine.
Otherwise looks nice.
 
I actually think Jody from Beyond looks better than that woman in QB. Much more realistic textures, shaders and conveys better emotion too.
beyond-two-souls_2013_03-21-13_006.jpg

BEYOND_SCREEN_091.jpg
 
Yes that does look better
(google) Is that from Beyond: Two Souls on the ps3. Surely that cant be realtime actual ingame non cutscene screenshot.
I had a quick google and cant really find many screenshots, are there any realtime actual ingame non cutscene screenshot with the charcters in some scenery?
 
Yes that does look better
(google) Is that from Beyond: Two Souls on the ps3. Surely that cant be realtime actual ingame non cutscene screenshot.
I had a quick google and cant really find many screenshots, are there any realtime actual ingame non cutscene screenshot with the charcters in some scenery?

Hm... depending on your view all the game is cutscene... and none of it is. It's been a while since I played it, but I can't remember a lot of LOD in slow scenes (there's a bunch of frantic escapes and whatnot...)
 
The only thing that would change is the resolution and antialiasing. Otherwise, yes, the game absolutely looks like that. I had to keep reminding myself as I played through it that it was a PS3 game and not PS4.
 
I'd be more appropriate to use these photos just to factor in what it actually looks like in-game (courtesy of Digital Foundry) . Other than that, yes, Beyond's facial animation is absolutely phenomenal. Would still love to see more of Quantum Break.

hair1.png

hair2.png
 
Teeth are actually pretty hard to get right. Crown has three non-opaque layers that you've got to simulate to get lighting right (you can get away with enamel and dentin alone I guess) and then you've got gum to simulate to get reflections and what not right. Beyond has some serious problems with red highlights on teeth, highlights that shouldn't be there. We're pretty far from rendering teeth real time. Same for eyes and hair. ;) That said - I haven't seen any papers discussing teeth rendering. Do you know of any?
 
beyond 2 souls is in game and you even able to "expand" the render area by removing the black bar, as long as you have hacked ps3.

although its also possible that they just a recording that have additional black bar applied :/
 
That said - I haven't seen any papers discussing teeth rendering. Do you know of any?

I think there was paper (maybe from Siggraph) few years ago. Let me see if i can find it.

Real-Time Rendering of Teeth
with No Preprocessing
http://orbit.dtu.dk/files/10393700/RealTimeRenderingOfTeeth.pdf

Abstract.
We present a technique for real-time rendering of teeth with
no need for computational or artistic preprocessing. Teeth constitute a
translucent material consisting of several layers; a highly scattering mate-
rial (dentine) beneath a semitransparent layer (enamel) with a transpar-
ent coating (saliva). In this study we examine how light interacts with
this multilayered structure. In the past, rendering of teeth has mostly
been done using image-based texturing or volumetric scans. We work
with surface scans and have therefore developed a simple way of esti-
mating layer thicknesses. We use scattering properties based on mea-
surements reported in the optics literature, and we compare rendered
results qualitatively to images of ceramic teeth created by denturists.
 
I had a google, quantum break looks perhpas the best so far, though here the woman looks a lot better than the man
Quantum-Break-E3-2013-Xbox-One-Trailer_8.jpg

also this screenshot is quite old E3 2013. I think we will have to wait for some 'real' ingame screenshots, before truly judging

Here is screen with acceptable compression

image_quantum_break-22372-2722_0002.jpg



Beyond's Jodie looks great, but QB definitely has better shading and geometry on theirs characters.
 
Last edited by a moderator:
The facial animation and deformation system is more advanced as well.

Both Beyond and the Sorcerer demo are using only bones to deform the face, driving them by a straight translation of facial marker 3D mocap data. This makes some of the deformations notoriusly hard to reproduce, especially the mouth and the eyelids, even with a large number of bones.
Basically they don't capture the performance itself, the intent of the actor, but just the surface of the face instead; and at a very rough level of detail, as the number of markers that can be placed on a face is very limited.
The result is that the facial deformations have good dynamics, but look very weird at times; and the more extreme the facial deformation has to be, the more obvious this issue becomes.

Quantum Break however is using a more advanced method where two different capture techniques are combined. This is also the approach used in games like Ryse or Infamous Second Son.

First each actor's base facial expressions are captured, things like raising an eyebrow, puckering the lips, or blinking an eye. The breakdown of these expressions is based on the Facial Action Coding System (FACS) developed by clinical psychologists in the `60s, and first adapted for 3D facial animation on the LOTR movies and most notably Gollum. Nowadays it's pretty much an industry standard in VFX and spreading very quickly in game development.
These expressions are created basically as full 3D scans at a very high level of detail, up to millions of polygons, and capture not only the general shape of the face but also the tiniest folds and wrinkles and skin sliding and all. One can also capture the skin color changes from compression where blood is retained or pushed out from certain areas of the face. The scanning itself is usually done by stereo photogrammetry, basically using 15-50 digital cameras to shoot pictures from many different angles; the software then generates a 3D point cloud based on matching millions of points in the images based on color changes. Human skin is fortunately detailed enough for that :)
The entire expression library usually has 40-80 individual scans.

These scans are used to build a facerig where each basic expression can be dialed in at any desired intensity, and the mix of various FACS Action Units can be corrected if they look wrong on their own, too. Games with this approach usually also use a bones based rig, finetuned by a set of corrective blendshapes or morfs (which can be turned off as the character moves away from the camera and accuracy becomes less important); and they also add various wrinkle maps on top of the normal map to create the creases and folds for stuff like raising the eyebrows. These wrinkle maps are also generated from the facial scans.
Also, correctives are only used when necessary, because blendshapes are computationally expensive on realtime systems (no real GPU support) and they also take up a lot of memory. The main facial movement is covered by up to 150+ bones (in Ryse) and the riggers are mostly trying to match the scans by manually adjusting the bones for each expression.

The second capture then is the actual performance, which is usually an imaged based approach, using a face camera hinged from a helmet worn by the actor during the mocap (or in this case P-cap).
Software called a solver analyzes the facial movements (usually with the help of painted dots on the actor's face) and translates it into animation data driving the various basic expressions (AUs). There's no need to capture the actual facial deformation itself during the performance as it's already stored in the face rig itself, generated from the facial scans.
The result is more accurate and better looking deformation, and it's also much easier to manually animate on top to correct or mix or replace, as the animators only need to work with the expressions.

I hope all the above is clear and easy to understand. Also note that I'm in no way trying to bash Quantic's team, but there's no other way to say it, their methodology is outdated and inferior, and it shows. It was easy to please the hardcore audience with their demo mostly because it was an early move, but even today we already have games using more advanced techniques and producing much better animation. Quantic's texture and shader work is still good, so it looks nice in stills, but once those faces start to move the illusion breaks very quickly. If I were them, I'd look into the new approaches ASAP.
 
Here you can see just how few markers are used to capture the facial deformations in Beyond:

Beyound-two-souls-making-of.jpg


It is possible to add some automatic secondary deformation into the facial rigs, but not easy, and if there's no facial expression scanning then the results won't be that good. I'm also not aware of any system that solves 3D mocap data into expressions, so there's probably a lot of guesswork going on. The ingame results also don't look that good.

In QD's defense though, solvers for facial capture are rare - the big VFX studios don't release their in-house solutions and there are very few actual software packages on the market. Most contractor companies prefer to sell a service where they keep their solver to themselves and charge by minute, for example Cubic Motion who did the work on the Spartan Ops CG episodes. Some game studios like 343 wrote their own solver but it takes very good software developers and a lot of time and money.

So Quantic, being a relatively small studio with modest sales compared to the big ones, has probably had to look for a more cost effective solution. Then there's also the PS3's limits on memory and such, making it much harder to build a FACS based rig with corrective blendshapes.
Then again, that's exactly what Naughty Dog had in the Last of Us... And that's quite better too, IMHO. But then again they manually animated everything and they had a lot less work compared to Beyond.
 
but QD definitely has better shading and geometry on theirs characters.
that second screenshot you post looks worse than the other one, perhaps you should of choossen one with unacceptable compression?

better geometry = yes, better shaders = no
you can see instantly she's CGI
 
The facial animation and deformation system is more advanced as well.

Both Beyond and the Sorcerer demo are using only bones to deform the face, driving them by a straight translation of facial marker 3D mocap data. This makes some of the deformations notoriusly hard to reproduce, especially the mouth and the eyelids, even with a large number of bones.
Basically they don't capture the performance itself, the intent of the actor, but just the surface of the face instead; and at a very rough level of detail, as the number of markers that can be placed on a face is very limited.
The result is that the facial deformations have good dynamics, but look very weird at times; and the more extreme the facial deformation has to be, the more obvious this issue becomes.

Quantum Break however is using a more advanced method where two different capture techniques are combined. This is also the approach used in games like Ryse or Infamous Second Son.

First each actor's base facial expressions are captured, things like raising an eyebrow, puckering the lips, or blinking an eye. The breakdown of these expressions is based on the Facial Action Coding System (FACS) developed by clinical psychologists in the `60s, and first adapted for 3D facial animation on the LOTR movies and most notably Gollum. Nowadays it's pretty much an industry standard in VFX and spreading very quickly in game development.
These expressions are created basically as full 3D scans at a very high level of detail, up to millions of polygons, and capture not only the general shape of the face but also the tiniest folds and wrinkles and skin sliding and all. One can also capture the skin color changes from compression where blood is retained or pushed out from certain areas of the face. The scanning itself is usually done by stereo photogrammetry, basically using 15-50 digital cameras to shoot pictures from many different angles; the software then generates a 3D point cloud based on matching millions of points in the images based on color changes. Human skin is fortunately detailed enough for that :)
The entire expression library usually has 40-80 individual scans.

These scans are used to build a facerig where each basic expression can be dialed in at any desired intensity, and the mix of various FACS Action Units can be corrected if they look wrong on their own, too. Games with this approach usually also use a bones based rig, finetuned by a set of corrective blendshapes or morfs (which can be turned off as the character moves away from the camera and accuracy becomes less important); and they also add various wrinkle maps on top of the normal map to create the creases and folds for stuff like raising the eyebrows. These wrinkle maps are also generated from the facial scans.
Also, correctives are only used when necessary, because blendshapes are computationally expensive on realtime systems (no real GPU support) and they also take up a lot of memory. The main facial movement is covered by up to 150+ bones (in Ryse) and the riggers are mostly trying to match the scans by manually adjusting the bones for each expression.

The second capture then is the actual performance, which is usually an imaged based approach, using a face camera hinged from a helmet worn by the actor during the mocap (or in this case P-cap).
Software called a solver analyzes the facial movements (usually with the help of painted dots on the actor's face) and translates it into animation data driving the various basic expressions (AUs). There's no need to capture the actual facial deformation itself during the performance as it's already stored in the face rig itself, generated from the facial scans.
The result is more accurate and better looking deformation, and it's also much easier to manually animate on top to correct or mix or replace, as the animators only need to work with the expressions.

I hope all the above is clear and easy to understand. Also note that I'm in no way trying to bash Quantic's team, but there's no other way to say it, their methodology is outdated and inferior, and it shows. It was easy to please the hardcore audience with their demo mostly because it was an early move, but even today we already have games using more advanced techniques and producing much better animation. Quantic's texture and shader work is still good, so it looks nice in stills, but once those faces start to move the illusion breaks very quickly. If I were them, I'd look into the new approaches ASAP.

Thanks for the infos.
 
The facial animation and deformation system is more advanced as well.

Both Beyond and the Sorcerer demo are using only bones to deform the face, driving them by a straight translation of facial marker 3D mocap data. This makes some of the deformations notoriusly hard to reproduce, especially the mouth and the eyelids, even with a large number of bones.
Basically they don't capture the performance itself, the intent of the actor, but just the surface of the face instead; and at a very rough level of detail, as the number of markers that can be placed on a face is very limited.
The result is that the facial deformations have good dynamics, but look very weird at times; and the more extreme the facial deformation has to be, the more obvious this issue becomes.

Quantum Break however is using a more advanced method where two different capture techniques are combined. This is also the approach used in games like Ryse or Infamous Second Son.

First each actor's base facial expressions are captured, things like raising an eyebrow, puckering the lips, or blinking an eye. The breakdown of these expressions is based on the Facial Action Coding System (FACS) developed by clinical psychologists in the `60s, and first adapted for 3D facial animation on the LOTR movies and most notably Gollum. Nowadays it's pretty much an industry standard in VFX and spreading very quickly in game development.
These expressions are created basically as full 3D scans at a very high level of detail, up to millions of polygons, and capture not only the general shape of the face but also the tiniest folds and wrinkles and skin sliding and all. One can also capture the skin color changes from compression where blood is retained or pushed out from certain areas of the face. The scanning itself is usually done by stereo photogrammetry, basically using 15-50 digital cameras to shoot pictures from many different angles; the software then generates a 3D point cloud based on matching millions of points in the images based on color changes. Human skin is fortunately detailed enough for that :)
The entire expression library usually has 40-80 individual scans.

These scans are used to build a facerig where each basic expression can be dialed in at any desired intensity, and the mix of various FACS Action Units can be corrected if they look wrong on their own, too. Games with this approach usually also use a bones based rig, finetuned by a set of corrective blendshapes or morfs (which can be turned off as the character moves away from the camera and accuracy becomes less important); and they also add various wrinkle maps on top of the normal map to create the creases and folds for stuff like raising the eyebrows. These wrinkle maps are also generated from the facial scans.
Also, correctives are only used when necessary, because blendshapes are computationally expensive on realtime systems (no real GPU support) and they also take up a lot of memory. The main facial movement is covered by up to 150+ bones (in Ryse) and the riggers are mostly trying to match the scans by manually adjusting the bones for each expression.

The second capture then is the actual performance, which is usually an imaged based approach, using a face camera hinged from a helmet worn by the actor during the mocap (or in this case P-cap).
Software called a solver analyzes the facial movements (usually with the help of painted dots on the actor's face) and translates it into animation data driving the various basic expressions (AUs). There's no need to capture the actual facial deformation itself during the performance as it's already stored in the face rig itself, generated from the facial scans.
The result is more accurate and better looking deformation, and it's also much easier to manually animate on top to correct or mix or replace, as the animators only need to work with the expressions.

I hope all the above is clear and easy to understand. Also note that I'm in no way trying to bash Quantic's team, but there's no other way to say it, their methodology is outdated and inferior, and it shows. It was easy to please the hardcore audience with their demo mostly because it was an early move, but even today we already have games using more advanced techniques and producing much better animation. Quantic's texture and shader work is still good, so it looks nice in stills, but once those faces start to move the illusion breaks very quickly. If I were them, I'd look into the new approaches ASAP.

All well and good, but despite the clear advances in animation, the woman in the QB screens has this awfully rubbery looking CG skin (Second Son suffers from that as well) whereas Jodie's looks a lot more convincing and life-like to me. QD pulled of some pretty impressive stuff on Sony's aging warhorse there (heck, I even prefer Beyond's SSS aproximation to Second Son's)
 
Status
Not open for further replies.
Back
Top