I have been trying to follow the latest PlayStation 4 and Xbox games to determine if any of them have been able to produce in game, real time, and near photorealistic human skin. What I'm talking about is the type of skin Activision has been able to produce in tech demos. from what I have seen so far, no game comes close. How far away do you think we are from photo realistic skin in games? Are the shaders so expensive that they cannot be used in this generation of consoles? For example, could the next Killzone game actually have characters with photorealistic skin? Personally, I think the human characters or what looks the worst and most games. Buildings, cars, and environments are all looking pretty good. If a game like Killzone Shadowfall could actually have realistic looking characters, I think that it would add to the immersion.
Accurate human rendering in game [2014-2016]
- Thread starter babcat
- Start date
- Status
TheWretched
Regular
I would say it depends on the developer. Just look back to games like Beyond. Now... it's a fairly limited experience and very linear, but... it's still running on a PS3.
Dominik D
Regular
Human skin is not only about surfaces. Right now it has to do with animation more than anything else. We're looking a lot at faces and our brains are wired to perceive a lot of subtle things that are not easy to emulate in games. Sure, good shaders and material data are important (not only for skin but also eyes) but you've got to animate a lot of tiny things to escape the uncanny valley for more than few seconds. On top of that you need convincing hair simulation which is also super difficult. So nailing human appearance this gen is IMO possible, as long as that's band Caucasian guy with no facial hair. 
The stills of the Order:1886 are exceptional IMO. eg. The female protagonist is pretty darned photorealistic in those screenshots. Although it's more CG like in game. In motion, it might be a different story though. As Laa-Yosh has educated us all over the years, there are lots of fine details that need to be captured, and a lot of that has been beyond realtime technologies. We could have The Order's rendering coupled with LA Noire's performance recording (although that may not be possible as LA Noire effectively slaps a video feed on a face mesh AFAICS), but it'd be great art of limited scope.
So shaders, yes, I think it's possible. Realistic people overall - maybe not so much, because of everything else the game has to do. The budget won't be there to blow it all on the characters.
So shaders, yes, I think it's possible. Realistic people overall - maybe not so much, because of everything else the game has to do. The budget won't be there to blow it all on the characters.
LA Noire facial capture tech:
http://www.youtube.com/watch?v=aL9wsEFohTw
http://www.youtube.com/watch?v=aL9wsEFohTw
The human face is a problem because of how our perception is so tuned through evolution; and yet it's also almost completely instinctive, so it takes decades of work and experience to gain a proper understanding of its workings.
Skin is mostly about a nice balance of translucency and reflectivity.
Most of our tissues are tiny packs of water enclosed in a transparent membrane, so a lot of light passes through and bounces around and then exits the surface quite far from where it has entered; but various wavelengths are absorbed in the process so the color changes a bit too.
Even VFX-level skin shaders using raytraced subsurface scattering with a lot of different textures controlling the various parameters are just approximations. Some of them are quite good (Weta has used a new one since Prometheus and the Hobbit movies) but we haven't seen them tested on a virtual human character yet. Still, the Avatar sequels are probably going to try to push the tech as far as possible; but Cameron's preference for blue skin tones will keep it from becoming completely convincing.
Reflectivity is about multiple layers - the skin itself and the oily layer on top of it. It's also about the scattering of the reflection itself, which is determined by the surface irregularity - the micro level structure of the upper layer of the skin, in other words the bump/displacement/normal map.
There's a lot of measured data on both the transclucency and the reflectivity, so even game devs can get quite close with their approximations by testing them against the research results. But approximations will never be completely correct.
Another important component is the "peach fuzz", the tiny white translucent hairs on people's faces that tend to pick up rim lights and soften up the look of the face. It's a relatively cheap extra in VFX compared to the complex raytraced skin shaders, but a near impossible nightmare for games (rendering thousands of less than a pixel thick strands).
Facial movement is another complex issue, worth it's own post - but this is about the skin only so I'm not gonna get there.
Also, LA Noire had terrible skin shading, and the facial animation was basically a 3D version of recording your actor with a VHS camera and playing it back in the game.
Skin is mostly about a nice balance of translucency and reflectivity.
Most of our tissues are tiny packs of water enclosed in a transparent membrane, so a lot of light passes through and bounces around and then exits the surface quite far from where it has entered; but various wavelengths are absorbed in the process so the color changes a bit too.
Even VFX-level skin shaders using raytraced subsurface scattering with a lot of different textures controlling the various parameters are just approximations. Some of them are quite good (Weta has used a new one since Prometheus and the Hobbit movies) but we haven't seen them tested on a virtual human character yet. Still, the Avatar sequels are probably going to try to push the tech as far as possible; but Cameron's preference for blue skin tones will keep it from becoming completely convincing.
Reflectivity is about multiple layers - the skin itself and the oily layer on top of it. It's also about the scattering of the reflection itself, which is determined by the surface irregularity - the micro level structure of the upper layer of the skin, in other words the bump/displacement/normal map.
There's a lot of measured data on both the transclucency and the reflectivity, so even game devs can get quite close with their approximations by testing them against the research results. But approximations will never be completely correct.
Another important component is the "peach fuzz", the tiny white translucent hairs on people's faces that tend to pick up rim lights and soften up the look of the face. It's a relatively cheap extra in VFX compared to the complex raytraced skin shaders, but a near impossible nightmare for games (rendering thousands of less than a pixel thick strands).
Facial movement is another complex issue, worth it's own post - but this is about the skin only so I'm not gonna get there.
Also, LA Noire had terrible skin shading, and the facial animation was basically a 3D version of recording your actor with a VHS camera and playing it back in the game.
True dat. The most realistic skin shader in the world won't help if it moves like a robot. Infamous SS is a pretty good example. The skin on the humans looks good, but it's just textures near as I can tell, there's not much actual shading going on (things like subsurface scattering, for example), but the animation is top-notch and you're able to read a lot just from the characters' facial expressions. The lip-sync isn't as advanced as LA Noire, but the overall expression of the face is better. For example, when a character smiles (earlier in the game, if you've seen it you probably know what I'm talking about), her whole face changes, far beyond just her mouth, all those little subtle things are happening. LA Noire could have done that, I'm sure, but they were limited by tech to a lower poly count in the faces and fewer control points for animation.
That's not true, they actually use the same SSS technique that we use on The Order. You'd really notice if they didn't. Without SSS, any small details from a normal map just ends up looking harsh and "rocky".
Seems like a lot of seventh-gen games were caught in a ballpark where they could target some interesting shading models and throw in some specularity, but weren't quite to the point of "completing" the look with appropriate scattering and such. The result being that characters looked like they were made of some kind of waxed opaque material. Which worked for some not-quite-realistic visual styles (i.e. Final Fantasy), but in some more realistic styles looked incomplete and harsh. Which probably helped the extended popularity of that Silent Hill 3 gif; some decent textures and a fairly basic diffuse response is obviously doing a lot less, but in a simpler style it doesn't feel like it's missing anything.
I would disagree. The skin in Ryse did not look photorealistic to me. I wonder what their skin shader is not doing compared to Activisions.
I've taken a look at Infamous SS and I believe their specularity isn't right... it's as if they're also missing skin pore details as well.
Their facial capture tech is very complex though, seems to be the most advanced I've seen so far. Strange that I still find the actual results in Last of Us more convincing, even though that's hand animated and doesn't use any facial scans at all.
Their facial capture tech is very complex though, seems to be the most advanced I've seen so far. Strange that I still find the actual results in Last of Us more convincing, even though that's hand animated and doesn't use any facial scans at all.
Beyond is also my first thought.
Yeah, stuff that realistic eyes need in our experience:
- subsurface scattering for the whites
- depth and refraction for the iris
- proper shadowing from the eyelids
- the "sparkles" in eyes are actually some quite disgusting stuff, sort of a very bumpy fluid that gets thicker near the eyelids, creating some unique looking reflections
They seem to have come up with some good approximations for these indeed.
So i made a quick comparison just to show how much the eyes can make or break the character model. I compare Marius to Lafayette and the Sorcerer as a bonus. While the Order character models look more detailed than the Ryse models, the eyes are just not up to par. If RAD manages to incorporate the same technique Crytek did, just imagine how much better the models would look.
I did the second comparison for fun just to show how close we are getting to the Aki render (besides the hair).
I did the second comparison for fun just to show how close we are getting to the Aki render (besides the hair).
It's funny that you included Aki, because there are a couple of interesting technical details there...
- Back at that time there was no SSS implementation, and very little understanding of human skin shading at all. Square used a couple of tricks but the lack of proper translucency was a big hurdle. No wonder that the most realistic character was Dr. Sid, whose old skin required very little of it.
- There was also no proper sculpting tool to add the necessary detail, thus the shapes and forms were a little rough and artificial. Also, scanning wasn't really an option either as there were very few laser scanning facilities, whereas nowadays a few dozen DSLRs and Agisoft Photoscan can get you pretty good results - and both Ryse and Order characters were relying on such tools heavily.
- On the other hand, hair rendering is still not using hair strands in game engines, but only textured polygons as a shortcut. Aki's hair is still way over the horizon. Same goes for the displacements, although it wasn't as dominant - because of the lack of scanned/sculpted source meshes - on the FF characters either.
- Back at that time there was no SSS implementation, and very little understanding of human skin shading at all. Square used a couple of tricks but the lack of proper translucency was a big hurdle. No wonder that the most realistic character was Dr. Sid, whose old skin required very little of it.
- There was also no proper sculpting tool to add the necessary detail, thus the shapes and forms were a little rough and artificial. Also, scanning wasn't really an option either as there were very few laser scanning facilities, whereas nowadays a few dozen DSLRs and Agisoft Photoscan can get you pretty good results - and both Ryse and Order characters were relying on such tools heavily.
- On the other hand, hair rendering is still not using hair strands in game engines, but only textured polygons as a shortcut. Aki's hair is still way over the horizon. Same goes for the displacements, although it wasn't as dominant - because of the lack of scanned/sculpted source meshes - on the FF characters either.
So i made a quick comparison just to show how much the eyes can make or break the character model. I compare Marius to Lafayette and the Sorcerer as a bonus. While the Order character models look more detailed than the Ryse models, the eyes are just not up to par. If RAD manages to incorporate the same technique Crytek did, just imagine how much better the models would look.
I did the second comparison for fun just to show how close we are getting to the Aki render (besides the hair).
The sorcerer and the woman from the order with scars on her neck look the best. However, none of them look as good as the woman in the activision demo. Is that shot of the woman from the order in game and real time?
Direct or on video?
Skin pores are there, but they seem.. fuzzy, shallow, as if the normal map for the pores was too low-res (odd given the rest of it). Really only visible in a few closeups of Delsin with the light just right.
- Status
