Accurate human rendering in game [2014-2016]

[Billy Idol]

I think that facial animation is one of the most important things...at least for me to make a character believable.

The movement of the mouth and lips in combination with the teeth position seems to be the most difficult thing and typically this breaks the illusion for me.

 

[Shifty Geezer]

wow the old dude, top right looks very good, is this a real existing game? or just a cutscene.

It's from the realtime PS4 demo by Quantic Dream, using a motion captured, scanned actor.

And yeah, hair has to be individual strands, with very high levels of supersampling AA and there's no shortcut or substitute for that.

I'm not sure of that. I think an alternative line rendering algorithm would be faster and more accurate. The problem is fitting that algorithm to GPUs - a monster CPU or Cell-like processor would be a more obvious fit. But as GPUs become increasingly clever, alternative rendering (the dream of Software Rendering!) for different elements makes far more sense. eg. I know at least one game out there uses line drawing for power lines instead of polygons, giving crisp, correct, alias-free cables. And with deferred renderers, swapping rendering technique for different layers and compositing is pretty straight-forward to incorporate. So in future, I'd be pleased to see a 'hair buffer' drawing various strands with a completely different algorithm to polygon rasterising.

 

[Deleted member 86764]

Each character's base body model was built from more than 100,000 polygons, plus more than 300,000 for clothing alone. Each character's model bears 60,000 hairs, each of which were separately and fully animated and rendered.

I wonder how close it'd be possible to get on a 1-to-1 fighter with minimal scenery. Current-gen Soul Calibur game could be very interesting.

 

[Laa-Yosh]

The problem with line rendering for hair could be shadowing, self shadowing in particular. Shading might also be problematic, the color of hair is coming mostly from anisotropic specular highlights.

 

[Shifty Geezer]

I'm sure it could be done somehow.

 

[Karamazov]

who remembers this old ps2 demo

100 000 polygons at 60fps ?

http://youtu.be/Hd1zR-nxcPE?t=1m28s

anyway no ps2 game reached this quality for a character.

 

[babcat]

human skin is what needs to be photo realistic most in my opinion. When I look at games or tech demos, if the skin is not nailed it disrupts the illusion that I am looking at real people. The only two faces that look virtually real to me is the Dark Sorcerer and the Activision demo. There is currently not a next gen game in which the characters look real to me. The characters in Second Son scream plastic, for example. What Im looking forward to is the first next gen game that shoots for realism and nails human skin. I wish The Order would shoot for realism, but the game seems stylized. Is this to save money? They are already producing a rich and detailed world, so I don't see why it would cost so much more to make the characters look real. It would make the game go from looking like a very very well made animation to looking like an actual movie.

 

[HTupolev]

I wish The Order would shoot for realism, but the game seems stylized. Is this to save money?

It's probably partly because it looks awesome, and partly because they want it to run on a PS4.

Even as-is, I'm kind of surprised that you'd bring it up as an example of stylism in the context of video games. Sure, it's not aiming for pure hyperrealism, but it's pretty close by video game standards.

 

[babcat]

It's probably partly because it looks awesome, and partly because they want it to run on a PS4.

Even as-is, I'm kind of surprised that you'd bring it up as an example of stylism in the context of video games. Sure, it's not aiming for pure hyperrealism, but it's pretty close by video game standards.

first of all, I don't see how it would take a huge amount of extra processing power to make the game look fully realistic. I don't see how it would increase the polycount, texture budget, or drawcalls much. If anything, it may require more complex shaders. However, their skin shading is supposed to be very modern already, so what would it take to boost it to the level of the Activision demo? If the stylism is intentional, maybe just removing it and altering the shaders would work with minimal additional power needed.

Secondly, I am currently thinking that modern consoles should be able to pull off photorealistic skin and characters. Look at the real time Dark Sorcerer demo. It looked close to what Activision has achieved with a high framerate, animation running, other special effects, and multiple characters on screen. And this was with an engine designed for the PS4 with next gen assets thrown in. If this is possible unoptimised for the PS4 at 1080p, just imagine what could be acieved with an enigine made to get the most from the PS4.

Finally, if more power is needed, the resolution should be decreased to 720p. If The Order is stylized due to a lack of power, which I dont think is a big issue, there is more power available if the resolution is reduced. Personally, I watch 98 percent of all my media at that resolution. It makes movies look great. If the goal is realism, I think the lowest resolution - down to 480p - should be used. I would be ten times more likely to buy the order if it was truly photorealistic at 720 P than 1080 p like the game looks now.

 

[HTupolev]

first of all, I don't see how it would take a huge amount of extra processing power to make the game look fully realistic.

Where are you drawing the line for "fully realistic"?

Because if you mean that a human cannot distinguish it from a decent-quality live-action capture of reality, you'd need a TON of extra processing power. Like, real-time graphics aren't even in that ballpark yet, for many sorts of scenes.

 

[babcat]

Where are you drawing the line for "fully realistic"?

Because if you mean that a human cannot distinguish it from a decent-quality live-action capture of reality, you'd need a TON of extra processing power. Like, real-time graphics aren't even in that ballpark yet, for many sorts of scenes.

I disagree. I think we can get very close today, and that the movement of characters is not what needs the most improvement. The two issues that need work are skin shading and lighting. No actual game for the PS4 or XBox One have nailed these issues. However, the Dark Sorcerer demo came close.

I would consider the line for photorealism to be an Activision head on the Dark Sorcerer in an actual game made to look as real as possible without any stylism. Right now, we have elements of everything we need in different games and tech demos. We just need to bring it all together. I personally think The Order could be such a game if the stylism was dropped and a realistic mode was used.

For example, look at the PS4 demo of The Division. The back grounds, particles, textures, and object density make everything but the skin of the characters look great. The skin looks plastic and fake. Why cant a game like this - even if it would need to run at a lower resolution- use activisions tech?

 

[babcat]

Just imagine The Order with heads as real as this....

 

[Laa-Yosh]

Like, real-time graphics aren't even in that ballpark yet, for many sorts of scenes.

Closest to a convincing CG human I've seen so far is Bruce Banner in Avengers in the shot where he transforms into the Hulk and hits the Chitauiri monster.

ILM found that it was easier to do a 100% CG transformation instead of trying to go from live action, and they've already scanned and modeled Mark Ruffalo down to nose and ear hairs to get a good basis for the creature anyway, so they were half way there from the start. I've only learned it at their presentation, but then again it was a very quick shot and the character didn't do much acting in human form.

A completely convincing CG human is still beyond the horizon IMHO. Expecting it to happen in real time in these days is hopeless.

 

[babcat]

Closest to a convincing CG human I've seen so far is Bruce Banner in Avengers in the shot where he transforms into the Hulk and hits the Chitauiri monster.

ILM found that it was easier to do a 100% CG transformation instead of trying to go from live action, and they've already scanned and modeled Mark Ruffalo down to nose and ear hairs to get a good basis for the creature anyway, so they were half way there from the start. I've only learned it at their presentation, but then again it was a very quick shot and the character didn't do much acting in human form.

A completely convincing CG human is still beyond the horizon IMHO. Expecting it to happen in real time in these days is hopeless.

The issue is how convincing, especially to non experts. The real time CGI of the Dark Sorcerer looked 99% convincing to me. It was not perfect, but the fact it was unoptimized and just next gen assets thrown into a PS3 engine makes me hopeful we can see some very convincing real time CGI. Will it be perfect? Of course not. But if the Dark Sorcerer already looks this good, I think it can get much better with current hardware. This is of course if the trend of making almost every game stylized ends.

 

[DieH@rd]

Upcoming EA's UFC has awesome modeling and simulation of muscles/sub surface scatter/sweat/collisions/stretching... Really HQ stuff.

They announced it to be 1080p30 on PS4, which is strange for fightning game, but at least they are really pushing the visuals.


BTW, in this game and EVERY OTHER, animating breathing of characters is ignored. I am slowly getting pissed at devs who promote "great performance capture" sessions, but they are totally missing breathing, moving of the chest, stomach, shoulders and very slight but important movement of the nose when actor is breathing hard... Good breathing animation really make Avatar stand out from current videogame efforts.

 

[KKRT]

Upcoming EA's UFC has awesome modeling and simulation of muscles/sub surface scatter/sweat/collisions/stretching... Really HQ stuff.

They announced it to be 1080p30 on PS4, which is strange for fightning game, but at least they are really pushing the visuals.


BTW, in this game and EVERY OTHER, animating breathing of characters is ignored. I am slowly getting pissed at devs who promote "great performance capture" sessions, but they are totally missing breathing, moving of the chest, stomach, shoulders and very slight but important movement of the nose when actor is breathing hard... Good breathing animation really make Avatar stand out from current videogame efforts.

The physics of collisions and muscle displacement is amazing in this game, i wonder when we will get similar quality in normal games. We still dont have some of physics simulations from past gen sport games in current games ;\

 

[Davros]

Cloth is not really the shading but the simulation - very processing intensive and usually with lots of tiny itnersection errors that require manual fixing even in movie VFX. It's also something you can't really accelerate with GPUs or just bandwidth, .

You know cloth simulation is bulletpoint for nvidia physx

 

[Jedi2016]

You know cloth simulation is bulletpoint for nvidia physx

But it's a very simple implementation, nowhere near the level of clothing like Laa-Yosh is talking about. PhysX can't do things like trousers, and how they ride and kink in the back of your knee when you walk. It tops out at things like thick leather coats (just the tail, not the sleeves) that are very stiff and can use very rough collision meshes to help them move.

 

[MJP]

first of all, I don't see how it would take a huge amount of extra processing power to make the game look fully realistic. I don't see how it would increase the polycount, texture budget, or drawcalls much. If anything, it may require more complex shaders. However, their skin shading is supposed to be very modern already, so what would it take to boost it to the level of the Activision demo? If the stylism is intentional, maybe just removing it and altering the shaders would work with minimal additional power needed.

Secondly, I am currently thinking that modern consoles should be able to pull off photorealistic skin and characters. Look at the real time Dark Sorcerer demo. It looked close to what Activision has achieved with a high framerate, animation running, other special effects, and multiple characters on screen. And this was with an engine designed for the PS4 with next gen assets thrown in. If this is possible unoptimised for the PS4 at 1080p, just imagine what could be acieved with an enigine made to get the most from the PS4.

Finally, if more power is needed, the resolution should be decreased to 720p. If The Order is stylized due to a lack of power, which I dont think is a big issue, there is more power available if the resolution is reduced. Personally, I watch 98 percent of all my media at that resolution. It makes movies look great. If the goal is realism, I think the lowest resolution - down to 480p - should be used. I would be ten times more likely to buy the order if it was truly photorealistic at 720 P than 1080 p like the game looks now.

I don't really want to comment on the stylization, performance, our characters vs. Activision's, etc. I've had enough of my comments end up as fodder for NeoGAF.

What I will say is that sometimes even small visual improvements are the results of tremendous amounts of time and effort. Sometimes it may sound like something should be as simple as me or another programmer opening up the shader code and hacking away until it looks better, but in reality almost everything to do with skin/faces/materials ends up being part of a rather complex authoring pipeline. Some things might require the rigging department to set up their rigs differently so that we can hook things up to joint animations. Some things might require iterating with our material artists for weeks to come up with a UI and workflow that allows them to quickly make the changes for all of our principle characters. Sometimes we're just so bogged down with actually making a game playable that we have to put off making quick shader fixes until a later point. No matter what, we have to be very careful about how we spend our time. We always need to make sure that we're putting our resources towards whatever should be the highest priority given our overall project goals and time/budgetary contraints. It's complicated and frustrating at times, but that's what engineering is all about.

 

[Inuhanyou]

I wonder if Agni's philosophy has surpassed Spirit's within. It certainly looks like it in terms of things on on on screen...