Movie Reviews 2.0

My point was that

with the 2D enhancements in LOLA's work on the Marvel movies, you still have the actual footage of the actor there in the shot.

Whereas with a 3D character, the performance recorded on set or in a mocap studio is analyzed and translated by software, manually modified by animators, and it is driving a 3D model with artificially created facial deformations (like, through blendshapes).

So in the first case, you start with an existing footage of the actor, you already have a performance there, and you build on top of that. Erase facial wrinkles, remove various skin blemishes, white hairs etc. (or in case of Vision and old Peggy, add some other stuff). But in the second case, you start with an empty frame (okay, maybe you have to paint out the actor first) and then have to add everything from scratch. So there's a fundamental difference here, as there still is something real with the 2D approach in the final shot.

Maybe I should have used 'footage' or something else though, to make my point more clear ;)
 
On rogue one

spoilers
I think the movie was very good, but they really pushed the plot armor in this one, it was almost a miracle for various events to take place and it is almost unbelievable that the plans got to the start of episode iv like that, way way too unbelievable stuff, this was like dodging bullets without matrix explanation, I know there's the force and stuff but here it felt like the characters made it through by, oh I forget the saying, if anyone recalls it I'd be grateful to know(it's either skin teeth hair or something..... edit: remembered while writing by the skin of their teeth, though that sounds like a nonsense phrase).

In any case as regards the characters, I'm sorry but look at the Princess she's only a few seconds on screen and you can totally tell she's CG. People may disagree but the CG Arnold from salvation was more convincing, and that's old hat and many said bad cg. The final fantasy kingsglaive movie, while most of the time you can tell it's cg, there are moments where it can be very convincing and easily fool one, yet that likely had lower budget.

Sure some might say you don't have state of the art 3d scans, but there's plenty of video of the actors, likely film footage from various angles. A good artist can capture realism, and yes it need not necessarily look exactly like the original actor for it to be convincing and realistic. One would think they'd have plenty of unreleased footage that would enable photogrammetry to create a reasonable model that could be touched up by the artists, but it seems not. At times the old guy did look convincing, but like 98+% of the time he looked quite uncanny and unreal. Perhaps it was on purpose, but there's no excuse with the Princess, imho.
 
RIP Carrie Fisher. She was possibly my first crush (I was nine) and I've always found her to be a wonderfully entertaining character.

2016. Seriously 2016 can fuck off.
 
I'm sorry but your assumptions are wrong. Recreating someone's realistic likeness from old footage is probably the hardest challenge in CGI. ILM has some of the best artists in the industry, and arguably the best tech - especially considering the high-end research work from Disney's institutes - so this is actually the state of the art in this field today.
 
I don't think they'd go that far. Even if Episode VIII gets compromised in any way. This thing is just too new and even corporations can have respect.
 
It's entirely possible she (and others like Ford) may have given permission for such a thing during the work leading up to FA. just in case.
 
The controversy would be too much, so I don't expect Disney to make this move. It's just too early.
 
I'm sorry but your assumptions are wrong. Recreating someone's realistic likeness from old footage is probably the hardest challenge in CGI. ILM has some of the best artists in the industry, and arguably the best tech - especially considering the high-end research work from Disney's institutes - so this is actually the state of the art in this field today.
Perhaps recreating it to look exactly like the old actor is still hard, though one would think photogrammetry could help. But in any case you don't have to recreate the exact likeness to make a convincing character, it would just look like an uncannily similar stunt double, close but someone different for those who've seen the prior works.

The thing is you can't tell me that the best of the best for humanoid characters is of so uncanny results unless you've got the living actor and can scan them right then and there for the movie. Again you don't have to exactly replicate the previous actor, just pass of as human, not some creature, unless that is the intent.
Heck at least one of the actors was alive during filming and could be de-aged. Tron's clu seemed more convincing, minus that ending scene, you could tell it was cg, but the quality was quite good.

As I said the terminator face from salvation also looked very good and convincing, even superior to genisys.

Wouldn't surprise if characters with significant facial structure change or entirely fictional humans end up looking more convincing in upcoming games within the next few years, and doubt drastic changes would have taken place on the underlying tech.
I don't think they'd go that far. Even if Episode VIII gets compromised in any way. This thing is just too new and even corporations can have respect.

Fast and the furious already set the stage. And didn't that come quite close to the actor's death?
and it was very good quality, very convincing

ps wow couldn't even tell on gladiator when I watched it!

PPS
Salvation, Gladiator, Fast and the Furious, and a bit less Tron legacy, some of that might have been called bad cg, but it was quite convincing and didn't take you out of the movie.
 
Last edited:
I read that Carrie had already finished filming for Episode 8, so this would only be topical for Episode 9 and that is some time away with likely a smallish part anyway?. I think CGI could be an option. I think it can be seen as respect also to include a person into a movie like that.

Fast and The Furious saw Paul Walker's death while the movie was being filmed. He had shot all the expensive action scenes already, but was missing quite a few conversation scenes. Dropping him from the movie simply wouldn't have worked. They did a good and throughout job with that.
 
Last edited:
The tech in rogue one sets an interesting er something
The future of acting could be you spend a few days in the studio being captured and thats it they dont need you any more
 
I wanted to try to give a thorough response, because it seems that there is some confusion and some misunderstandings here. Also, I think we're beyond the need to use spoilers, it'll make everything easier and anyone who hasn't seen the movie yet is probably not worried about this detail - which is all over news sites anyway ;)

Perhaps recreating it to look exactly like the old actor is still hard, though one would think photogrammetry could help.

Photogrammetry needs really high quality imagery to create good results. Think 50-100 8K digital photographs with perfect focus and even lighting, using the same type of camera with the same settings. Without that, the pattern matching across the images won't have anything to work with, and all you'll get is a big lumpy mess.
More importantly, matching the facial expressions of your CG model is where the entire thing gets really complex. It would be easy to create a convincing static image, but the character also has to talk and act.
So it wasn't possible to use photogrammetry in this case and that made everything much much more complex from the start.

But in any case you don't have to recreate the exact likeness to make a convincing character, it would just look like an uncannily similar stunt double, close but someone different for those who've seen the prior works.

I'm not sure what your point here is meant to be. The only way to do this was to aim for a 100% likeness of the actor, or forfeit CG completely and only use a lookalike actor. Disney made a choice to go all the way, because they must have felt that the time was right - they had the tools and tech and experience and the story has justified their use.

The thing is you can't tell me that the best of the best for humanoid characters is of so uncanny results unless you've got the living actor and can scan them right then and there for the movie. Again you don't have to exactly replicate the previous actor, just pass of as human, not some creature, unless that is the intent.

This is a point that I've already talked about, but let's go through this again...

First of all, Tarkin has worked for a very large part of the audience. I immediately realized that it wasn't working well and wasn't able to fool me, but I'm not a part of the majority here. I'm also specialized in facial modeling and animation, and supervised dozens of scan-based CGI human characters at work, so I'm probably quite sensitive to a lot of this stuff. But the same is true for most of you guys here at B3D, being 3D graphics enthusiasts; so you guys should also be counted as experts here, unlike the average movie goer.

Also consider that only hardcore SW fans would remember Tarkin's character, and have a fresh and clear mental image of how he is supposed to look like. These people would also know that the actor has passed away a long time ago so it has to be an artificial creation. That will immediately mess with the results, as sight is not just a sensory function - it involves a lot of brain work to interpret the image. On top of that, the ability to thoroughly read another human's face is a strong evolutionary advantage, so we all have a lot of unconscious processes at work; most of which isn't understood yet.

So, I'm quite convinced that even most of us here could be easily tricked by a CG human if we were seeing it under the right circumstances.
First of all, we shouldn't have any hint or knowledge of it being there at all. Also, the more 'generic' the face and the less complex the performance would be, the more convincing it'd get. Peter Cushing had an extremely characteristic face, almost caricature-like, whereas Walker for example had a more usual "handsome guy" look. Which is also why the two actors were usually getting the types of roles they had in these two movies as well in the first place.

Heck at least one of the actors was alive during filming and could be de-aged.

I've already explained somewhere that "de-aging" is an entirely 2D process that has a very limited range. You can't turn a 60-years old woman into a 19-years old younger self, the changes in the face, the expressions, the skin etc. are too extensive. Think about this process as something very similar to Photoshop retouching - you can remove tiny wrinkles and even some larger folds, skin blemishes and unwanted colors - but you can't re-sculpt the entire face.

Tron's clu seemed more convincing, minus that ending scene, you could tell it was cg, but the quality was quite good.

Perhaps you should see that movie again ;) because CG Bridges rarely works as well as Tarkin does in most of his scenes. If we're at Tron BTW then let me remind you of that one short scene with the young Boxleitner - granted, there was an interlace filter and lots of grading on top of the image, but that character worked much much better.

As I said the terminator face from salvation also looked very good and convincing, even superior to genisys.

That I completely disagree with :) Genisys Arnold is really, really good, the other one absolutely not - at least IMHO....

Wouldn't surprise if characters with significant facial structure change or entirely fictional humans end up looking more convincing in upcoming games within the next few years, and doubt drastic changes would have taken place on the underlying tech.

Again, sorry but I don't really understand this...

Fast and the furious already set the stage. And didn't that come quite close to the actor's death?

As other's have already pointed out, FF7 was a very different case - they've already had most of a high-budget movie in the can. So it was a question of forfeiting most of that material and a large investment, or finding a solution that would still honor the deceased actor and his family, but also allow them to complete the film.

ps wow couldn't even tell on gladiator when I watched it!

Gladiator was a very simple 2D compositing trick, I think the footage is actually re-used from the scene where they first visit the Colosseum. Completely unrelated to CG Tarkin, it's actually a parallel with Red and Gold Leader's scenes.
 
Without that, the pattern matching across the images won't have anything to work with, and all you'll get is a big lumpy mess.
I've heard of photogrammetry being used with 4k camera on drones, which obviously low cost ones aren't that smooth. And actors are much closer to the film cameras that were used for the originals.

Also the microsoft phone software that's apparently working in their demos does provide 3d reconstructions with a phone camera


I've also heard of neural net solutions extracting 3d models even from a single photo. Such can be further retouched, and used as a base. Perhaps it hasn't been done, but it would seem that with a few high def photos from a few angles, a similar deep learning solution could be developed that created an even higher quality output from them.

The only way to do this was to aim for a 100% likeness of the actor, or forfeit CG completely and only use a lookalike actor.
Yeah you can aim for 100% likeness, but once you're seeing you've gotten close enough if you can't get closer, you should focus on increasing the realism of the model. For a few scenes it was quite convincing, but for most others it wasn't.
You can't turn a 60-years old woman into a 19-years old younger self, the changes in the face, the expressions, the skin etc. are too extensive.
Here's the thing even if she looked mid 30s there's not that much difference with good makeup.

Also what about ant man? Michael Douglas was what like 70? they turned back the clock by several decades on him.
Again, sorry but I don't really understand this...
The thing is I Look at something like the Death Stranding trailer, and some of the faces are extraordinarily convincing at times. Very likely not all characters are going to be 100% based off of real scans, yet I don't doubt that some are going to be quite convincing in this and other games. The budget far lower, the rendering time far lower, yet extraordinarily convincing. I can't believe that without actor scans, it'd changed from passing for real to looking totally uncanny.

Oh and also forgot the following tech, which if done not in real time and with touch ups with higher quality source should prove convincing
 
Last edited:
I've heard of photogrammetry being used with 4k camera on drones, which obviously low cost ones aren't that smooth. And actors are much closer to the film cameras that were used for the originals.
Also the microsoft phone software that's apparently working in their demos does provide 3d reconstructions with a phone camera

The key word is precision. For a realistic face, you need that to be in the sub-millimeter range; your examples range between a few millimeters to several centimeters. That's nowhere near what you would need for a convincing human likeness. Trust me, I've seen scan data of all kinds, and have used it in professional and hobbyist projects as well :)

I've also heard of neural net solutions extracting 3d models even from a single photo. Such can be further retouched, and used as a base. Perhaps it hasn't been done, but it would seem that with a few high def photos from a few angles, a similar deep learning solution could be developed that created an even higher quality output from them.

These kinds of tech would only work for a highly compressed low-res youtube video. Nowhere near enough for a movie character - fortunately.

Yeah you can aim for 100% likeness, but once you're seeing you've gotten close enough if you can't get closer, you should focus on increasing the realism of the model. For a few scenes it was quite convincing, but for most others it wasn't.

ILM had a sort of stumble in the early 2000s, when Weta was pushing the envelope a lot more; but they've started to recover spectacularly with Pirates 2 (Davy Jones was pretty damn convincing) and they also have bleeding edge research support from Disney's labs all around the world. Uniting all these facilities under the same flag (sorry about the pun) is starting to show.

I wasn't convinced on my first seeing of the movie, but I'm willing to look beyond my impressions, and apparently Tarkin has worked well enough for most viewers. That's what matters in the end.

Here's the thing even if she looked mid 30s there's not that much difference with good makeup.

That's the point. Old age changes a lot about the face, the flexibility of tissues, the shape of the eyes and lips and such, there is a point where it becomes impossible to go with just retouching.

Also what about ant man? Michael Douglas was what like 70? they turned back the clock by several decades on him.

Notice that he's not depicted in his early 20s, and also that he is in a remarkably good shape for his age. Fisher has aged a lot more, let's not be hypocritical about it - decades of drug and alcohol abuse leave a mark.
With all that said, Douglas was an example of the approach taken to its extremes, and they were also lucky with the source material. They could not have done it for the CG Leia.

The thing is I Look at something like the Death Stranding trailer, and some of the faces are extraordinarily convincing at times. Very likely not all characters are going to be 100% based off of real scans, yet I don't doubt that some are going to be quite convincing in this and other games. The budget far lower, the rendering time far lower, yet extraordinarily convincing. I can't believe that without actor scans, it'd changed from passing for real to looking totally uncanny.

Context helps a lot. Those faces wouldn't work nearly as well if they were inserted into live plates of real actors - you'd spot the differences immediately.[/QUOTE][/QUOTE]
 
These kinds of tech would only work for a highly compressed low-res youtube video. Nowhere near enough for a movie character - fortunately.
I think some of those behind this paper are students. When a few students did similar deep learning on voice recognition, the real experts got in and managed to thoroughly beat the state of the art with other techniques and decades of research. Similar has happened in computer vision. I do not believe this technique has reached its limits, and similar could in the future create realistic models. But it seems such technology is not available behind closed doors yet.

I don't see why you say fortunately, I can't wait for the moment when you can create photoreal video featuring anyone even from a bit of footage. I'm very intrigued by the research where one network tries to fool the other and the other gets better and better at detecting the fakery from the real, causing the faker to get ever better. We've already seen similar deep learning approaches surpass human level performance at some tasks. Imagine if one such became superior to human at telling fake from real, and the other could trick it still, or if it became so good that it was virtually impossible to tell by any means that the faked footage is fake. Video evidence would no longer mean anything.


Context helps a lot. Those faces wouldn't work nearly as well if they were inserted into live plates of real actors - you'd spot the differences immediately
Here's the thing that's supposedly real time, if we had the original source assets and were going prerendered more advanced lighting, shading, etc could be provided.

Mads Mikkelsen looks extraordinary, and if the source assets with better shading and lighting were done prerendered that'd be incredible.

Also with Guillermo del Toro he's not even doing the voices, I think he only provided the base model scan, not sure. Would have to check if Mad Mikkelsen is doing facial motion capture, or if he only provided a base scan and the motions are artist generated perhaps with someone else's motion capture data.
 
if it became so good that it was virtually impossible to tell by any means that the faked footage is fake. Video evidence would no longer mean anything.

Consider what 2016 has shown us about this. I'm afraid that the result wold be exactly the opposite of what you think - there's a considerable amount of people all around the world who would gladly accept such fake video footage as real. I mean people became millionaires (!) by running fake news websites about the US election and the candidates; there was a guy who actually believed that certain politicians were performing child sacrifices in the cellar of a restaurant and went there with a gun to stop it.

Technology like this can easily be abused, to terrible consequences. It's probably not going to get as convincing as even CG Tarkin within a reasonable time - say, another 10 years - but it's almost good enough already to create a fake video and share it on facebook, as "proof" of some kind of crazy conspiracy theory...
 
Notice that he's not depicted in his early 20s, and also that he is in a remarkably good shape for his age. Fisher has aged a lot more, let's not be hypocritical about it - decades of drug and alcohol abuse leave a mark.
With all that said, Douglas was an example of the approach taken to its extremes, and they were also lucky with the source material. They could not have done it for the CG Leia.

If you look at the BTS material on Ant-Man, Douglas looks pretty good with just makeup, hair (wig?) and no beard. I think that was easier to do him than to do RDJ in Civil War.
 
Back
Top