Digital Foundry Article Technical Discussion [2021]

[JPT]

can you sell me after first rage quit for half of price ?

I only buy digital
And I just bought it, so it's safe and sound on my PS5 now. I am still stuck in Vigor land, will try and play Returnal sometime tomorrow.

 

[Deleted member 11852]

couse in theory will "automacitlcy" work with proper rays bouncing and artist doesn't have to use bake lighting per room etc

I feel like I'm stuck in a loop but without the benefit of humour of Bill Murray. Yes, I get this. But what happens when you have a RT lighting system but for specific scenes you do not want wholly realistic lighting, i.e. you want this area to be darker and this bit to be lighter, but for there although to be enough light to see things even if the few scene lights from which RT lighting is sourced isn't enough. I.e. the kind of thing the Director on the movie asks the Director of Lighting to work some magic on.

I'm getting the impression a lot of people here have no idea what a lighting director does but perhaps that they are the person in charge of all the lightbulbs.

 

[JoeJ]

But what happens when you have a RT lighting system but for specific scenes you do not want wholly realistic lighting, i.e. you want this area to be darker and this bit to be lighter

You could use two different materials - one to be shaded for the screen buffer, the other to be used by the lighting system. Or just a single parameter per texel/model/light/etc. to tweak reflection strength for the latter.
And you can use invisible lights ofc., so you get it brighter without a visible source. (Personally i tried even 'negative' lights, but looked shitty.)

 

[PSman1700]

You could use two different materials - one to be shaded for the screen buffer, the other to be used by the lighting system. Or just a single parameter per texel/model/light/etc. to tweak reflection strength for the latter.
And you can use invisible lights ofc., so you get it brighter without a visible source. (Personally i tried even 'negative' lights, but looked shitty.)

Thanks Joej and others for teaching those that arent really in the know regarding ray tracing/lighthing

 

[JoeJ]

Thanks Joej and others for teaching those that arent really in the know regarding ray tracing/lighthing

haha, well, i did no pay attention to the whole discussion, but on the art side, surely there are new problems to expect, and worries are justified.
Before: 'Damn - spot looks too dark! How to fix that? Put in a fill light '
After: 'Damn - spot is too bright, i wan't it dark and hidden. How to fix that? Ruin the nice lighting of the brighter spot beside it '

However, remembering the coworkers which did television production, their usual problem was not having enough lambs and lighting stuff, not having too much of it. So i guess above problem won't happen that often.
I expect everybody will see overall benefit and easing things up, exactly like it was with the move to PBS last gen. Only by adding global light transport as well we get full PBR this time.

 

[iroboto]

No. NO. Seriously man, have you read any of my posts? I'm asking folks like you who said RT lighting will save time to explain why. I've not said it'll be more time, the same time or less time. I'm asking a question because I don't know the answer. You've have also admitted you don't know the answer which is fine but please stop trying to project some position onto me for asking a question.

Fair enough but I did do so and with proof. The time is saved on the very least the baking aspect of each iteration you bake which can be hours on end of baking depending on how large the world is, how many changes you make to the environment, the quests, etc, how many platforms you want to deploy onto as each console may represent the bake differently.

And your response to me at the time: is well how much more work do they have to add to the pipeline by removing this baking.

I don't know how else to interpret this except to think that you think there is more work that could be added that could reduce the amount of time saved from baking. In a point about iteration time in which adding, removing, moving a light requires a re-bake to see the results; that means you can only make 12 iterations every hour if a rebake takes 5 minutes. imo, even if you had to fight real-time GI, you can at least do more than 12 iterations per hour. I think the amount of time baking and waiting for a bake far outweighs any possible additional labour associated here. I disagree that rasterized lighting is going to be easier for a lighting director to do than working with real-time GI. They still got the same job. The only difference is now they are fighting with the challenge of things perhaps being too bright or too dark. But with traditional T&L models they have to deal with lighting artifacts, areas looking artificially lit, lighting that doesn't look unified, light leaks, and shadows that aren't grounded, heck even making shadow maps, and so forth. Not to mention the whole lack of dynamic lighting possibilities that pretty much represent how quickly gamers will define what next-generation games actually look like. Because as much as I understand your argument that it could be tougher to get the exact artistic lighting that very few teams would be willing to invest in a particular scene or area, it's even tougher to integrate these next generation lighting techniques with a fixed lighting system consistently across a whole world.

I'm sure that documentary left a huge impression on you (sometimes they focus on the challenges only to really tell the story of hardship). But looking at blogs from lighting artists from Pixar:
This Animated Life: Pixar's Lightspeed Brings New Light to Monsters University

There's a huge difference in MU compared to past films. Even people that don't know anything about our tech change going in walk out going "HOLY CRAP!"....but they have a hard time putting their finger on why it looks so awesome. Personally I see a huge difference between MU and Brave - there's more shaping, more little splashes of color, and everything feels a little bit more dynamic and pulled together. This is particularly evident in the toxic urchins sequence - where every single urchin is a light source. We couldn't have done that sequence in the past with our old technology.

Historically we don't use raytracing. It wasn't until Cars that we actually supported raytracing (and even then it was a haphazard and mostly broken support). We really only used it for highly reflective smooth curved surfaces that absolutely needed to be truly reflected and not faked. We fake almost everything - mirrors, wet surfaces, eyes, shiny props like belt buckles/spoons/swords/etc. We obviously can't get away with that on Lightning McQueen - so we would cache out the scene into a brickmap (essentially a kd-tree with shading attached to the voxels) and fire rays against that (so even then....we aren't doing traditional raytracing). For shadows - we would sometimes use raytracing when we needed particularly awesome looking contact shadows. The same shadow would ramp off to using a shadowmap to help lower the expense.

So our Director of Photography went to a studio that is so clearly raytracing averse and essentially said "We're raytracing everything. True reflection and refraction in the eyes reflecting actual SCENE GEOMETRY and not a brickmap. Yep - we're refracting through the cornea onto the sclera and iris. Oh and all your shadows are raytraced now - no more shadowmaps. Nope. None. Yes I know you like them but no. And global illumination! We're doing that now. By default. Everywhere. Oh and I almost forgot - all reflective surfaces will do real true reflection....and deciding what's reflective will be a shading decision instead of a lighting one. Yes you heard me right. Now get to work" It was extremely controversial, but it made a huge impact and really was one of the true success stories of the film. And now I'm working on that.

In a dynamic night and day cycle where the player can look anywhere and the time of day can be anything, in any part of the world, how is the lighting director going to have an easier time there with traditional lighting with inaccurate probes vs RTGI?

This is the part I don't understand from your argument. To me this is a tooling issue if you can't modify your lighting parameters. If you want more bounces or less bounces, no bounces, more GI or less GI, more focused light, less focused light, those should all eventually become parameter sliders as lighting models matures with the tooling. What I don't understand is why you think the system would be so inflexible and that there is no method around this inflexibility.

I get that there will be a transitionary period in which there will be chaos moving the whole lighting stack and all the tools and processes that come with it designed for one thing but now being retooled for another. But that's going to apply to every new big change. Once that transition is done and is a well oiled machine, I have an incredibly hard time believing there is no significant amount of work being saved.

 

[chris1515]

 

[Arwin]

Fair enough but I did do so and with proof. The time is saved on the very least the baking aspect of each iteration you bake which can be hours on end of baking depending on how large the world is, how many changes you make to the environment, the quests, etc, how many platforms you want to deploy onto as each console may represent the bake differently.

And your response to me at the time: is well how much more work do they have to add to the pipeline by removing this baking.

I don't know how else to interpret this except to think that you think there is more work that could be added that could reduce the amount of time saved from baking. In a point about iteration time in which adding, removing, moving a light requires a re-bake to see the results; that means you can only make 12 iterations every hour if a rebake takes 5 minutes. imo, even if you had to fight real-time GI, you can at least do more than 12 iterations per hour. I think the amount of time baking and waiting for a bake far outweighs any possible additional labour associated here. I disagree that rasterized lighting is going to be easier for a lighting director to do than working with real-time GI. They still got the same job. The only difference is now they are fighting with the challenge of things perhaps being too bright or too dark. But with traditional T&L models they have to deal with lighting artifacts, areas looking artificially lit, lighting that doesn't look unified, light leaks, and shadows that aren't grounded, heck even making shadow maps, and so forth. Not to mention the whole lack of dynamic lighting possibilities that pretty much represent how quickly gamers will define what next-generation games actually look like. Because as much as I understand your argument that it could be tougher to get the exact artistic lighting that very few teams would be willing to invest in a particular scene or area, it's even tougher to integrate these next generation lighting techniques with a fixed lighting system consistently across a whole world.

In a dynamic night and day cycle where the player can look anywhere and the time of day can be anything, in any part of the world, how is the lighting director going to have an easier time there with traditional lighting with inaccurate probes vs RTGI?

This is the part I don't understand from your argument. To me this is a tooling issue if you can't modify your lighting parameters. If you want more bounces or less bounces, no bounces, more GI or less GI, more focused light, less focused light, those should all eventually become parameter sliders as lighting models matures with the tooling. What I don't understand is why you think the system would be so inflexible and that there is no method around this inflexibility.

Also we should never forget that the game industry is always several steps behind the CGI movie industry. In other words, almost all these problems have been solved already. And finally, as light simulation come ever closer to real life, all the old and new tricks from real life shooting cross over together with all the digital trickery at our disposal.

Ever since we moved from 2D to 3D, which was a genuine step back on many levels, nothing was ultimately ever lost by gaining something, only some aspects were temporarily more or less expensive depending on the amount of pixels VS bandwidth and so on.

In other words, not having to bake doesn’t mean you can’t. Not having to use area lights doesn’t mean you can’t. Having GI doesn’t mean you can’t turn it off in certain areas, rooms and so on. Having realistic materials doesn’t mean you can’t make them unrealistic. And so on and so forth.

 

[chris1515]

The Ratchet video is great. Ratchet miss the realtime GI for the CGI look. Ratchet density an detail mixed with Metro lighting would probably look pre rendered. Not comparable to movie but very difficult to know if it is realtime gameplay or in engine stuff.

Wait a lot all the mesh shading/geometry innovation around Level of detail transition.

 

[Shortbread]

While the best-looking game so far this generation, IMHO, I'm still hoping Sony would consider a PC edition further down the line.

 

[Cyan]

Put this another way. 3D animated movies (Pixar, Dreamworks, etc.) have had access to and used RT in films for many many years.

RT didn't solve their lighting. It made some things easier but they still require lots of jiggery do in order to attain the lighting look that they desire. Sometimes that means not using RT for lighting.

And that is when the director and artists have 100% control over what the camera is looking at.

Now, take the camera out of the control of the creator and give it to the viewer. Suddenly the lighting in those Pixar movies breaks down depending on where the camera is pointed.

In games, if you're just going for a realistic look with no concern about using lighting as an artistic tool (other than just changing the color of the lighting), then RT is, in most cases, an obvious and easy win.

If you want to use it artistically, you will need to put more work into it than say Pixar does in order to get the look you want. Because you no longer get to determine where the camera is looking at any given moment in time.

I've already used Horror games as an example where control and use of lighting in non-realistic ways is often used and is beneficial to the mood of a scene. Again take the case of a Strong light beneath the face of a monster so that the face is dramatically lit while the rest is in shadow. To use RT, you'd have to somehow limit bounces to only affect the face. You'd have to limit the distance it can travel so it doesn't say hit the ceiling (unwanted) bounce to unintentionally light the rest of the room (unwanted). Or any sort of leakage to anything other than the facial features of the creature.

Another case would be where you have a spotlight that illuminates one portion of a room or even only one object in a room but doesn't touch anything else in that room. Conventional lighting techniques handle those with ease, but with RT you have to have the tools where you can severely limit any light propagation. Perhaps those tools already exist, and thus this is a non-issue.

Film, both real life and 3D rendered spend lots and lots of time finding ways to do these things. This includes going through and hand editing individual frames of film in a given scene to achieve certain lighting effects.

Conventional game lighting is both a curse (you have to hand place almost everything and figure ways around the limitations) and a boon (you can hand place everything and the limitations are beneficial). We've yet to see how a game artist will approach RT lighting when you start requiring something more than just natural light propagation. It may be that we'll just see a continued mix of RT lighting with conventional lighting techniques (where the limitations potentially make it easier to control what is lit and not lit).

I'm a huge proponent of RT in getting rid of unwanted and disconcerting work arounds like AO (thank you Metro: Exodus), but I'm not ready to hand wave away potential problems when it comes to the artistic use of lighting (as seen in Metro: Exodus with characters faces).

Regards,
SB

great points as usual. That being said, you can see where @Dictator about the meaning of RT for him.

Most of the RT games showcased by Digital Founrdy, are games where RT could be PERFECT for the game's atmosphere. Games where artistic decisions favour RT more than anything else.

We mentioned here a few ot fhose games where RT could create the best and most realistic lighting on a videogame ever seen to date. Alien Isolation and Doom 3 come to mind.

Resident Evil 2 Remake could be a good candidate, though this could need some planning. There are scenes in the game (HDR mode) that reach about 10000 to 20000:1 in contrast for black colour (when Claire gets behind the metal shutter at the beginning of the game), so maybe some -but very few, I guess- scenes might be retouched for an artistic effect.

 

[snc]

While the best-looking game so far this generation, IMHO, I'm still hoping Sony would consider a PC edition further down the line.

I wonder what minimal requirement for ssd would be

 

[DavidGraham]

Ever since we moved from 2D to 3D, which was a genuine step back on many levels

I am interested to know why you said that ..

 

[Silent_Buddha]

I am interested to know why you said that ..

Character art, character expressions, character animation all took a HUGE step backwards in many cases compared to 2D animated artwork.

Even to this day, some 2D animated character art in games look significantly better than anything 3D has to offer.

And when it comes to cartoonish exaggerated expressions and animations? 3D has failed repeatedly to come close to capturing what you can do with animated 2D art.

Where 3D excels is with realistic animations (pretty much non-existent in the first few years of 3D games) and time to market (time and cost).

Just compare animated Disney characters in games to 3D Disney characters in games? Or worse yet, pretty much any 2D animated anime compared to a 3D game using characters from that anime. The One-Piece games come the closest, but even those aren't even close. If a 2D animated game were instead made, then if your artist was good, you'd be able to faithfully recreate all of the animations and expressions from the anime.

Heck, sticking with one medium. While I love Pixar animated movies, their character art and expressions pale in comparison to what is possible with 2D hand drawn animations. However, the benefit is that it's FAR less time consuming and costly to do 3D animation than 2D hand drawn animation.

Regards,
SB

 

[snc]

Character art, character expressions, character animation all took a HUGE step backwards in many cases compared to 2D animated artwork.

Even to this day, some 2D animated character art in games look significantly better than anything 3D has to offer.

And when it comes to cartoonish exaggerated expressions and animations? 3D has failed repeatedly to come close to capturing what you can do with animated 2D art.

Where 3D excels is with realistic animations (pretty much non-existent in the first few years of 3D games).

Regards,
SB

anybody remember street fighter ex or mortal kombat 4 ? ;d it took long time for street fighter and mortal kombat to keep feeling of 2d version in 3d environment

 

[DavidGraham]

Character art, character expressions, character animation all took a HUGE step backwards in many cases compared to 2D animated artwork.

Thinking about it, lighting and shadowing took a massive hit as well, environment details too. Early 3D games had very simple geometry, empty rooms and corridors, minimal particle work, and static lights and shadows.

 

[Shortbread]

I wonder what minimal requirement for ssd would be

Something that's fast enough or capable of matching or surpassing PS5 SSD/IO data prioritization management. If I remember correctly, PS5 SSD/IO has 6 levels of prioritization for IO commands, to essentially reduce as much latency as possible when streaming in large amounts of data.

Character art, character expressions, character animation all took a HUGE step backwards in many cases compared to 2D animated artwork.

Even to this day, some 2D animated character art in games look significantly better than anything 3D has to offer.

And when it comes to cartoonish exaggerated expressions and animations? 3D has failed repeatedly to come close to capturing what you can do with animated 2D art.

Where 3D excels is with realistic animations (pretty much non-existent in the first few years of 3D games) and time to market (time and cost).

Just compare animated Disney characters in games to 3D Disney characters in games? Or worse yet, pretty much any 2D animated anime compared to a 3D game using characters from that anime. The One-Piece games come the closest, but even those aren't even close. If a 2D animated game were instead made, then if your artist was good, you'd be able to faithfully recreate all of the animations and expressions from the anime.

Heck, sticking with one medium. While I love Pixar animated movies, their character art and expressions pale in comparison to what is possible with 2D hand drawn animations. However, the benefit is that it's FAR less time consuming and costly to do 3D animation than 2D hand drawn animation.

Regards,
SB

Yup. Actually, 'Cuphead' 2D animation is still far superior to many of today's 3D animated games.

 

[Silent_Buddha]

It is kind of interesting to think that it's far harder to make a 3D cartoon style character model than it is to make a 3D realistic style character model. We have numerous examples of almost photorealistic 3D human models, but no convincing 3D cartoon model.

I've no idea whether this is due to no-one putting in the time to make a convincing 3D cartoon model or if it's just extremely difficult to make a 3D model that looks like a hand drawn cartoon character when viewed from multiple angles. I think it's the latter. If it's only ever viewed from one angle, it'd probably be pretty easy. But having it look like a hand drawn cartoon model as you move the camera around it? Or when it's animated? That's the incredibly difficult part, I think.

Regards,
SB

 

[PSman1700]

Watched the DF video on rift apart. Very impressive, really like the fantasy world/nightly city environments. Ranks as the most impressive PS5 game along with DS, and is the most visually impressive game of the console generation. CP2077 still is more impressive though if you run it all maxed out at 60fps (using dlss and a monster gpu).

 

[JoeJ]

The Ratchet video is great. Ratchet miss the realtime GI for the CGI look. Ratchet density an detail mixed with Metro lighting would probably look pre rendered.

... or very noisy. Not sure it works well for a game that fast.