Digital Foundry Article Technical Discussion [2022]

[Inuhanyou]

If Forbidden West gets announced for PC the heads of PS fanboys on Twitter will explode.

I would love to see it

But yea, RTGI would be my pick for the game too, but if they want to implement every ray tracing feature so it's future proof I'll accept that too.

Wouldent that just be par the course at this point? Why are people still surprised lmao

 

[Flappy Pannus]

I haven't watched it yet, but I felt the same thing after reading the 5.1 Release Notes.

However, here's the thing.. While it's apparent that it's not going to make it something developers don't have to worry about and that it all just happens automatically, I think it will certainly help reduce longer initial PSO compilation times, and can also help with quicker loading times. The developer can mix and match compilation styles.. and the heavy scenes with shaders that might not be compiled in time (which should be far less than before) can still be precompiled before hand.

But in the end, it's still going to be on the developer to ensure that those things are pre-compiled. Which is why it's really important that we keep CLEARLY sending the message that this stuttering isn't acceptable... so that they don't repeat the same issue in their future games.

Oh it's definitely a big improvement no doubt, but like Alex implied, it's a little difficult to not be cynical at this point.

As I've said before, having Digital Foundry bring this issue to to the forefront is great, but outside of them, what does the PC community as a voice have to promote this issue? Forum posters, the cesspool of Steam community forums, and...? The DIY PC youtube ecosystem needs to start actually understanding that the software technology that makes the hardware they continually promote actually useful. Gamers Nexus/Hardware Unboxed/Linus TT need to actually start at least occasionally looking at new releases on a technical level, especially if they're going to be immediately injecting them into GPU card reviews. The viewer should know that those graphs showing huge bars for the performance of new cards can be at least partially dependent on actually having run through that benchmark segment once previously.

 

[Remij]

Oh it's definitely a big improvement no doubt, but like Alex implied, it's a little difficult to not be cynical at this point.

As I've said before, having Digital Foundry bring this issue to to the forefront is great, but outside of them, what does the PC community as a voice have to promote this issue? Forum posters, the cesspool of Steam community forums, and...? The DIY PC youtube ecosystem needs to start actually understanding that the software technology that makes the hardware they continually promote actually useful. Gamers Nexus/Hardware Unboxed/Linus TT need to actually start at least occasionally looking at new releases on a technical level, especially if they're going to be immediately injecting them into GPU card reviews. The viewer should know that those graphs showing huge bars for the performance of new cards can be at least partially dependent on actually having run through that benchmark segment once previously.

Oh absolutely 100%. Neither Epic nor developers get ANY benefit of the doubt from me.

In fact... at this point I'm wondering why Epic themselves aren't contacting developers from some of these games to gently remind of the issue and maybe help them resolve it.... because quite frankly... it makes their engine look terrible when nearly EVERY game has the issue..

Yes, Digital Foundry is basically our only beacon of light at this point. I really hope they continue to bring the heat about this issue. Maybe Alex could make shorter 5-7 minute videos just showcasing games having the issue and basically just talking about how they still haven't been fixed yet. That gives something tangible for people like us to post all over the forums to hopefully bring about change? They wouldn't need massive production quality passes or big long scripts.. just basically posting footage from the games and talking about how pathetic it is that these issues continue to exist.

And yes, other tech sites could definitely be doing more than they are. It's unfortunate.

 

[Silent_Buddha]

Thinking about one potential solution to this (pre-compiled shaders). While it is almost impossible for the vast majority of developers to have the hardware in house to be able to precompile shaders for the vast and varied permutations of PC hardware and drivers, this would represent a potential opportunity for a cloud provider to provide services to precompile shaders for clients. This is something that doesn't require low latency and thus can be centralized anywhere in the world thus reducing server infrastructure costs somewhat.

Of course, the issue has to become large enough that developers are willing to pay for such a service.

I still think the cheapest and most elegant solution is pre-compiling shaders before a game is run for the first time on a user's machine or anytime a driver/hardware change warrants it. I think this weird desire to want PC games to operate similarly to console games is one of the root causes of the game. Side rant: I Hate THIS so much. Too many games try too hard to be like consoles and don't even allow you to change settings (audio WAY TOO FREAKING LOUD for example) before introductory gameplay and tutorial starts). AUUUUGGGGGHHHHH. Anyway, this tendency to want to make games on PC similar to console games experience means that many developers have thrown out or refused to learn the lessons that PC game developers learned years and years ago ... precompile the damn shaders before the game starts!

Yes, it's a small inconvenience, but it's balanced out by the benefits of having configurable hardware. And it's certainly way better than having disruptive stutters when shaders are compiled as they are called for during gameplay.

Regards,
SB

 

[Remij]

Exactly... we used to have to install PC games... let us "install" those damn shaders!

 

[PSman1700]

I still think the cheapest and most elegant solution is pre-compiling shaders before a game is run for the first time on a user's machine or anytime a driver/hardware change warrants it.

Isnt that what UT99 was already doing? BF2 sure did pre-compile everytime you installed new video drivers when loading a map for the first time after new driver installations.

 

[see colon]

Isnt that what UT99 was already doing? BF2 sure did pre-compile everytime you installed new video drivers when loading a map for the first time after new driver installations.

UT99 didn't precompile shaders because there weren't shaders to compile. BF2 did it, and it's the first game I can remember doing it and telling me about it. And there's nothing stopping modern games from doing it. Back 4 Blood compiles shaders after updates and new drivers before loading to he title screen, and it's a UE4 engine game that doesn't have the reputation of being very technically impressive. In fact, it launched in a fairly glitch filled state, but didn't have shader stutter. So it isn't like you have to be some sort of technical wizard to pull it off.

 

[Silent_Buddha]

UT99 didn't precompile shaders because there weren't shaders to compile. BF2 did it, and it's the first game I can remember doing it and telling me about it. And there's nothing stopping modern games from doing it. Back 4 Blood compiles shaders after updates and new drivers before loading to he title screen, and it's a UE4 engine game that doesn't have the reputation of being very technically impressive. In fact, it launched in a fairly glitch filled state, but didn't have shader stutter. So it isn't like you have to be some sort of technical wizard to pull it off.

Yup, there are still a few PC developers that pre-compiles shaders for their games. Digital Extremes goes so far as to give the user the option to delete the old shader cache and create a new one every time there's a new major patch with significant changes to their shaders.

Regards,
SB

 

[Flappy Pannus]

I still think the cheapest and most elegant solution is pre-compiling shaders before a game is run for the first time on a user's machine or anytime a driver/hardware change warrants it.

(This convo may be better suited to the shader thread)

I think the vast majority want this, the issue with UE4 in particular is that actually generating all of the PSO's beforehand may actually not be possible - even Epic admits that in their 5.1 notes which is why they're also focusing on asynchronous generation which is needed as well. That, and some developers just apparently aren't even aware of the issue in the first place.

One potential approach I've thought about for devs that can't devote a QA team to play through the entire game is to have 'Benchmark' modes, similar to Batman: Arkham City or perhaps shipped as a wholly separate executable like Metro 2033/Last Light does. Naturally you wouldn't want this to be used as an actual benchmark for review sites to use while it chugs its way through it so I wouldn't keep the name as a 'benchmark' mode, something just like 'shader burn-in' or whatnot.

Basically, the idea is you either script a stress test involving a sandbox level with every character/material/effect possible that you can cram into memory, or you cycle through as many levels and unique actions/effects as you can, unloading/loading them as need be. Naturally there could be spoilers so you give the user the option to blank out/mute the test or just do it automatically. While it stutters and jerks its way through this test, it's building the cache which will affect your actual gameplay.

I thought of this when playing around with Arkham City. I was in the middle of a play session with it when I upgraded to Windows 11, and found its performance was brutal after the upgrade. Partially this was due to Nvidia drivers, later updates helped. But a good portion of it was actually due to just having a virgin shader cache for the game, I just forgot that this game also has a lot of shader stutter early on. Turns out one of the ways to reduce that is to run the benchmark - it cycles through various indoor and outdoor environments and will stutter like crazy on the first run as the shader cache is built. These built shaders though impact gameplay for the better, and least early on.

Obviously, having all PSO's possible gathered properly and/or with an asynchronous mechanism to generate during gameplay is the preferred method. But I'm thinking this more of an approach for those UE4 games that have long since been released and aren't in active development, something that could be done without really modifying the core code that significantly or at all - a separate .exe or even just a command line launch option to have the game just load and run this stress-test level to generate as many compiled caches as possible? Something a single developer with access to the game's assets could do?

You wouldn't get all of the PSO's that you would with a properly gathered PSO process involving playing the full game of course, and it would also take longer than a proper precompiling stage of just going through those gathered PSO's as it has to load the actual assets. But you could likely get a hefty chunk of them and significantly improve the first-run experience at least. Dunno.

 

[cwjs]

You guys always bring up the shader compilation thing and point to the like ~3 games that have managed to tackle shader permutations such that they can reasonably pre-compile all of them at a loading screen without it taking 2 hours. However it's really not that easy of a switch to flip.

Being able to offer pre-compiling is the best case solution, but it's a lot harder than it probably sounds to you guys, and it only effects terrible platforms: pc and mobile. Not every company can justify completely overhauling their asset rendering pipeline and doing a ton of engineering work just to improve the user experience for the 5% of people who care on those platforms. Hopefully the few companies that can do that will continue to set standards and solve hard problems that will eventually migrate to the rest of the industry, though.

 

[Flappy Pannus]

You guys always bring up the shader compilation thing and point to the like ~3 games that have managed to tackle shader permutations such that they can reasonably pre-compile all of them at a loading screen without it taking 2 hours

I think the longest is Uncharted 4 at around 20 minutes on anything approaching a modern CPU, and you can skip it regardless and barely have it affect frame rate while it does it in the background. Sackboy, Psychonauts 2, The Ascent were all patched to remedy their problems with this without increasing load times significantly, games like Days Gone and the Gears games were UE4 and never had it to begin with.

Easy? Probably not. But certainly not an impossibility, and certainly without taking '2 hours'.

However it's really not that easy of a switch to flip.

Being able to offer pre-compiling is the best case solution, but it's a lot harder than it probably sounds to you guys,

Who is "you guys"? I've routinely brought up the difficulty with generating these PSO's in U4 and how it's not so simple. The actual implementation details have been discussed at length in other threads as well, by all means chime in with your analysis there if you have something to add.

and it only effects terrible platforms: pc and mobile.

This is a forum that was founded to discuss PC 3D graphics tech, tech that has made its way into mainstream consoles. If you don't care about the PC as a platform that's your prerogative (lord knows I have my frustrations with it) but I question why then you're even here in the first place. This isn't a console-specific thread, and DF has been a paramount source in discussing this problem, so naturally it's going to be discussed here as well.

Not every company can justify completely overhauling their asset rendering pipeline and doing a ton of engineering work

This makes no sense in the context of complaining about the lack of a pre-compiling stage. You need that stage precisely because the approach that iD took with Doom:Eternal is not viable with the limited time/budget/expertise other dev teams have.

That is the 'best case' solution, not just including a length pre-compiling stage that will not handle every case regardless - just mitigate it as best as possible. Nobody is arguing for an 'overhaul of the rendering pipeline' here - they're asking for the documented feature of UE4 to be utilized in some fashion to improve the first-run experience as much as possible.

 

[cwjs]

I think the longest is Uncharted 4 at around 20 minutes on anything approaching a modern CPU, and you can skip it regardless and barely have it affect frame rate while it does it in the background. Sackboy, Psychonauts 2, The Ascent were all patched to remedy their problems with this without increasing load times significantly, games like Days Gone and the Gears games were UE4 and never had it to begin with.

Easy? Probably not. But certainly not an impossibility, and certainly without taking '2 hours'.

Naively writing some code that says "loop through every shader permutation and compile it" would certainly take a very long time. Trying to produce some kind of a best guess you can compile is expensive, and that best guess resulting in a small enough shader count to take ~10 minutes requires discipline with the number of shader variants you produce in the first place (also expensive, unless you by coincidence made choices which produced a lower number for other reasons.)

This is a forum that was founded to discuss PC 3D graphics tech, tech that has made its way into mainstream consoles. If you don't care about the PC as a platform that's your prerogative, but I question why you're here in the first place. This isn't a console-specific thread, and DF has been a paramount source in discussing this problem, so naturally it's going to be discussed here as well.

When I say "terrible platform" I don't imply I have zero interest in it -- I have a 3080, I play pc games, etc. However, fundamentally, PC gaming is about slapping an amazing piece of graphics hardware into a machine designed around tasks like loading emails. Shipping a game is never going to have the kind high touch polish on pc that the same amount of resources can produce on consoles. Despite this, somehow, pc gamers consistently show higher expectations and complain about more marginal issues than console players do.

Editing my post to include a response to this addition:

This makes no sense in the context of complaining about the lack of a pre-compiling stage. You need that stage precisely because the approach that iD took with Doom:Eternal is not viable with the limited time/budget/expertise other dev teams have.

That is the 'best case' solution, not just including a length pre-compiling stage that will not handle every case regardless - just mitigate it as best as possible. Nobody is arguing for an 'overhaul of the rendering pipeline' here - they're asking for the documented feature of UE4 to be utilized in some fashion to improve the first-run experience as much as possible.

I disagree -- I think a small variant count approach (featured in doom eternal and call of duty mw2, which use very similar clustered forward renderers and both avoid shader stutter, among others) is clearly the correct longterm path forward. Aside from shader stutter, it decreases build time on the developer side to generate those variants, can improve rendering performance and batching, and decreases memory and storage overhead from shaders. Over the next several years I expect all games will migrate towards smaller numbers of shader variants than they have right now.

Edit again:
Also, I don't have personal experience with the documented ue4 workflows to mitigate shader compilation stutter, but the equivalent workflows in unity that I've seen posters (not sure if here or elsewhere) imply everyone should use are terrible and involve a lot more trial and error and work than I think people expect.

 

[Remij]

You guys always bring up the shader compilation thing and point to the like ~3 games that have managed to tackle shader permutations such that they can reasonably pre-compile all of them at a loading screen without it taking 2 hours. However it's really not that easy of a switch to flip.

Being able to offer pre-compiling is the best case solution, but it's a lot harder than it probably sounds to you guys, and it only effects terrible platforms: pc and mobile. Not every company can justify completely overhauling their asset rendering pipeline and doing a ton of engineering work just to improve the user experience for the 5% of people who care on those platforms. Hopefully the few companies that can do that will continue to set standards and solve hard problems that will eventually migrate to the rest of the industry, though.

Excuse me? Almost everything you said here was complete BS... First of all, plenty PLENTY of games don't have issues with shader compilation, and there's also plenty of games that did... and were later patched and fixed.

There's no 2 hour waiting either... And it was easy enough for Sumo Digital, or whoever ported Sackboy to have it fixed in ONE WEEK after Digital Foundry called them out on it... But wait, I forgot... why am I supposed to care how hard it is for a developer to do their damn job correctly? And calling PC and mobile terrible platforms huh.... how about this... if the developers suck.. and the engine they're using sucks... how about they don't release a broken POS of a game in the first place? How about Epic come out with a press release admitting that their engine (which entirely EXISTS because of PC in the first place) sucks on that very platform??

See, the problem is... that there's people out there who think like you do. Think about what you're doing... you're making excuses for a company to not put out a quality product... because it's more work....

No... things are going to change. We're tired of this BS... we're tired of developers "excuses"... You put out a product... it needs to f-ing work properly.

 

[Flappy Pannus]

Naively writing some code that says "loop through every shader permutation and compile it" would certainly take a very long time.

We're talking about UE4, and it's built-in function to gather PSO's so it can precompile them.

The issue is the time it takes to gather all these PSO's as it requires extensive play testing, and cannot get them all regardless. Again you're arguing against a strawman, nobody is expecting every developer to write their own asynchronous compiler from scratch.

Hell, these recent posts in this thread are all focused on what method could get potentially get these shaders compiled with the least responsibility on the developer possible!

When I say "terrible platform" I don't imply I have zero interest in it -- I have a 3080, I play pc games, etc. However, fundamentally, PC gaming is about slapping an amazing piece of graphics hardware into a machine designed around tasks like loading emails. Shipping a game is never going to have the kind high touch polish on pc that the same amount of resources can produce on consoles. Despite this, somehow, pc gamers consistently show higher expectations and complain about more marginal issues than console players do.

The 'higher expectations' here are 'don't have 100ms stutters due to ignoring the requirement of shader compiling that has existed for the platform since shaders were introduced by it'.

It's not a 'marginal issue' on consoles - it's a nonexistent issue there. We're not getting these huge stutters because Outlook is checking email in the background. If you got these kind of shaders stutters on console games they would be absolutely vilified.

I mean this is a tech forum, people are going to bitch about issues without taking a market survey of what % of buyers it may ultimately effect. This is a thread in such a forum specifically about a channel which bitches about technical issues, and on this issue predominately. It's a niche of a niche of a niche, don't know what to tell you.

 

[cwjs]

It's not a 'marginal issue' on consoles - it's a nonexistent issue there. We're not getting these huge stutters because Outlook is checking email in the background. If you got these kind of shaders stutters on console games they would be absolutely vilified.

When I said "more marginal issues" I was referring to the broad corpus of 'things gamers complain about', not this issue. You get those huge stutters because the platform, by design, does not guarantee a set hardware specification. The reasons for that are numerous and historical but ultimately come down to "PC as a platform is not intended for playing videogames, there's a different kind of computer which is meant for that"

And it was easy enough for Sumo Digital, or whoever ported Sackboy to have it fixed in ONE WEEK after Digital Foundry called them out on it... But wait, I forgot... why am I supposed to care how hard it is for a developer to do their damn job correctly? And calling PC and mobile terrible platforms huh.... how about this... if the developers suck.. and the engine they're using sucks... how about they don't release a broken POS of a game in the first place? How about Epic come out with a press release admitting that their engine (which entirely EXISTS because of PC in the first place) sucks on that very platform??

No... things are going to change. We're tired of this BS... we're tired of developers "excuses"... You put out a product... it needs to f-ing work properly.

Come on dude this kind of goofball post is exactly why I'm derisive of people constantly banging on about this issue. Developers know when the game doesn't run great. We make the best choices we can considering the business cases. I'm personally critical of some of the design choices that led to the particularly excessive shader variant counts in engines like ue4 and unity, but there are sound reasons why a dev can't talk their coworkers into scrapping everything and starting over with their own game engine.

 

[Remij]

"PC as a platform is not intended for playing videogames, there's a different kind of computer which is meant for that"

More complete BS...

 

[cwjs]

More complete BS...

I mean look dude, it's blatantly, obviously true that if AMD or Nvidia and a platform holder and the developers of their OS all coordinated they could ship a machine that doesn't have to compile shaders at runtime under any circumstances. That's happened several times. They're called game consoles.

 

[Remij]

I mean look dude, it's blatantly, obviously true that if AMD or Nvidia and a platform holder and the developers of their OS all coordinated they could ship a machine that doesn't have to compile shaders at runtime under any circumstances. That's happened several times. They're called game consoles.

More than one of those runs Windows... doesn't it? LOL

And yes, developers need to compile shaders for those consoles... the games are developed on PCs.......

 

[Flappy Pannus]

When I said "more marginal issues" I was referring to the broad corpus of 'things gamers complain about', not this issue.

You replied to a thread referencing this issue specifically my dude, replying to a post talking solely about this issue. I'm sure there are plenty of issues from PC and console games that I don't give a shit about, but they're not relevant to this thread.

there's a different kind of computer which is meant for that"

But it's not a 'different kind of computer' - it's not a computer. If it was I wouldn't be gaming on the PC at all, no one would.

This is the argument PC fanboys routinely make in dismissing consoles - consoles are now 'just PC's' - they're not. They have similar base technology, but the way you access games and the freedom you have provides a decidedly different experience in many cases.

Like, why do you have a 3080? Why not just sell it and put it towards from PS5 games? I think that's a perfectly viable option and I can't recall the last time I recommended PC gaming over console to someone who's not already knowledgeable about the platform, but I find it curious if you really think consoles are just a 'better type of gaming computer' why you even have a gaming PC in the first place?

You get those huge stutters because the platform, by design, does not guarantee a set hardware specification.

Which has been the case since its inception. This is not a new development requirement, hence we see games with included shader compiling stages from decades ago. Which is why, by design, UE4 documents a process to gather these PSO's to alleviate this issue as best as it can.

This isn't about PC gamers complaining that they're not getting full mod support in every game. This isn't a thread on how developers are screwing the platform due to not having full Dualsense Haptic support for every game, or not shipping the PC version with a completely new RT-based global lighting system.

They're asking for something that was actually more prevalent in years past to return, something that is completely fundamental to the architecture and has always been.

As another poster said, part of the issue is just that there are likely less experienced PC developers on the market these days, some developers are genuinely in the dark about this - The Ascent developers said as much, they just didn't know. Yet they fixed it.

There is not doubt a clueless cadre of some fanboys (of all platforms) that will point the finger at 'lazy developers' with having no understanding of the technical and project management hurdles they face, sure. But it's perfectly reasonable to complain about products not functionating the way they should, and you don't need to swing the other way and treat ever developer/publisher as some sweet smol baby that ultimately doesn't have a responsibility to ship a working product because it will 'only' sell ~1-2 million vs 6+ million on consoles.

I mean you can read the experience of Special K's developer when RE:Village dropped and he was looking at reducing the stuttering. Aside from the copy protection, it was also doing insane shit like querying the available resolutions a thousand times per frame. You can understand the pressure on developers without deifying them, there is some poor work out there, and ultimately these are products being sold at a price, this aren't open source projects. They deserve critique, especially when encountering issues that should be understandable to anyone working on the platform.

 

[Remij]

This isn't about PC gamers complaining that they're not getting full mod support in every game. They're not demanding full Dualsense Haptic support for every game. They're asking for something that was actually more prevalent in years past to return.

Imagine asking for a game to perform as it should... without large stutters every time something new happens for the first time.. The nerve of PC gamers huh. Obviously that's just WAY too hard for developers to do... so screw them... because they're on shitty platforms.

Imagine thinking that asking for a game to run properly is too much! Poor developers!! Why wont somebody think of them?!