They will be motivated one way or another. It's isn't an issue they will be able to continue to ignore.
Shader Compilation on PC: About to become a bigger bottleneck?
- Thread starter Flappy Pannus
- Start date
D
Deleted member 11852
Guest
I'm not sure I see this as Microsoft's problem. This is a consequence of technology, that was developed with heavy industry input. The obvious solution is pre-compile shaders before you run the game but that's down the seller, or the delivery platform. If I were Microsoft I would point to all these other APIs that make up modern Windows and say.. why not make something out of those? ;-)
So Epic had an "Ask Unreal Anything" session (their first one apparently) and I tossed in a question asking about Shader/PSO compilation issues in Unreal Engine. They didn't answer my question directly, but referred me to another person's question, which they did answer. This is what they said:
Another asked about similar things regarding DX12 and PSO collection, and they have updated their UE5 Documentation to more clearly explain what developers need to do to ensure they capture as good and current PSO data as possible.
It's quite clear just from reading that, that it's quite an involved process for developers. It requires basically a ton of playing. Of course, ideally the developers would be doing things and would do so right up until the very last minute that their game is done. But its easy to see how this stuff could easily fall through the cracks from developers who don't have the time/budget/staff to devote those resources to it... and who may not know how to do it properly. This seems to be what clearly separates studios like The Coalition and a few others, from the rest.
I hope with UE5, and improved documentation and PSO collection systems, that it becomes easier for ALL developers to do this properly.
As a side note, I wonder why there isn't some company that is dedicated to providing this service that developers could tap into? Essentially have people play through completed versions of the games and their sole purpose is to run through every inch of the game in all settings, and building these caches as thoroughly as possible..
Another asked about similar things regarding DX12 and PSO collection, and they have updated their UE5 Documentation to more clearly explain what developers need to do to ensure they capture as good and current PSO data as possible.
It's quite clear just from reading that, that it's quite an involved process for developers. It requires basically a ton of playing. Of course, ideally the developers would be doing things and would do so right up until the very last minute that their game is done. But its easy to see how this stuff could easily fall through the cracks from developers who don't have the time/budget/staff to devote those resources to it... and who may not know how to do it properly. This seems to be what clearly separates studios like The Coalition and a few others, from the rest.
I hope with UE5, and improved documentation and PSO collection systems, that it becomes easier for ALL developers to do this properly.
As a side note, I wonder why there isn't some company that is dedicated to providing this service that developers could tap into? Essentially have people play through completed versions of the games and their sole purpose is to run through every inch of the game in all settings, and building these caches as thoroughly as possible..
Rootax
Veteran
I still don't get why you don't precompile at first load. At this point you know the os, gpu, drivers,etc...
Honestly, while some devs point at true technical problems, other seem just outclassed. "If the solution is not in the engine I use (and know nothing about) in a one big "fix it" button, then it can't be fixed" attitude....
Honestly, while some devs point at true technical problems, other seem just outclassed. "If the solution is not in the engine I use (and know nothing about) in a one big "fix it" button, then it can't be fixed" attitude....
The issue doesn't seem to be about when to pre-compile... but rather the difficulty for developers themselves to collect all the pipeline state data so that they CAN properly pre-compile at any given point. They need to know which pipelines and shaders will be in use so they can pre-compile them. If they don't have the pipeline data, they can't pre-compile everything they need before time to avoid hitches.
To get this data they basically need to play the game as they develop it, which contributes to the collection of PSO data, and continually do it after any changes which might affect it. Not only that but they have to play through everything multiple times, and at different graphics settings levels, to generate a more complete PSO dataset. So it's not exactly easy to catch everything when you're in active development with things constantly changing.
Some devs seemingly create their own methods for somewhat automating it as much as possible, which may make it easier for them to do so, but obviously not every studio that uses UE will have the knowledge and ability to do so.
For the most part they should:
-Play the game as they develop it, continually going through all the levels collecting all the pipeline state after any changes, and do so multiple times to ensure best coverage
-Build mini levels (time-demo type run throughs) which contain the graphics and effects they have in their levels they can run to update the PSO collection
-Probably some code specific methods I'd know nothing about
-ect ect
So it seems to be an issue that some developers do a more thorough job at this currently than others. Some may think that everything is caught during normal development? I dunno.. UE4 documentation also apparently maybe wasn't perfectly clear on how to do this more efficiently either. This is one area they've already addressed for UE5, and as stated above, it seems that they're working on implementing some way of automatically gathering all that information when the assets are loaded. This will include data from particle systems like Cascade and Niagara, which I believe Alex said some developers had mention was difficult to do before. And if I'm understanding this right, the way it is currently is that they essentially have to build the project and play it to gather the data (it can only be gathered from cooked projects because that's the only way to generate the files which map the shaders to materials) but in the future there may be a way for them to gather this data without having to do so???
I dunno, wayyy over my head.. but anyway lets hope the folks at Epic can figure out a better way to make it as easy as possible for ALL developers to collect the data they need so they can properly pre-compile their shaders.
Last edited:
DegustatoR
Legend
The issue is that PSOs can (and are) be generated on the fly by the engine which means that there are no complete repository of all PSOs which must be pre-compiled - unless you've made such a depository yourself and shipped it with your game.
I still think that it's a bit flawed engine logic though. Ideally you'd want the task of generation of all possible PSOs to be automated on the engine side instead of forcing someone to play the whole game with all possible combinations of settings and gear. The engine can probably also have some sort of heuristic which would predict which PSOs will be generated next and would thus be able to send them to driver for compilation before they will actually be needed.
Basically yes there's a way of solving this in UE4 now - but it's a bit of a manual task which could take 100s of hours multiplied by number of game engine builds. Hardly an elegant solution for an engine which should presumably be very developer friendly.
I still think that it's a bit flawed engine logic though. Ideally you'd want the task of generation of all possible PSOs to be automated on the engine side instead of forcing someone to play the whole game with all possible combinations of settings and gear. The engine can probably also have some sort of heuristic which would predict which PSOs will be generated next and would thus be able to send them to driver for compilation before they will actually be needed.
Basically yes there's a way of solving this in UE4 now - but it's a bit of a manual task which could take 100s of hours multiplied by number of game engine builds. Hardly an elegant solution for an engine which should presumably be very developer friendly.
In the UE4 documentation they mention that it's possible to crowdsource PSO repositories from gamers, and then the developers can update the game with the collected info... but that there's no built in mechanism to do this (at least currently)
That's essentially what Steam is doing with Vulkan and their pre-caching system. Gamers on Steam could very quickly build a complete repository for games. With UE, having that functionality working and built-in could be a game changer for indie or smaller AA developers who can't take all the measures themselves to generate a complete PSO dataset during development. Towards the end of the game they could hand it off to a group of play testers and have them build it as they play, then update for the finished release.
Flappy Pannus
Veteran
It would definitely help, but getting those PSO's generated as much as possible before first run is still paramount.
The problem with crowd sourcing compiled shaders is that you're depending on other users to have a bad experience before you so you'll have a good one, also the game must be played enough on the driver you're running on by others in the first place to potentially have any improvement as well.
I've been playing around with Linux recently and my Steam library, and while the shader pre-caching step does help, it clearly is insufficient as I can still run into plenty of shader stutter when getting permutations the crowdsource solution missed.
For sure. It's absolutely not a replacement for developers doing as thorough of a capture as possible.
But I'm not talking about compiled shaders though.. I'm talking about crowdsourced pipeline caches which, if I understand correctly, allow them re-cook the updated code into the game build. It's all about knowing which shaders/pipelines need to be compiled so they can do it upfront. Then the player runs the game, and depending on their GPU/driver the code will be compiled/optimized for their specific hardware. It should be GPU/driver agnostic code shipped with the game.
Anyway, as I've learned over the years of bringing up this issue on forums, there's no shortage of idiots who claim there's no problem ever with any PC game. They have perfect rigs.. and if those people can help contribute to crowd sourcing shaders for games, then it at least benefits some people. As has been mentioned on the forum numerous times... the best way is a multi pronged approach which covers as many bases as possible.
Flappy Pannus
Veteran
Rootax
Veteran
I would believe that all the Q&A peoples playing the games a lot can help with that ? Some games/engine don't have any problem with that....
Honestly, I'm pretty dumb
I just believed that, in the end, the devs knew their shaders, so they can simply precompiles them during the first load when the engine know the drivers used...
Honestly, I'm pretty dumb
I just believed that, in the end, the devs knew their shaders, so they can simply precompiles them during the first load when the engine know the drivers used...Flappy Pannus
Veteran
It's extremely complex. @Dictator's video is going to be 3 hours long at this rate.I would believe that all the Q&A peoples playing the games a lot can help with that ? Some games/engine don't have any problem with that....
Honestly, I'm pretty dumb
Thanks for the shout, but the answer from the Unreal AMA was in fact to Alex's question, not mine.. just to be clear. They just copy pasted the response to Alex's question as a reply to mine, since it was about the same subject. So all credit goes to Alex!
I just informed him they were doing it.
Also has some stream and load stutter from what I can gather.. fun stuff, right?Oh, a new Unreal Engine 4 game was reviewed today, I wonder if it doesn't have shader compi-
Oh that's good! I would hate for one type of stutter to feel completely at fault for everything when there are others!
Anyway I saw Sam (reviewer) from Ars Technica tweet that the team contacted him and confirming that the game has shader compilation stutter, and stating that they're working on a patch to fix known issues and hope to be able to address the issue with the upcoming patch, but gave no estimate for when that could be. That's nice of them to acknowledge the issue and respond to him. Most wouldn't.
First of all, let me just say thank you to Sam for speaking up about this issue. He posted directly about the issue in his review page, specifically calling it out and not dancing around it by just saying "some frame drops" which some others tend to do. He made a "Good" "Bad" and "Ugly" summary laying it bare. He not only did that but also tweeted about it, and if what he stated in his review is true, he actually messaged Epic about this issue, as he's noticed himself that it is becoming very prevalent in titles using the engine. I know that developers will probably not like that every stutter in existence is being attributed to shader compilation... but too damn bad. You can literally watch the driver caches being created as games hitch lol.. It's definitely the biggest one, and in the end it makes no difference what the consumer attributes the stutter to.. just that it's happening and that it gets fixed/doesn't happen in the future.
This type of reviewing is exactly what we need more of! I guarantee you that if half the other sites and reviewers would do this type of thing as well, and even dock scores when severe enough, that games releasing in this state would be fewer and farther between. It really is imperative that they call this stuff out when they see it.. and Sam from Ars gets a big shout out for having some balls to do it.
And that's the thing.. You see this trend of Unreal Engine games releasing which have these stutter issues at launch, and then sometimes they get fixed, and other times they don't. It's becoming irksome to me. I'm not trying to be rude, but it's getting to the point where I feel like developers are just releasing games which they know have these issues, and they're just waiting to see if enough of a stink is made about it to see if it's worth fixing or not. I don't want to believe that... but come on. This is happening with damn near every release.. there's no way they can't know about it.
The problem is that they keep releasing them anyway... I would prefer them to delay the PC release and state that they need more time, and get it right the first time. I would respect that. It's completely understandable that a new or smaller team may realize late in the project that there's an issue and that they need more time to figure it out. At the end of the day, it's their decision when to release, and they are still releasing it to the public in this condition. There's no excuse for that.
I look forward to the day when we can open the fridge with confidence knowing that there's always something good in there.
And this right here... is why it's imperative that you don't give a single damn about their feelings and continue to call them out...
In a ResetERA thread about Stray being the best user rated game on Steam this year...
Developers simply don't actually care. Just like I said.. the majority of them will put whatever trash out there because they know that most PC gamers will accept it. Put out a game with stutters all over the place... and if... AND ONLY IF.... there's a big stink made about performance... will they look into it and potentially fix it.
All this talk about their being nothing they can do is BS, imo... Many of these games DO actually get fixed through patches... funny how that works... If you listen to developers online talk about this issue, you'd question how PC games even run in the first place.. However the reality is, most of them simply have no damn reason to care until it potentially contributes to a big loss of sales.
I give credit to this guy actually saying what we all figured in the first place. That's why there's so many trash ports out there.
This is why sites like Digital Foundry, and reviewers like Sam from ArsTechnica need to be ruthless and not give them any outs.
In a ResetERA thread about Stray being the best user rated game on Steam this year...
Developers simply don't actually care. Just like I said.. the majority of them will put whatever trash out there because they know that most PC gamers will accept it. Put out a game with stutters all over the place... and if... AND ONLY IF.... there's a big stink made about performance... will they look into it and potentially fix it.
All this talk about their being nothing they can do is BS, imo... Many of these games DO actually get fixed through patches... funny how that works... If you listen to developers online talk about this issue, you'd question how PC games even run in the first place.. However the reality is, most of them simply have no damn reason to care until it potentially contributes to a big loss of sales.
I give credit to this guy actually saying what we all figured in the first place. That's why there's so many trash ports out there.
This is why sites like Digital Foundry, and reviewers like Sam from ArsTechnica need to be ruthless and not give them any outs.
