Shader Complilation on PC: About to become a bigger bottleneck?

So apparently the Day-1 patch for Horizon:Zero Dawn on PC will now compile all shaders at startup and create its cache after every new driver release - for 10-15 minutes. This will supposedly help reduce stuttering during gameplay.

This is a particular egregious example and from early accounts the port is somewhat shoddy all-around, but I've thought about this becoming a problem ever since the new consoles SSD architecture was unveiled. I've certainly noticed it in other games, Dishonored 2 takes 2-3 minutes to load after every new driver update (or when it just decides to recompile the shaders for the hell of it), the Call of Duty games can take an inordinate time to compile shaders as well if you choose than option. Loading levels in Mankind Divided after a new driver update can take considerably longer as well vs. playing it for a while on the same driver revision.

Shader compilation can normally be 'hidden' on PC titles during level loads or run in the background, but that can induce stuttering, and the risk of it happening is far greater in large open world games with lots of streaming and few, if any level load intermissions. Not having to compile shaders on consoles may be far more significant in overall load times when their storage/CPU speed is roughly equivalent to PC's as opposed to now.

Even if Microsoft for example were to provide a suspend-to-disk option for apps/games in a revision of Windows 10 that would allow a similar suspend/resume than the Series X is offering, how would that deal with a driver update when all the shaders would have to be recompiled? I'm sure the OS could tell the game "hold on, have to recompile shaders", but having to do so frequently would kind of suck out the utility of such a quick resume feature.

Steam does have the ability to download shader caches, but honestly I don't think I've ever seen an example of it actually working. So I'm curious - are there any approaches to this which could help alleviate this? Driver updates are frequent enough where I can see this requirement that consoles don't need would mean PC load times may end up being the longest of any platform, at least in the near future.
 

BRiT

(>• •)>⌐■-■ (⌐■-■)
Moderator
Legend
Alpha
All the more reason for there to be a game shader by driver repository where then consumers merely need the game to just download the appropriate package.
 

BRiT

(>• •)>⌐■-■ (⌐■-■)
Moderator
Legend
Alpha
Isn't this exactly the purpose of Steam's downloadable shader cache?

Don't know. Never used it. What I'm talking about is where the user doesn't have to do anything, it just happens, like it does on consoles, where its part of the game or part of the launching platform. Doesn't sound like the PC has anything like that at all yet.
 

iroboto

Daft Funk
Legend
Subscriber
So apparently the Day-1 patch for Horizon:Zero Dawn on PC will now compile all shaders at startup and create its cache after every new driver release - for 10-15 minutes. This will supposedly help reduce stuttering during gameplay.

This is a particular egregious example and from early accounts the port is somewhat shoddy all-around, but I've thought about this becoming a problem ever since the new consoles SSD architecture was unveiled. I've certainly noticed it in other games, Dishonored 2 takes 2-3 minutes to load after every new driver update (or when it just decides to recompile the shaders for the hell of it), the Call of Duty games can take an inordinate time to compile shaders as well if you choose than option. Loading levels in Mankind Divided after a new driver update can take considerably longer as well vs. playing it for a while on the same driver revision.

Shader compilation can normally be 'hidden' on PC titles during level loads or run in the background, but that can induce stuttering, and the risk of it happening is far greater in large open world games with lots of streaming and few, if any level load intermissions. Not having to compile shaders on consoles may be far more significant in overall load times when their storage/CPU speed is roughly equivalent to PC's as opposed to now.

Even if Microsoft for example were to provide a suspend-to-disk option for apps/games in a revision of Windows 10 that would allow a similar suspend/resume than the Series X is offering, how would that deal with a driver update when all the shaders would have to be recompiled? I'm sure the OS could tell the game "hold on, have to recompile shaders", but having to do so frequently would kind of suck out the utility of such a quick resume feature.

Steam does have the ability to download shader caches, but honestly I don't think I've ever seen an example of it actually working. So I'm curious - are there any approaches to this which could help alleviate this? Driver updates are frequent enough where I can see this requirement that consoles don't need would mean PC load times may end up being the longest of any platform, at least in the near future.
hmm...
I'm guessing consoles do not have to compile shaders?
 

Lurkmass

Regular
hmm...
I'm guessing consoles do not have to compile shaders?

They don't or at least not at runtime usually because the shaders on consoles are shipped as precompiled binaries ...

On PC, the shaders can't be completely precompiled like we see on consoles so they do a JIT compilation pass to translate these shaders into native hardware binaries. This is done for hardware architectural design and compatibility reasons ...
 

iroboto

Daft Funk
Legend
Subscriber
They don't or at least not at runtime usually because the shaders on consoles are shipped as precompiled binaries ...

On PC, the shaders can't be completely precompiled like we see on consoles so they do a JIT compilation pass to translate these shaders into native hardware binaries. This is done for hardware architectural design and compatibility reasons ...
thanks for the insight
 

BRiT

(>• •)>⌐■-■ (⌐■-■)
Moderator
Legend
Alpha
Or maybe only do it for the section of the game you're at/upto. Surely you don't need end-game shades setup when you're just starting out.
 
Sure. Don't do shader compilation for all shaders of the game on game's launch. Do it like the rest of PC titles do - in the background, even during gameplay.
But a lot of games don't, as it can cause stuttering problems - as it did in Death Stranding, plenty of posts where people were complaining about choppy framerates during the first few minutes of gameplay, when it was discovered it was because it was compiling shaders in the background. The Call of Duty games offer the precompiling-at-start as an option as leaving it in the background can induce stuttering. Metro Exodus requires it when just changing certain graphical settings.

Like I said, 'even during gameplay' isn't really an option going forward when games are basically streamed in constantly from an SSD.
 
Last edited:

Lurkmass

Regular

To date, the Mantle API still has a superior compilation model compared to it's replacements like D3D12 or Vulkan ...

JITing would still exist but compilation times would be measured in the single digit minutes rather than in double digit minutes like we see now which would be an order of a magnitude in improvement. JIT compilation is really heavy on the drivers so a lower level bytecode representation like AMDIL on Mantle would be highly helpful in this scenario ...
 

iroboto

Daft Funk
Legend
Subscriber
To date, the Mantle API still has a superior compilation model compared to it's replacements like D3D12 or Vulkan ...

JITing would still exist but compilation times would be measured in the single digit minutes rather than in double digit minutes like we see now which would be an order of a magnitude in improvement. JIT compilation is really heavy on the drivers so a lower level bytecode representation like AMDIL on Mantle would be highly helpful in this scenario ...
what determines compile times mainly? CPU?
 
Top