Shader Complilation on PC: About to become a bigger bottleneck?

Discussion in 'Rendering Technology and APIs' started by Flappy Pannus, Aug 6, 2020.

  1. Flappy Pannus

    Newcomer

    Joined:
    Jul 4, 2016
    Messages:
    145
    Likes Received:
    172
    So apparently the Day-1 patch for Horizon:Zero Dawn on PC will now compile all shaders at startup and create its cache after every new driver release - for 10-15 minutes. This will supposedly help reduce stuttering during gameplay.

    This is a particular egregious example and from early accounts the port is somewhat shoddy all-around, but I've thought about this becoming a problem ever since the new consoles SSD architecture was unveiled. I've certainly noticed it in other games, Dishonored 2 takes 2-3 minutes to load after every new driver update (or when it just decides to recompile the shaders for the hell of it), the Call of Duty games can take an inordinate time to compile shaders as well if you choose than option. Loading levels in Mankind Divided after a new driver update can take considerably longer as well vs. playing it for a while on the same driver revision.

    Shader compilation can normally be 'hidden' on PC titles during level loads or run in the background, but that can induce stuttering, and the risk of it happening is far greater in large open world games with lots of streaming and few, if any level load intermissions. Not having to compile shaders on consoles may be far more significant in overall load times when their storage/CPU speed is roughly equivalent to PC's as opposed to now.

    Even if Microsoft for example were to provide a suspend-to-disk option for apps/games in a revision of Windows 10 that would allow a similar suspend/resume than the Series X is offering, how would that deal with a driver update when all the shaders would have to be recompiled? I'm sure the OS could tell the game "hold on, have to recompile shaders", but having to do so frequently would kind of suck out the utility of such a quick resume feature.

    Steam does have the ability to download shader caches, but honestly I don't think I've ever seen an example of it actually working. So I'm curious - are there any approaches to this which could help alleviate this? Driver updates are frequent enough where I can see this requirement that consoles don't need would mean PC load times may end up being the longest of any platform, at least in the near future.
     
  2. BRiT

    BRiT (>• •)>⌐■-■ (⌐■-■)
    Moderator Legend Alpha

    Joined:
    Feb 7, 2002
    Messages:
    18,795
    Likes Received:
    21,097
    All the more reason for there to be a game shader by driver repository where then consumers merely need the game to just download the appropriate package.
     
    Remij and digitalwanderer like this.
  3. Flappy Pannus

    Newcomer

    Joined:
    Jul 4, 2016
    Messages:
    145
    Likes Received:
    172
    Isn't this exactly the purpose of Steam's downloadable shader cache?
     
    g__day and digitalwanderer like this.
  4. BRiT

    BRiT (>• •)>⌐■-■ (⌐■-■)
    Moderator Legend Alpha

    Joined:
    Feb 7, 2002
    Messages:
    18,795
    Likes Received:
    21,097
    Don't know. Never used it. What I'm talking about is where the user doesn't have to do anything, it just happens, like it does on consoles, where its part of the game or part of the launching platform. Doesn't sound like the PC has anything like that at all yet.
     
    digitalwanderer likes this.
  5. iroboto

    iroboto Daft Funk
    Legend Regular Subscriber

    Joined:
    Mar 6, 2014
    Messages:
    13,021
    Likes Received:
    15,765
    Location:
    The North
    hmm...
    I'm guessing consoles do not have to compile shaders?
     
    digitalwanderer likes this.
  6. Lurkmass

    Regular Newcomer

    Joined:
    Mar 3, 2020
    Messages:
    306
    Likes Received:
    345
    They don't or at least not at runtime usually because the shaders on consoles are shipped as precompiled binaries ...

    On PC, the shaders can't be completely precompiled like we see on consoles so they do a JIT compilation pass to translate these shaders into native hardware binaries. This is done for hardware architectural design and compatibility reasons ...
     
  7. iroboto

    iroboto Daft Funk
    Legend Regular Subscriber

    Joined:
    Mar 6, 2014
    Messages:
    13,021
    Likes Received:
    15,765
    Location:
    The North
    thanks for the insight
     
    digitalwanderer likes this.
  8. digitalwanderer

    digitalwanderer Dangerously Mirthful
    Legend

    Joined:
    Feb 19, 2002
    Messages:
    18,163
    Likes Received:
    2,775
    Location:
    Winfield, IN USA
    10-15 minutes? On a console or PC?!?

    Great thread btw, fascinating stuff. Thank you!
     
  9. BRiT

    BRiT (>• •)>⌐■-■ (⌐■-■)
    Moderator Legend Alpha

    Joined:
    Feb 7, 2002
    Messages:
    18,795
    Likes Received:
    21,097
    That's PC shader compilation times. Consoles don't suffer this.
     
    digitalwanderer likes this.
  10. DegustatoR

    Veteran

    Joined:
    Mar 12, 2002
    Messages:
    2,206
    Likes Received:
    1,601
    Location:
    msk.ru/spb.ru
    Sure. Don't do shader compilation for all shaders of the game on game's launch. Do it like the rest of PC titles do - in the background, even during gameplay.
     
    digitalwanderer likes this.
  11. BRiT

    BRiT (>• •)>⌐■-■ (⌐■-■)
    Moderator Legend Alpha

    Joined:
    Feb 7, 2002
    Messages:
    18,795
    Likes Received:
    21,097
    Or maybe only do it for the section of the game you're at/upto. Surely you don't need end-game shades setup when you're just starting out.
     
    digitalwanderer likes this.
  12. Flappy Pannus

    Newcomer

    Joined:
    Jul 4, 2016
    Messages:
    145
    Likes Received:
    172
    But a lot of games don't, as it can cause stuttering problems - as it did in Death Stranding, plenty of posts where people were complaining about choppy framerates during the first few minutes of gameplay, when it was discovered it was because it was compiling shaders in the background. The Call of Duty games offer the precompiling-at-start as an option as leaving it in the background can induce stuttering. Metro Exodus requires it when just changing certain graphical settings.

    Like I said, 'even during gameplay' isn't really an option going forward when games are basically streamed in constantly from an SSD.
     
    #12 Flappy Pannus, Aug 6, 2020
    Last edited: Aug 6, 2020
    digitalwanderer likes this.
  13. TheAlSpark

    TheAlSpark Moderator
    Moderator Legend

    Joined:
    Feb 29, 2004
    Messages:
    22,146
    Likes Received:
    8,531
    Location:
    ಠ_ಠ
    Forza Horizon 4 also compiles every now and then, but the annoying part is that it forgets the settings every single time. :roll:
     
    orangpelupa likes this.
  14. digitalwanderer

    digitalwanderer Dangerously Mirthful
    Legend

    Joined:
    Feb 19, 2002
    Messages:
    18,163
    Likes Received:
    2,775
    Location:
    Winfield, IN USA
    Hmm. How about setting it up to do it during the pc down time, like windows update is supposed to work.

    ("supposed" because me and Microsoft seem to have very different ideas for when that time should be.)
     
    dobwal, orangpelupa and BRiT like this.
  15. Lurkmass

    Regular Newcomer

    Joined:
    Mar 3, 2020
    Messages:
    306
    Likes Received:
    345
    I'm starting to think we should just bring back the Mantle API ...
     
  16. TheAlSpark

    TheAlSpark Moderator
    Moderator Legend

    Joined:
    Feb 29, 2004
    Messages:
    22,146
    Likes Received:
    8,531
    Location:
    ಠ_ಠ
    hm?
     
  17. Lurkmass

    Regular Newcomer

    Joined:
    Mar 3, 2020
    Messages:
    306
    Likes Received:
    345
    To date, the Mantle API still has a superior compilation model compared to it's replacements like D3D12 or Vulkan ...

    JITing would still exist but compilation times would be measured in the single digit minutes rather than in double digit minutes like we see now which would be an order of a magnitude in improvement. JIT compilation is really heavy on the drivers so a lower level bytecode representation like AMDIL on Mantle would be highly helpful in this scenario ...
     
    digitalwanderer, BRiT and TheAlSpark like this.
  18. TheAlSpark

    TheAlSpark Moderator
    Moderator Legend

    Joined:
    Feb 29, 2004
    Messages:
    22,146
    Likes Received:
    8,531
    Location:
    ಠ_ಠ
    Thanks for explaining. :)
     
    digitalwanderer likes this.
  19. DegustatoR

    Veteran

    Joined:
    Mar 12, 2002
    Messages:
    2,206
    Likes Received:
    1,601
    Location:
    msk.ru/spb.ru
    It's an implementation issue. Still doesn't mean that you have to compile all shaders at a game's launch. There are more than enough games which do compilation as required and do not stutter.
     
    BRiT likes this.
  20. iroboto

    iroboto Daft Funk
    Legend Regular Subscriber

    Joined:
    Mar 6, 2014
    Messages:
    13,021
    Likes Received:
    15,765
    Location:
    The North
    what determines compile times mainly? CPU?
     
Loading...

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...