Digital Foundry Article Technical Discussion [2023]

[Flappy Pannus]

Only *game* updates matter to the IHV shader caches, not *driver* updates, since they are the ones that control the driver invalidation in the first place. Game updates can invalidate shader hashes (if those shaders are touched), but it's not always the case.

I would be surprised if the IHVs didn't harvest the hashes of shaders encountered both in their own benchmarking/test runs but also potentially in the wild (you agree to a lot of stuff with GFE in particular...),

Not quite sure what you're saying here - are you suggesting Nvidia might be harvesting shader hashes from users playing the games locally through GFE for this purpose? I guess it's possible, sure. However as you mentioned with their own necessary benchmarking/testing for the game creating usable caches before releasing it on Geforce Now in the first place, and the fact that a cache created from a single player on the service could be shared amongst thousands of instances. I'd assume that's more likely than pulling data from outside clients with unknown levels of trust.

As you said, doesn't have to be perfect. Maybe there are indeed instances where some players for some games get some shader stutter that Nvidia's own testing missed, but seeing as such caches could be portable across huge swaths of their infrastructure, the sheer volume of players means that 99% of them probably start playing the game with the vast majority of caches created. So Nvidia might not be 'hooking' into the game in any new fashion to create these caches, it's just a matter of scale.

 

[Remij]

Only *game* updates matter to the IHV shader caches, not *driver* updates, since they are the ones that control the driver invalidation in the first place. Game updates can invalidate shader hashes (if those shaders are touched), but it's not always the case.

I would be surprised if the IHVs didn't harvest the hashes of shaders encountered both in their own benchmarking/test runs but also potentially in the wild (you agree to a lot of stuff with GFE in particular...), which is the majority of the data that they need. Testing of titles/updates before they go live on Geforce Now likely covers most of the remaining cases.

Also worth noting... you don't necessarily need *perfect* coverage for every possible shader here to get the vast majority of the benefit. It's ok if a couple rare cases potentially slip through on a game update or similar.

In the Unreal Engine documentation it mentions that it is possible to gather PSO data caches from users.. but that Unreal Engine itself doesn't support/provide the infrastructure to do it.. but that it's something developers could potentially do to crowdsource PSO data and then reintegrate it into the game through updates. Seems possible that Nvidia are essentially doing that with GFE?

Anyway... what about what Valve is doing with Fossilize and Vulkan shaders? Doesn't Fossilize essentially capture PSO blobs, allow them to be saved and distributed to be compiled on different GPUs even without the application being required? Pretty sure I remember reading that Fossilize can do that without the application running.. essentially allowing Steam to download the pre-caches first, and then process them while the app package itself is downloading. I'm not sure it's technically even possible to do with the DX12 API however due to how it was designed.

And yeah some of us have stressed here that a game doesn't need to have 100% perfect coverage. A game can have a small stutter here or there.. nobody would even blink an eye at that. It's when every single time you do some thing which shows some new graphic effect or object and there's a big hitch... and/or when many happen in quick succession.. is when it simply becomes unacceptable.

 

[Andrew Lauritzen]

Seems possible that Nvidia are essentially doing that with GFE?
....
Doesn't Fossilize essentially capture PSO blobs, allow them to be saved and distributed to be compiled on different GPUs even without the application being required?

Yeah there's nothing magic here... PSOs are just a set of input parameters and shader bytecode. IHVs have always been caching the output of the shader compilations for as long as I remember... this is of course how it always worked in DX11 and before too where the application wasn't involved at all.

I'm not sure it's technically even possible to do with the DX12 API however due to how it was designed.

There's still a driver-side cache in addition to anything the application does, but it is true that DX12 and Vulkan arguably make the situation somewhat worse than it was in DX11. It's not *just* because some burden is moved to the application which may or may not participate in the caching (IHVs can and do still cache applications that don't do any of their own manual PSO caching), but the design of PSOs as the superset of state that *may* affect shader compilation on any given GPU actually makes this significantly worse.

In the past when drivers were entirely responsible for this caching, the drivers knew for a given GPU/shader compiler which pipeline states actually affect compilation and which don't. Thus they could maintain a minimal map and not recompile shaders that one differ in PSO inputs that do not affect the output on the users GPU. Unfortunately in DX12 the application actually has no idea which states *really* affect the compilation on a given GPU/driver, so it must take the most conservative path and assume that *every* state affects it and a separate PSO must be cached for each of these (this is enforced by the API or else it would break backwards/forwards compatibility). In reality, only a small subset of these actually produces unique shaders on a given GPU, and while the IHV driver will do some amount of de-duplication, by this point in the process the permutation space has expanded so much that a lot of the damage has already been done.

A game can have a small stutter here or there.. nobody would even blink an eye at that. It's when every single time you do some thing which shows some new graphic effect or object and there's a big hitch... and/or when many happen in quick succession.. is when it simply becomes unacceptable.

In reality I don't think there's going to be a single silver bullet here, at least for DX12 applications. For big games with large shader permutation spaces, it's going to be a combination of a lot of different things from pre-compiling and pre-caching to better handling of unexpected permutations encountered on the fly. As noted in Alex's videos, in many cases shaders can have reasonable fallbacks: either to simpler materials, or simply by deferring rendering an object for a few frames until the shader is ready similar to what virtual textures and other streaming systems would do. Obviously those systems often need various special-case hinting for situations like camera cuts and the like, but for various reasons the default in shader compilation has basically been to stop the world until the shader is ready, and that probably doesn't need to be the default.

Just want to add the usual caveats here - while I'm quite familiar with the graphics APIs and to some extent drivers and how they work, I don't work in the low level render stack of Unreal or similar, so there are additional challenges and subtleties I'm not representing here. Still, at a high level I think it's fair to say that DX12 may have actually been a bit of a regression in this area and made things more complicated (overall, not just for the application) than DX11. That's not even considering raytracing PSOs which have their own set of nasty unsolved problems too!

 

[Andrew Lauritzen]

Not quite sure what you're saying here - are you suggesting Nvidia might be harvesting shader hashes from users playing the games locally through GFE for this purpose? I guess it's possible, sure.

They are very explicit about the fact that they do collect this information, and more: https://www.nvidia.com/en-us/geforce/geforce-experience/license/. I agree that I don't think they *need* to be doing this for the purposes of shader compilation, and they instead primarily do it for the valuable demographics and technical information though.

To be clear though, no "hooking" is required for this information. The PSO compilation requests go through the IHV's user mode driver, as they always have. The consent questions are more just about whether it can collect and send back data through that mechanism, which the license above makes pretty clear that they can if you opt in to GFE at least.

 

[Andrew Lauritzen]

Listening to the latest DF Direct this week, and this question raised my blood pressure:

It is absolutely insane that people are not using NiMH batteries for (basically everything but particularly) their XBox controllers and the question itself seems to not be aware of their existence...? At this point they are way better than alkaline batteries in every way, including being cheap and paying themselves off in no time. They are far *more* "green" than built-in lithium batteries that have a shelf life and require you to throw away the controller.

Thank you John and team for also pointing out how annoying it is to have to plug in a controller to charge it (often while you are trying to play) rather than just do a 2 second battery swap (and recharge those batteries again).

If anyone here is not aware of this... PLEASE get yourselves some Eneloops and never have any of these issues again!

Built-in lithium batteries are good for cases where the size/weight/voltage of NiMH's are not appropriate, but this is absolutely not one of those cases. Same deal with the Oculus controllers - swappable AA is the ideal where possible.

 

[Dr. Nick]

Listening to the latest DF Direct this week, and this question raised my blood pressure:

It is absolutely insane that people are not using NiMH batteries for (basically everything but particularly) their XBox controllers and the question itself seems to not be aware of their existence...? At this point they are way better than alkaline batteries in every way, including being cheap and paying themselves off in no time. They are far *more* "green" than built-in lithium batteries that have a shelf life and require you to throw away the controller.

Thank you John and team for also pointing out how annoying it is to have to plug in a controller to charge it (often while you are trying to play) rather than just do a 2 second battery swap (and recharge those batteries again).

If anyone here is not aware of this... PLEASE get yourselves some Eneloops and never have any of these issues again!

Built-in lithium batteries are good for cases where the size/weight/voltage of NiMH's are not appropriate, but this is absolutely not one of those cases. Same deal with the Oculus controllers - swappable AA is the ideal where possible.

I use the Energizer rechargeable(The ones with green on them) batteries myself. I know they might cost more but in the long run if you have to use AA it will save you money. Usually the pack comes with enough to use the batteries in other applications like remote controls that might need it.

 

[davis.anthony]

Just game on PC with KB/Mouse and never worry about having to charge anything

 

[Phantom88]

^ hehe, we have to charge our wireless mice too But these can game for 70 hours and up fortunatelly. Im waiting for this guy to arrive, any day now

[Bruce Lee Edition] X2 Gaming Mouse

“Simple but not simpler.” We kept this in mind all the time when we designed X2 structure. We designed a structure as simple as possible but maintained its durability. In the result, you are getting high performance gaming mouse that is lighter than the egg in your hand.

www.pulsar.gg

 

[Remij]

Yeah there's nothing magic here... PSOs are just a set of input parameters and shader bytecode. IHVs have always been caching the output of the shader compilations for as long as I remember... this is of course how it always worked in DX11 and before too where the application wasn't involved at all.


There's still a driver-side cache in addition to anything the application does, but it is true that DX12 and Vulkan arguably make the situation somewhat worse than it was in DX11. It's not *just* because some burden is moved to the application which may or may not participate in the caching (IHVs can and do still cache applications that don't do any of their own manual PSO caching), but the design of PSOs as the superset of state that *may* affect shader compilation on any given GPU actually makes this significantly worse.

Drivers have been caching shader output for ages yes, but isn't this different in that regardless of whether the app generates an app-side cache or not, Fossilize will, and it will allow that cache to be downloaded and ran on various IHVs to generate driver-side caches completely independent of the app?

I basically look at it as being similar to how Unreal Engine devs can capture PSO data and ship that code with the game so that shaders can be pre-compiled ahead of time. Fossilize essentially does the same thing, but is independent of any specific engine, and doesn't require the game to be running or even downloaded/installed on the local machine to generate a IHV-side cache.

I guess the magic to me is that I believe it can be done independently from the application, and distributed to users... not really requiring developers to do anything special on their end. Basically one of the biggest issues that Alex and others like myself have here is that many developers simply don't do this application-side for whatever reason. I look at games like Sackboy, and The Callisto Protocol.. which launched in an unacceptable state, but were essentially completely fixed (regarding comp stuttering) after they were updated to pre-compile upon initial load.. It's unfortunate.. but we can't seem to depend on developers doing this on their own, so having a system in place as a backup seems pretty desirable to me.

In reality I don't think there's going to be a single silver bullet here, at least for DX12 applications. For big games with large shader permutation spaces, it's going to be a combination of a lot of different things from pre-compiling and pre-caching to better handling of unexpected permutations encountered on the fly. As noted in Alex's videos, in many cases shaders can have reasonable fallbacks: either to simpler materials, or simply by deferring rendering an object for a few frames until the shader is ready similar to what virtual textures and other streaming systems would do. Obviously those systems often need various special-case hinting for situations like camera cuts and the like, but for various reasons the default in shader compilation has basically been to stop the world until the shader is ready, and that probably doesn't need to be the default.

I definitely agree. The biggest thing is that it seems like with just a little more care put into QA, there's already lots that can be done to largely mitigate the issue, we've seen it time and time again.. but games are seeping through the cracks with egregious issues which signals a lack of basic QA due diligence. I know there's far more to it than what most of us can understand, but we have to bring awareness to the issue, and hopefully get things to change and improve, not only for ourselves as the end users, but developers themselves.

 

[troyan]

^ hehe, we have to charge our wireless mice too But these can game for 70 hours and up fortunatelly. Im waiting for this guy to arrive, any day now

[Bruce Lee Edition] X2 Gaming Mouse

“Simple but not simpler.” We kept this in mind all the time when we designed X2 structure. We designed a structure as simple as possible but maintained its durability. In the result, you are getting high performance gaming mouse that is lighter than the egg in your hand.

www.pulsar.gg

Cool, yellow is just a great color. I have the Cyberpunk version of the razer viper 2. Looks fantastic.

 

[Shifty Geezer]

They are far *more* "green" than built-in lithium batteries that have a shelf life and require you to throw away the controller.

Given availability of alkaline battery recycling, how much is that the case? The maintenance of keeping charged batteries and low cost of bulk-buy alkalines presents a case that recycling spent alkalines is a reasonable alternative to operating NiMHs. In the EU, alkaline recycling is a legal requirement and capacity available in many high-store stores.

 

[iroboto]

Given availability of alkaline battery recycling, how much is that the case? The maintenance of keeping charged batteries and low cost of bulk-buy alkalines presents a case that recycling spent alkalines is a reasonable alternative to operating NiMHs. In the EU, alkaline recycling is a legal requirement and capacity available in many high-store stores.

I think some people are still running their eneloops purchased from their 360 days. Well over 1500 recharges possible on these.

 

[Cyan]

^ hehe, we have to charge our wireless mice too But these can game for 70 hours and up fortunatelly. Im waiting for this guy to arrive, any day now

[Bruce Lee Edition] X2 Gaming Mouse

“Simple but not simpler.” We kept this in mind all the time when we designed X2 structure. We designed a structure as simple as possible but maintained its durability. In the result, you are getting high performance gaming mouse that is lighter than the egg in your hand.

www.pulsar.gg

cool device from the looks of it, although I wonder, what's the connection between a mouse and Bruce Lee? Lightness like feathers? I am used to the Logitech 604 Lightspeed and the Drakonia Sharkoon but never had a super light gaming mouse. I play like 85% on k/m and 15% on controller -according to 2022 Steam's stats-.

There are games where it's impossible to play with k/m without serious compromises, like one of my favourites, Redout 2, while others like Elden Ring, Resident Evil series, etc, I prefer to play with k/m. In Elden Ring the mouse and kb are just more responsive to me. However, I evade enemies better with the gamepad in certain situations.

 

[Cyan]

on a different note, after starting to play Elden Ring in 2023 -my first ever souls like game-, I can understand why it's John Linneman's favourite game of 2022, which he explained in a DF video. The game has me totally absorbed.

This video details why western developers are mad about Elden Ring success. Whether the guy is correct or not about that assumption, he makes some great points during the entire video -some interesting posts from actual developers in the video's comments too-.

 

[Phantom88]

cool device from the looks of it, although I wonder, what's the connection between a mouse and Bruce Lee? Lightness like feathers? I am used to the Logitech 604 Lightspeed and the Drakonia Sharkoon but never had a super light gaming mouse. I play like 85% on k/m and 15% on controller -according to 2022 Steam's stats-.

There are games where it's impossible to play with k/m without serious compromises, like one of my favourites, Redout 2, while others like Elden Ring, Resident Evil series, etc, I prefer to play with k/m. In Elden Ring the mouse and kb are just more responsive to me. However, I evade enemies better with the gamepad in certain situations.

hehe, i didn't think about the connection, i just liked the colour plus it has the most amazing coating, grippy as all hell and fantastic switches But it could be the lightness ) I've been using only ultralight mice since 2019 with the original Model O. Can't use anything else now, it's too ingrained in me. Pulsar has been very popular in the last year on reddit in the gaming mice area so i ordered their ergo shape small one, in blue.

Xlite V2 mini Wireless Gaming Mouse

“Simple but not simpler.” We kept this in mind all the time when we designed Xlite structure. We designed a structure as simple as possible but maintained its durability. In the result, you are getting high performance gaming mouse that is lighter than the egg in your hand.

www.pulsar.gg

And it's a great mouse, i really like it. And my copy is rock solid, you never know with these mice how the QA is gonna be on your copy, but i've been lucky with this one. Then i saw the Bruce Lee one, they only made 3000 copies, couldn't find one at first, then an online store got some copies and i managed to snatch one before they went out again. It's somewhere in sweden right now by the looks of it, on its way to me.

Sorry for small offtopic

 

[chris1515]

on a different note, after starting to play Elden Ring in 2023 -my first ever souls like game-, I can understand why it's John Linneman's favourite game of 2022, which he explained in a DF video. The game has me totally absorbed.

This video details why western developers are mad about Elden Ring success. Whether the guy is correct or not about that assumption, he makes some great points during the entire video -some interesting posts from actual developers in the video's comments too-.

I think Western developer will begin to understand they need to stop with handholding too much. For example TLOU design help to navigate with Ellie walking on the direction where the player needs to go. But in GOW Ragnarok or Horizon the companion or the protagonist begin to shout about what to do next. This is annoying and Sony need to stop this. Sometimes they said where to go when it is a fucking yellow ladder just in the middle of the screen. This is just ridiculous. They need to stop think the player is an idiot. The average age of the gamer is 35 years old. People play for years.

Gamer Demographics: Facts About the Most Popular Hobby

Gaming has become a mainstream form of entertainment, and a big business; in turn, gamers are a new demographic, one that’s more diverse than ever.

dataprot.net

 

[Davros]

I actually prefer a big heavy mouse, not that easy to find now days

 

[see colon]

I think Western developer will begin to understand they need to stop with handholding too much. For example TLOU design help to navigate with Ellie walking on the direction where the player needs to go. But in GOW Ragnarok or Horizon the companion or the protagonist begin to shout about what to do next. This is annoying and Sony need to stop this. Sometimes they said where to go when it is a fucking yellow ladder just in the middle of the screen. This is just ridiculous. They need to stop think the player is an idiot. The average age of the gamer is 35 years old. People play for years.

I think this is on purpose, though. I mean, obviously it's on purpose because they had someone voice those lines and put it in the game. But I think the design, where you have visual and auditory cues to help the player through the game aren't just there for "hand holding", but for accessibility. There has been a big push in the industry lately for accessibility, and subtle change like giving auditory and visual feedback allows for double the coverage of sensory feedback. I understand where you are coming from, and honestly I think the solution is to have an option that turns off/down the nagging. But I think they put it there for a reason that isn't just that they think players are dumb.

 

[chris1515]

I think this is on purpose, though. I mean, obviously it's on purpose because they had someone voice those lines and put it in the game. But I think the design, where you have visual and auditory cues to help the player through the game aren't just there for "hand holding", but for accessibility. There has been a big push in the industry lately for accessibility, and subtle change like giving auditory and visual feedback allows for double the coverage of sensory feedback. I understand where you are coming from, and honestly I think the solution is to have an option that turns off/down the nagging. But I think they put it there for a reason that isn't just that they think players are dumb.

+1

The visual cue is enough at least give the option to turn it off.

 

[Cyan]

I think Western developer will begin to understand they need to stop with handholding too much. For example TLOU design help to navigate with Ellie walking on the direction where the player needs to go. But in GOW Ragnarok or Horizon the companion or the protagonist begin to shout about what to do next. This is annoying and Sony need to stop this. Sometimes they said where to go when it is a fucking yellow ladder just in the middle of the screen. This is just ridiculous. They need to stop think the player is an idiot. The average age of the gamer is 35 years old. People play for years.

Gamer Demographics: Facts About the Most Popular Hobby

Gaming has become a mainstream form of entertainment, and a big business; in turn, gamers are a new demographic, one that’s more diverse than ever.

dataprot.net

another example of that is Gotham Knights, which is a game that I liked though (the story, bat girl, characters and stuff), but disliked the fact that during night patrols sometimes I got to a point where my mind was like "hope no new icons showing where things are happening appear I don't feel like stopping at every new location icon". You gain experience from that but you start moving and darn, a new icon appears of some baddy beating some poor human being and stuff like that, you interrupt the path you were taking to go to certain place, fight for a minute, and just move to the next location icon.

The way Gotham Knights is designed forces the user's handholding though. You never get to learn the city's streets for the most part, you just follow the icons and that's it. You get to learn the map in Elden Ring as vast as it is, 'cos you walk and run at your own leisure and you want to go everywhere to see what you might find.