Shader Compilation on PC: About to become a bigger bottleneck?

[Flappy Pannus]

You Can Use Vulkan Without Pipelines Today​

Today, Khronos® is releasing a new multi-vendor Vulkan® extension that aims to radically simplify how applications specify shaders and shader state while maintaining Vulkan’s ethos of being a highly performant “API without secrets.”

This extension is VK_EXT_shader_object. It introduces a new VkShaderEXT object type which represents a single compiled shader stage, along with 4 new functions to manipulate and use VkShaderEXT objects.

Shader objects serve a role similar to pipelines but expose a simpler, more flexible interface designed to empower applications whose architectures and/or needs for “dynamism” might have previously made the use of pipelines difficult. In environments where VK_EXT_shader_object is supported, applications can choose to use only pipelines, only shader objects, or an arbitrary mix of the two.

In comparison with pipelines, shader objects differ in a number of key ways:

• Pipelines require every desired combination of shaders (all stages when using conventional “monolithic” pipelines, or certain pre-defined combinations of stages when using pipeline libraries) to be compiled together, whereas shader objects allow stages to be compiled in arbitrary combinations, including compiling every shader individually.
• With pipelines, linking is an explicit step that creates a new object with its own lifetime that needs to be managed by the application. With shader objects, linking is simply a creation-time promise from the application to the implementation that it will always use certain combinations of shader objects together.
• Pipelines must always be linked before being used, whereas linking shader objects is optional.
• Pipelines allow implementations to require some state to be statically provided at compile time, whereas with shader objects all state is always set dynamically and is independent from shaders.
• Pipelines require rendering attachment formats to be specified at pipeline creation time, whereas shader objects don’t. Shader objects can be used with any valid combination of attachment formats supported by the device.
• With pipelines, compiled shader code can be retrieved and reused by the application in the form of a pipeline cache, but this data is not usable by itself to create new pipelines. With shader objects, compiled shader code can be retrieved directly from any shader object and is guaranteed to be usable to create an equivalent shader object on any compatible physical device without the need to provide the original SPIR-V.

In summary, shader objects impose substantially fewer restrictions on applications compared to pipelines, and enable dynamism-heavy applications like games and game engines to avoid the explosive pipeline permutation combinatorics, which until now might have been seen as a cost of admission for access to modern graphics APIs.

 

[Scott_Arm]

Was this new extension primarily driven by Nintendo?

Vulkan-ExtensionLayer/layers at main · KhronosGroup/Vulkan-ExtensionLayer

Layer providing Vulkan features when native support is unavailable - KhronosGroup/Vulkan-ExtensionLayer

github.com

A whole bunch of the files are copyright Nintendo. They use Nvidia gpus and I'm wondering if the space constraints in shipping Switch games drove this.

/*
* Copyright 2023 Nintendo
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

This section from the problem statement really sounds like it could be talking about Nintendo game cartridges since they're relatively small and probably expensive.

Vulkan-Docs/proposals/VK_EXT_shader_object.adoc at main · KhronosGroup/Vulkan-Docs

The Vulkan API Specification and related tools. Contribute to KhronosGroup/Vulkan-Docs development by creating an account on GitHub.

github.com

Finally, on fixed hardware platforms like game consoles and embedded systems pipelines have created some additional and unique challenges. These platforms tend to have limited CPU performance, memory, and storage capacity all at the same time. Because of this it is generally not desirable for applications on these platforms to waste storage space shipping both uncompiled SPIR-V and precompiled pipeline caches, however it is also not desirable to compile the same shaders from scratch on each system (even if they could be cached for subsequent runs). Also, the hardware and even driver versions on these systems are typically known in advance, and drivers might only ever change in tandem with applications. Vulkan applications on these systems are forced to waste precious storage space on not only shipping both SPIR-V and pipeline cached versions of their shaders, but on their pipeline caches containing potentially large numbers of slightly differently optimized permutations of the same shader code, with only minor differences in pipeline state (arguably this last point is a compression problem, but opaque pipeline caches mostly leave applications at the mercy of the driver to solve it for them).

Edit:
Also Valve contributors, which would make sense with steam deck. The main contact is a person at Nintendo (I think), Daniel Story.

VK_EXT_shader_object(3)

 

[Lurkmass]

Was this new extension primarily driven by Nintendo?

Vulkan-ExtensionLayer/layers at main · KhronosGroup/Vulkan-ExtensionLayer

Layer providing Vulkan features when native support is unavailable - KhronosGroup/Vulkan-ExtensionLayer

github.com

A whole bunch of the files are copyright Nintendo. They use Nvidia gpus and I'm wondering if the space constraints in shipping Switch games drove this.



This section from the problem statement really sounds like it could be talking about Nintendo game cartridges since they're relatively small and probably expensive.

Vulkan-Docs/proposals/VK_EXT_shader_object.adoc at main · KhronosGroup/Vulkan-Docs

The Vulkan API Specification and related tools. Contribute to KhronosGroup/Vulkan-Docs development by creating an account on GitHub.

github.com

Edit:
Also Valve contributors, which would make sense with steam deck. The main contact is a person at Nintendo (I think), Daniel Story.

VK_EXT_shader_object(3)

So there were likely 3 interested parties for obvious reasons ...

Nvidia is interested since the extension is a good match for their hardware ...

Nintendo is interested since Vulkan is available on their current console and they want to bypass shipping SPIR-V binaries over there in favour of pre-compiled separate shader objects as opposed to pre-compiled PSOs which takes up more storage memory (caused by redundant pipelines). They also use Nvidia hardware as well ...

Collabora has a side project where they use Mesa infrastructure to implement OpenGL over Vulkan translation layer (Zink). Their use case for this extension is OpenGL emulation ...

Valve graphics programmers are divided. Their D3D translation layer developers detest the functionality. As for their user-space driver developers (RADV/radeonsi), there's no real case for them to implement the extension either since they offer and support native OpenGL drivers so I imagine the desire to implement the extension is purely out of academic interest for them ...

The last two aforementioned parties are likely the reason as to why the extension is multi-vendor despite the fact that there's no other hardware vendors who are truly interested ...

 

[Remij]

Get the feeling a lot of stuff is happening behind the scenes now and it's moving pretty quickly. For better or worse we'll see what happens.

The folks working on DirectX have stated they have upcoming features which will be announced in the coming months, and I'm wondering if they're not going to be doing something similar.

 

[DavidGraham]

Regarding this vendor thing, Intel is a non factor, they have no market share, no presence and no leverage, they will also modify their arch in a heart beat to stay competitive, Qualcomm is not even in the PC space to even mention, who even mentions them and for what purpose? They don't even support half of DX12U features!

The only real factor here is AMD and NVIDIA, AMD strong armed Microsoft and the whole industry into this position where literally no one benefits. We all lose, perhaps AMD has decided they can not write proper DX11 drivers or maybe they want to save some pretty penny and share desktop hardware with console hardware.

AMD is probably less flexible than ever for this reason, they have to support a decade of legacy architecture decisions because they need backward compatibility for consoles and inter-compatibility with PC and consoles at the same time. For which they are dragging the whole industry into a lockdown and into the unknown. FWIW NVIDIA did the same thing with DXR in 2018, but they had the hardware, software and research for it and were ready before everyone else. And to be fair, AMD too held many cards back then, as they had all of consoles in 2013, so they had better leverage, also back then, everyone thought If PC can become a little bit like consoles then it's a good win, now we know that's actually wrong.

Anyway, considering NVIDIA now has the largest market share by far, and considering they are ahead with DXR and AI, I think it maybe time for them to use this as a leverage to either force a new DX12 extension to be better compatible with their hardware or maybe fork out their own Mantle like extension.

Get the feeling a lot of stuff is happening behind the scenes now and it's moving pretty quickly

I totally agree, 8 years ago I asked this very forum whether Vulkan (and by extension DX12) was a GCN low level construct, I received a solid NO NO answer, yet 8 years later we all received the proper truthful answer, a resounding YES. I don't blame anyone but me, I wasn't sharp enough to realize that back room deals and behind the scene stuff still control even a seemingly neutral world like the tech world.

 

[Scott_Arm]

@DavidGraham I don't know if we can say that AMD strong-armed anybody, when Microsoft was the one releasing DX12 and building Xbox with AMD parts inside. I also can't believe that Nvidia would have been kept on the sidelines for any of this stuff. I don't think Nvidia or AMD wanted to continue with constant driver hacks to fix broken DX11 games. The DX12 and Vulkan model was supposed to lead to explicit game programming and simpler drivers. Maybe it didn't totally work out, but I don't think it's a conspiracy or anything.

 

[troyan]

Reading the introduction makes it clear that the API is not the problem. It hasnt just worked out the way people believed it would:

Many of these assumptions have since proven to be unrealistic.

 

[Remij]

https://twitter.com/x/status/1642905513745862657

Yay! haha
I can honestly say, the amount of games that are doing this now compared to just a year or so ago and the whole #stutterstruggle movement began is telling. Being vocal DOES bring about change.. It's really weird because the functionality has been there for a long time, and games just didn't do it for whatever reason. Word is out now and I'm thankful we're seeing more and more devs do this to improve the experience for their players. Good stuff!

 

[DegustatoR]

So there were likely 3 interested parties for obvious reasons ...

To me it looks more like there is just one party who isn't interested for obvious reasons. The rest are either indifferent or think that this is a good idea.

 

[Lurkmass]

Maybe Nvidia should make their own proprietary API to prove a point instead of leeching off the success of D3D which is Microsoft's property and can do whatever they want. More freebies for AMD and Intel from Microsoft if Nvidia does stop participating ...

A message from one of the Direct3D designers himself in response to a user mentioning about the feature during exchanges from discord:

none if you ask VK_EXT_shader_object
which is kinda cool, i think eventually all APIs will start pivoting to pipeline-less rendering

That's a Vulkan extension, not a D3D12 extension

im aware, just wanted to use it as an example

since its not uncommon for directx to adopt some decisions from vulkan if they end up working very well

At least in the time that I've been here, it has been quite uncommon - D3D has often published features first. That said, yes, given our mapping efforts for GL/CL/Vk, yes we've added some things. We're watching and listening, but I wouldn't count on it

So they're clearly watching how the it'll turn out but even they're leaning in doubt that they'll expose this capability so far ...

Nvidia should get cozy as soon as possible with opposition from both AMD and Intel indefinitely holding out the idea of it ever happening. Intel has an even stronger preference for PSOs than AMD does if their D3D11 drivers and preference for D3D12 is anything to go by ...

 

[Scott_Arm]

@Lurkmass Wrote it earlier in the thread, but Nvidia has like 85% of the dgpu market, so they probably could roll their own api for pc gaming without much issue. They could provide a translation layer from D3D or Vulkan for their native api. I kind of wonder if that's not the ideal case for both Nvidia and AMD.

 

[Remij]

@Lurkmass Wrote it earlier in the thread, but Nvidia has like 85% of the dgpu market, so they probably could roll their own api for pc gaming without much issue. They could provide a translation layer from D3D or Vulkan for their native api. I kind of wonder if that's not the ideal case for both Nvidia and AMD.

Ugh, we really don't want to go this route I don't think.

 

[BRiT]

@Lurkmass Wrote it earlier in the thread, but Nvidia has like 85% of the dgpu market, so they probably could roll their own api for pc gaming without much issue. They could provide a translation layer from D3D or Vulkan for their native api. I kind of wonder if that's not the ideal case for both Nvidia and AMD.

Possibly, but then that puts them smack dab back into the region of custom tailored drivers for every new game coming out like the times of DX11. I don't think any manufacturer wants to go back to needig separate driver internal codepaths for every individual popular game.

 

[Scott_Arm]

Ugh, we really don't want to go this route I don't think.

Only reason I'm saying that is it's very easy to find people saying they wanted to learn vulkan or dx12, but it was too hard so they went back to dx11 or opengl. To me, if we're going to have a high level language that's more abstraction, that's a good place for D3D and Vulkan. If we're going low level, it probably makes more sense to have apis that actually map to the hardware. It's clear that AMD and Nvidia actually aren't the same.

I get it. We don't want games that support Nvidia API but not the AMD and vice versa. But tons of games are going UE, and UE supports Vulkan, DX11, DX12, Metal, PS5 and more shit for mobile. Lots of game engines support support Xbox, PS5 and PC. They basically write an abstraction in engine to support multiple back ends for their renderers. Maybe Nvidia, AMD apis would actually fit that nicely.

 

[Flappy Pannus]

Developer Technology Engineer @ AMD is less than impressed apparently:

https://twitter.com/x/status/1642922396293185537

 

[DegustatoR]

Maybe Nvidia should make their own proprietary API to prove a point instead of leeching off the success of D3D which is Microsoft's property and can do whatever they want. More freebies for AMD and Intel from Microsoft if Nvidia does stop participating ...

They kinda do, it's called NVAPI and allows to do things which D3D doesn't. Same with AMD's AGS btw.

A message from one of the Direct3D designers himself in response to a user mentioning about the feature during exchanges from discord:

That message completely avoids the fact that the whole D3D12 is a result of Microsoft looking at Mantle.

Possibly, but then that puts them smack dab back into the region of custom tailored drivers for every new game coming out like the times of DX11. I don't think any manufacturer wants to go back to needig separate driver internal codepaths for every individual popular game.

Not necessarily. Also this is the case with D3D12 as well right now but with all the stutters and issues to boot.

 

[DavidGraham]

@DavidGraham I don't know if we can say that AMD strong-armed anybody, when Microsoft was the one releasing DX12 and building Xbox with AMD parts inside.

I want to believe that, I truly do, it would make me sleep better at night knowing the tech world operates with a little bit of integrity. But the events back then happened in a certain order, AMD held to the open source mantra back then, especially against NVIDIA's GameWorks and PhysX programs, but suddenly they leaped over NVIDIA and came up with a massively more proprietary feature, a whole API (Mantle), which was clearly against their mantra.

Shortly after, Microsoft quickly assembled DX12 (with backward compatibility for DX11 hardware for maybe the first time ever, where a new DX core is comaptible with old DX hardware), Vulkan was quickly formed too and declared itself based on Mantle.

Mantle itself was a very short lived API, designed from the start to strong arm Microsoft into releasing DX12 and adopting the Mantle core. AMD wouldn't have released Mantle if DX12 and Vulkan were being prepared and about to be released soon. It happened like this:

2013: Mantle was co developed between AMD and DICE.
2014: Microsot announced DX12, Mantle development began to dwindle down
2015: Microsoft released DX12, Mantle development suspended.
2016: Vulkan was released on the basis of Mantle.

So you see, it all happened too quickly and in haste, there was no industry wide consensus, only AMD releasing an exclusive API, then ending it's life prematurly. AMD wouldn't have released Mantle and spent all the cost associated with debugging and releasing it if DX12 was known to be in development, and about to be released soon. So clearly DX12 wasn't a thing before Mantle, and only came to fruition AFTER Mantle, not before. So the whole thing is suspicious.

I don't think any manufacturer wants to go back to needig separate driver internal codepaths for every individual popular game.

It still happens with DX12, driver involvement is still very much a thing, and NVIDIA does it even more with DX12.

The D3D12 binding model causes some grief on Nvidia HW. Microsoft forgot to include STATIC descriptors in RS 1.0 which then got fixed with RS 1.1 but no developers use RS 1.1 so in the end Nvidia likely have app profiles or game specific hacks in their drivers. Mismatched descriptor types are technically undefined behaviour in D3D12 but there are now cases in games where shaders are using sampler descriptors in place of UAV descriptors but somehow it works without crashing! No one has an idea of what workaround Nvidia is applying.

Nvidia's 3000 Series RTX GPU [3090s with different memory capacity]

I'm disputing the Steam Survey since it's obviously a system full of flukes & flaws lacking even basic sanity checks on the provided data... I often see people claiming Steam stats are flawed. What I never see is an explanation of why those flaws would favor one product over another in a large...

forum.beyond3d.com

 

[Lurkmass]

@Lurkmass Wrote it earlier in the thread, but Nvidia has like 85% of the dgpu market, so they probably could roll their own api for pc gaming without much issue. They could provide a translation layer from D3D or Vulkan for their native api. I kind of wonder if that's not the ideal case for both Nvidia and AMD.

They very well could roll out their own API but then backwards compatibility mostly becomes their concern rather than Microsoft's and as D3D drifts away overtime from their hardware design and more closely resembles the active participants left, a translation layer starts for D3D in the future starts becoming a nasty proposition when there is little in commonality left ...

Why emulate with less performance when I can get the very hardware itself that operates exactly as the API advertises ?

They kinda do, it's called NVAPI and allows to do things which D3D doesn't. Same with AMD's AGS btw.

Driver extensions don't serve as an equivalent for the core default API design itself ...

That message completely avoids the fact that the whole D3D12 is a result of Microsoft looking at Mantle.

Because Microsoft can act however they prefer and they're giving us a glimpse of their true colour on this topic. It's as they say, "don't count on it" ...

 

[DegustatoR]

Because Microsoft can act however they prefer and they're giving us a glimpse of their true colour on this topic. It's as they say, "don't count on it" ...

Which is exactly what they were saying publicly about a new DX back when Mantle was introduced

Also I think it's a bit funny that you suggest Nvidia do the exact same thing which AMD did and for what were the exact same (stated) reasons - that MS wouldn't do anything with the API problem they had.

 

[DavidGraham]

Also I think it's a bit funny that you suggest Nvidia do the exact same thing which AMD did and for what were the exact same (stated) reasons - that MS wouldn't do anything with the API problem they had.

A couple of months ago he suggested Microsoft should just kill DXR and make a new API that favors AMD! Or give AMD some API advantage in DXR!

Microsoft should start unilaterally giving either AMD or Intel an API advantage for once instead so that it will totally motivate Nvidia to implement a real solution or else they lose out ...

AMD RDNA3 Specifications Discussion Thread

Meh. I want room sized, British cold fusion reactors first. Bullshit? Likely not. They sold ARM because they no longer need it. Meh. Give me a rdna4 PSP instead. best I can do is a rdna 4 steam deck.

forum.beyond3d.com

Hence why we should propose Microsoft to extract concessions out of Nvidia to get whatever they want from them since they clearly have the power to reverse this trend otherwise AMD should feel free to kill off adoption of hardware ray tracing if DXR continues to be a dead end (been 3 years since the last update) or keep going in the direction they don't want ...

It's that simple, Microsoft should introduce an API so that Nvidia loses in order for the others to be able to have a chance in a temporary spotlight so that they can all innovate on their own solutions. There's literally nothing stopping Microsoft from making these unilateral moves since they took the initiative to ignore whatever AMD or Intel had to say regarding HW features ...

AMD RDNA3 Specifications Discussion Thread

Meh. I want room sized, British cold fusion reactors first. Bullshit? Likely not. They sold ARM because they no longer need it. Meh. Give me a rdna4 PSP instead. best I can do is a rdna 4 steam deck.

forum.beyond3d.com