Vertex Shaders and Pixel Shaders beyond version 2.0:

[Hyp-X]

I fear there's some serious painting themselfs into a corner going on with the HLSLs. Both Cg and DX9 HLSL is built on the idea of standardising the compiler rather than just making the language a standard and let the compiler be a part of the driver. Making a compiler targetting a standardized instruction set and we'll repeat the whole x86 problem but for GPU's.

I agree.

Now having the final DX9 SDK - HLSL is a pretty big disappointment.

It's has 1 (one) page dedicated in the documentation, and has 0 (zero) example programs.

It looks like its one of the doomed features of DX.

(Reminds me of RT-patches in DX8. DrawTriPatch accepts a user supplied amount of vertices - how many can there be, how they constuct the patch, etc. is not documented. After 1 year of being part of the API, nVidia could safely remove driver support as no developer bothered to figure out how does it work. )

 

[Ilfirin]

HLSL is far from a doomed feature. Expect the docs to be available very soon (tm).

HLSL also has the runtime compiler that CG has, which allows it to optimize for the user's card at run-time. This also means that it has full access to the caps of the card. I believe the only reason why the compiler does not yet support 2.0extended and 3.0 is simply a shortage of time with not sufficent enough of demand. They have already said that they plan on updating the compiler often, and there is no real 2.0extended nor 3.0 hardware out yet so why hold back the entire SDK from release?

The only part that might make it a less-used feature is that it is part of D3DX, not standard D3D. D3DX seems to get very rare use among professional studios (who usually write their own, API independent, libraries).

 

[MDolenc]

HLSL also has the runtime compiler that CG has, which allows it to optimize for the user's card at run-time. This also means that it has full access to the caps of the card.

And how the hell it does that? Does it check the registry?
You can only call this function (and the fxc utility again calls this function and can do a few other things):
HRESULT WINAPI D3DXCompileShader(
LPCTSTR pSrcData,
UINT srcDataLen,
CONST D3DXMACRO* pDefines,
LPD3DXINCLUDE pInclude,
LPCTSTR pFunctionName,
LPCTSTR pTarget,
DWORD Flags,
LPD3DXBUFFER* ppShader,
LPD3DXBUFFER* ppErrorMsgs,
LPD3DXSHADER_CONSTANTTABLE* ppConstantTable
);
You don't even need to have a D3D device created to compile a shader...

 

[Ilfirin]

Atleast I am pretty sure I heard someone from Microsoft say that on the newsgroup a while ago. I may be mistaken (and probably am, my memory gets all mixed up now 'a days :/).

 

[790]

Beyond compiling for the shader target (which D3DX can do at runtime, with a switch statement, duh), what exactly do you think could be done for a specific card? The backend is already heavily optimized for all vendors, and they work closely with Microsoft to get maximum performance. The low-level stuff is then handed off to the driver.

The idea that there's something "more" that could be done, implies that there are multiple assembly-level configurations that are faster for different cards, and this is something which I don't believe is true. Can you demonstrate otherwise?

 

[790]

On the subject of HLSL documentation, there is more to come, but everything you need is already available. The best way to learn is to dissect the sample shaders provided by MS and ATI. ATI also have some good slides over @ devrel. There are a few in the docs (HLSL skinning, HLSL glow), and even more in the Media folder. If you have any specific questions, ask on the NG/ML.

 

[Ilfirin]

All I meant was that it could be compiled towards a shader target (with caps, since that is what defines the shader targets above 2.0standard now), the idea that a HLSL compiler can target a specific card for added performance came from nVidia with 'CG'.

Their reasoning was that some instructions run better on some cards than others, and that for any given problem there are usually a few instructions/macros that can do about the same thing. That is the extent of CG's card-specific 'optimizations'.

 

[RussSchultz]

Beyond compiling for the shader target (which D3DX can do at runtime, with a switch statement, duh), what exactly do you think could be done for a specific card? The backend is already heavily optimized for all vendors, and they work closely with Microsoft to get maximum performance. The low-level stuff is then handed off to the driver.

The idea that there's something "more" that could be done, implies that there are multiple assembly-level configurations that are faster for different cards, and this is something which I don't believe is true. Can you demonstrate otherwise?

I can't say I can demonstrate otherwise, but I can say that there's more than one way to skin a cat in nearly every problem. The difference between optimal code for RISC engines and VLIW/DSP engines is vast, and there's even different performance behaviors between x86 ISA machines;

Having the back end under microsoft's control means that you're (essentially) stuck with whatever back end exists when that version of directX came out, or you must conform to some standard ISA, or come up with some scheme to map that ISA to your ISA.

I think the backend should be part of the installable driver so that radically different approaches could be possible. It wouldn't necessarily put an onus on the IHV because they could just as well go with the standard backend that microsoft provides, but it does open the doors for architectural innovation, rather than 'I can get my shader engine to run at X mhz faster than my competitor'.

 

[demalion]

Hmm....I tend to agree, Russ. But it isn't a step backwards for DX to keep it targetted at a "common" assembly syntax, just a limit on how much of a step forward it is taking.

I think the capability detection, if it can be used to affect compiled output, can achieve similar performance gains, though. If HLSL usage is supposed to become widespread, what's the point of the PS and VS 2.0 "extended" specifications unless it can? This still does concern me as I don't have anything against IHV's being rewarded for having more capable hardware (in the near future that would be the nv30)...I guess we'll have to see how the DX compiler is updated.

 

[790]

In reality we're only going to see gains between different shader targets, something which can already be chosen at runtime. Not the same target on different platforms. Microsoft could have very easily included runtime compilation if they thought otherwise. They already do so for other parts of D3DX, including mesh optimization.

 

[RussSchultz]

I guess, but capability bits do not indicate any sort of optimal useage patterns. The 56k DSP, for does have a divide instruction, but its quicker by an order of magnitude to multiply by the reciprocal.

Though, as you said, its really a moot point if Microsoft keeps the back end up to date and makes updating the back end easy and painless for end users. (I will say they really didn't do us many favors when they ported DRM to our WMA/MP3 processor. Its probably 20%-30% larger than it should be. )

 

[DemoCoder]

You can view DX9 assembly syntax is just an intermediate representation of the HLSL. You can write a sophisticated compiler to do oodles of optimizations on the assembly. The driver itself only needs to handle instruction scheduling, peephole optimizations, etc. Most compilers do their optimizations on IL. I could read DX9 assembly, convert to SSA form, do optimizations, convert back, etc.

It is infact, a step backwards to let Microsoft control the compiler and be the only implementation. There should be a large market for compilers and languages. Only the assembly target syntax and front end language needs standardization, not the IMPLEMENTATION of the compiler.

 

[ERP]

The only real problem I have with MS specifying the DX9 Assembly language, is that by specifying it they push development towards a solution that tends to mirror the specification.

I guess I'd have less of a problem with it if the design were geared towards being an intermediate representation for compiler ooptimisation. Rather than it being some minimal subset of current hardware implementations which is what it currently is.

Ideally there should probably be some sort of support for plugin compilers, that supports managing shaders without reference to any intermediate or explicit machine level representation at all. The DX9 HLSL would then just be one possible compiler that could ship with DX9.

 

[RussSchultz]

Only the assembly target syntax and front end language needs standardization

I whole heartedly disagree with this, for reasons stated above. The front end language needs standardization, but I think that having the target assembly syntax standardized set breeds a common means of getting to the solution.

 

[KimB]

In reality we're only going to see gains between different shader targets, something which can already be chosen at runtime. Not the same target on different platforms. Microsoft could have very easily included runtime compilation if they thought otherwise. They already do so for other parts of D3DX, including mesh optimization.

Which is why runtime compilation is a necessity! That, and some sort of "default" target that compiles to whatever is the most optimal target for current hardware, set by the driver.