Skyrim Shaders

[Alenet]

I worked on Oblivion Reloaded - OBGE v4 rewriting totally a new version of the Ethatron's plugin.
I am working now on modding Skyrim creating the Skyrim Reloaded - SKGE (a first version is already online).
The plugin is able to inject/replace the vanilla game shaders by custom ones but i need to rebuild the vanilla HLSL shaders before modifying them.

I dumped the Skyrim shaders (more then 800) and disassembled them.
Now i need to rebuild the source code (they are DX9 shader model 3).
How can i do that by a tool that create at least a pseudo HLSL code (then i adjust it)? I am not able to find a tool to "convert" the asm to the HLSL (the Oblivion shaders were been decompiled by Ethatron).

I tried to take one of the water shader and "converted" the asm to HLSL manually one line per line replacing each the asm instruction with its relative HLSL istruction, but when i compile it and put it in the game the result is not the same (while if re-assemble the disasmed the result is correct).
I noticed also that if i disassemble the compiled HLSL the disassembled code is different from the original disassembled code. The HLSL seems correct (registers, variables, etc) but not the result.

Any suggestion?

 

[Alenet]

An other thing, i do not understand this:

mul o3.zw, r0.w, c15.xyyz

o3 is an output coord
c15 is a const
Basically it should be OUT.texcoord3.zw = r0.w * c15.xyyz;

But i do not understand why c15 is used in four component while only zw is assigned. Infact the result (if i compile the HLSL and i disassemble it) is mul o3.zw, r0.w, c15.xyxy due to the truncation during compilation.

 

[PixResearch]

I don't think it works how you think... The swizzle on the destination register tells you which channels are being read from the inputs and specifying a single channel implicitly means same thing in 4 channels.

mul o3.zw, r0.w, c15.xyyz

is the same as:
o3.zw = r0.wwww * c15.xyyz

then the zw swizzle picks the 3rd and 4th channels from the input registers, ie:

o3.z = r0.w * c15.y (the 'z' channel of the input registers)
o3.w = r0.w = c15.z (the 'w' channel of the input registers)


edit: here's a link: https://msdn.microsoft.com/en-us/library/windows/desktop/bb219869(v=vs.85).aspx

 

[Ethatron]

You have the Oblivion Shader DB on your github, you should be able to run the shaders through that scripts. The result may not come out correct first-try though. But's a starting point.

 

[Jawed]

Destination write mask doesn't pick channels from result, it specifies channels in destination:

o3.z = r0.w * c15.x
o3.w = r0.w = c15.y

http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter35.html

The other two channels on C15 can be ignored. Depending on the IHV assembler you'll sometimes see these excess channels.

Doesn't Code XL disassemble shaders (from the program as it runs)? I've done disassembly with OpenCL kernels, not sure about graphics...

 

[PixResearch]

Yeah, ignore me - too many languages in one head when tired

 

[Alenet]

@Ethatron: it is a pleasure to hear you. Your work on Oblivion's shaders is impressive. You are great. The compliments are true.
I noticed your scripts but they seems Linux scripts. How can i run them on Windows? Probably by Cygwin?

@Jawed: that is what i think but it seems what PixResearch it is not so wrong. Because when i compile and decompile the line, the result is different and the compilator is the same used in Skyrim. I will investigate on that.
I do not think CodeXL is able to decompile shaders. Probably it can disassemble them, but i do not think it "re-costructs" the HLSL code. Anyway i will look at it.

 

[Jawed]

CodeXL reconstructs OpenCL (so functions have "arg1", "arg2" etc.), which is why I think it's worth a try.

 

[Alenet]

@Ethatron: i run the "convert.sh" via winbash but it uses perl so i have to install it.......i go ahead.

@Jawed: i am trying the Ethatron's scripts then i will look also CodeXL. I saw quickly the site, it seems interesting.

 

[Ethatron]

Yeah, just make yourself a nice cygwin installation, perl and sed are your best friend when trying to make HLSL out of asm. The reorder.pl is actually an out-of-order instruction reorderer based on perl regexp, no need to go full blown LLVM to achieve something sort of similar. Some stages of the script might just work out of the box, other depend on databases of the debug-names of the variables in Oblivion, just try to figure out what's the difference between the two and permute the script.

 

[Alenet]

Damn!!....it works! I try the water shaders and your converters do the work at 99%. I adjusted some little things and the result is exactly the same of the original shader. Great! Your scripts work with a good precision.

 

[Alenet]

I read the shader tandem.
While in Oblivion the water is rendered like in a lake, in Skyrim there are rivers so the water runs along the river. The same shaders are used also in game's lakes (so the water "scrolls").
The water shader uses three normal sampler and the vertex returns the coord to scrolls the normals to have the movement effect.
I would mainly modify some things: the refractions -diffusion is not good-, a depth correction -too dark in vanilla-, and i will implement the way to have unique settings. These should not be a problem.
But, i would adjust the shoreline. The main problem is that the water plane seems to scroll "under" the terrain and to "exit" from the terrain and it does not give an effect like a water "impact" on the shoreline.
What do you suggest to adjust it?

 

[Ethatron]

If you can't get extra stuff into the shader it's basically impossible. Extra stuff being your own constant buffer data, your own textures ad so on.
Water is always rendered after solid geometry. Just get a copy of the depth-buffer before water rendering commences and read it in the water shader, to calculate the depth of it.

 

[Alenet]

I can (like in Oblivion Reloaded - OBGE v4) add to any shader extra registers and pass what i want (also the depth). The method is mainly the same you used in OBGE v3 but i implemented it directly in the Creation Engine without hooking the d3d9.dll. The engine gets and passes the depth if the shader needs it.
Anyway the vanilla Skyrim shaders are better then the vanilla Oblivion ones and the depth sampler is already passed to the shader by the game so i do not need to pass it.
I adjusted the shoreline trasparency with the world z pos. My problem is that i do not know how to make the effect of the water that "hits" the terrain due to the scrolling movement. I would to "stop" the water plane to "cut" into the terrain in the shoreline and give it an effect like the wave that "hits" the terrain.
I do not know a technique for that.

 

[Alenet]

I found a not perfect solution but good. I improved the depth control and reversed the normal scroll direction and re-merged it to have a sort of "wave" along the shoreline.
This is the result:

You can see here the difference with the original water. On the right i do not replace the displacement shader so it is the vanilla water. The original plane seems to "exit" from the terrain (think it while it is scrolling horizontally):

 

[Alenet]

Is there somebody who can help me to render the caustics?

 

[Alenet]

Nevermind. I made them myself.

 

[homerdog]

You know the remaster is coming right? I assume it will include a DX11 renderer which might make some of your work irrelevant.

 

[Alenet]

Yes. I know, but when i started to work on SKGE (a couple of months ago) i did know that Bethesda will exit with a remastered version of Skyrim. It is curious and unusual.
Anyway differences between SKGE and the remaster edition (reading about it) are many.
Skyrim cannot be DX11. It requires to be re-written at all to be DX11. Skyrim is DX9 and the Creation Engine is DX9/DX10 (by now).

 

[PixResearch]

Remastered Skyrim actually could be DX11 in this instance...
I believe the overhaul was part of their internal development of Fallout 4. So it's probably running on the newer Fallout 4 engine and if it is then it will be DX11.