Deferred shading: Crytek's normalized linear depth

[sergi.gonzalez]

Hi,

In my deferred shading implementation, I store the depth as the length of the position in modelview space (range 0 - far).

In the lighting pass, I render a fullscreen quad and recover the world space position as:

float depth = tex2d(depthMap,i.screenPos);
float4 wsPos = eyePos + normalize(i.eyeRay)*depth;

where eyeRay is the interpolated ray from eyePosition to the far frustum corners.

The code is working perfectly but I'd like to remove the normalize instruction.

Recently, I've read in a Crytek's paper (Wenzel) that it is possible to recover the world space position with a single mad instruction storing the normalized linear depth.

In my tests, I've tried to store depth in two different ways

1. depth = mul(modelView,i.position).z * (1.0/farDistance)
2. depth = length (mul(modelView,i.position)) * (1.0/farDistance)

and again, my eyeRay is the interpolated ray from eyePosition to the far frustum corners.

but I have no success with the recovered positions. Any advice?


Thanks in advance,


Sergi

[Mintmaster]

The value you store in the rendertarget should be the distance from the camera plane, not the camera position:

depth = dot(vEye, objectPos - eyePos)

where vEye is the direction the camera is pointing and has length 1.0/zFar (you can make this a separate factor if you want). This is constant across the screen, and of course all the parameters above are in the same space. I think you have the same thing in #1 of your post.

When recovering the position, you need to interpolate the worldspace location of the far plane that you used above. Basically, imagine where the fullscreen quad would be in worldspace if it was located at the far plane. The position of that quad will be your eyeRay.

Then you just do:
wsPos = eyePos + eyeRay * depth

[sergi.gonzalez]

Ok, the code is working now without normalizing the ray in the pixel shader.

My mistake was in the eye ray interpolation. I was using

wsEyeRay = wsFrustumCornerPosition - wsEyePosition

instead of using simply the wsFrustumCornerPosition as ray vector.

Thanks a lot for your help,

Sergi

[sergi.gonzalez]

I was wondering if this method can be adapted to render convex volumes instead of rendering a fullscreen quad in order to improve fillrate (a fullscreen quad with scissor test enabled is much bigger than an accurate convex volume).

I've tried the infinity projection method with the convex mesh vertices (used to extrude shadow volumes at infinity) but it does not work. It seems that the eyeRay is wrong interpolated again.

Any idea?

Sergi

[sebbbi]

I was wondering if this method can be adapted to render convex volumes instead of rendering a fullscreen quad in order to improve fillrate (a fullscreen quad with scissor test enabled is much bigger than an accurate convex volume).

Yes, you can render convex volumes with this technique. Easiest way is to render the volume to the stencil buffer first, by using a single 2-sided stencil pass (use a similar setup than you would in reverse stencil shadows). After this you will have the first stencil buffer bit 1 in all pixels that are inside the volume, and 0 that are outside. This provides you full pixel perfect 3d-clipping (if the screen space pixel is further than the volume backface or nearer than the frontface, the stencil bit will be 0). After the stencil pass, render the fullscreen quad just like you are already doing, but enable the stencil test (equals 0x1). This way the pixel shader is executed only on the pixels inside the volume.

You can also optimize this technique by changing stencil bit masks in order to render several volumes to the buffer one after each other, and then switch the shader and states and run the pixel shader for the volumes. This way you will save considerable amount of shader and state changes (if you need to run the same pixel shader for more than one volume).

[sergi.gonzalez]

sebbbi: Thanks for the reply, your method should work.

However, besides of lighting, I'm going to use the depth buffer to render soft particles and with your method I'll have to render all particles twice (fillrate intensive).

My old depth buffer (length(positionViewSpace.xyz) ) was more useful than I thought.

Does anyone have another solution (crytek's boys :lol:)?

[Mintmaster]

Ok, the code is working now without normalizing the ray in the pixel shader.

My mistake was in the eye ray interpolation. I was using

wsEyeRay = wsFrustumCornerPosition - wsEyePosition

instead of using simply the wsFrustumCornerPosition as ray vector.

Thanks a lot for your help,

SergiThat's odd. I would expect the first one to be correct.

I've tried the infinity projection method with the convex mesh vertices (used to extrude shadow volumes at infinity) but it does not work. It seems that the eyeRay is wrong interpolated again.Rather than projecting to infinity, make sure you project to a plane perpendicular to the camera direction, i.e. a fixed cameraspace Z value.

[sergi.gonzalez]

That's odd. I would expect the first one to be correct.

Rather than projecting to infinity, make sure you project to a plane perpendicular to the camera direction, i.e. a fixed cameraspace Z value.

You were right with the ray interpolation. I found an error in my shaders permutations code. At last, fullscreen passes without normalize instruction.

In regards to the convex mesh deferred pass (useful to render light volumes, particles, etc), finally I found a smart method.

For the people interested, what I've done is to interpolate the positionInWorldSpace and later (in the pixel shader), compute the distance vector from this position to the far plane, throwing a ray from the interpolated position to the far plane. It seems expensive, but it's cheap. 2 dots, 1 div and 1 mul to compute the eyeRay.

Here I paste the code,




// origin: ray origin
// direction: ray direction
float3 getDistanceVectorToPlane(float4 origin, float3 direction, float4 plane)
{
float denum = dot(plane.xyz,direction.xyz);
float num = dot(plane,origin);
float t = -num/denum;

return direction.xyz*t;
}

...

float3 ieyeRay = getDistanceVectorToPlane(eye_point, i.positionWorldSpace, farPlane);



where positionWorldSpace is computed (in the vertex shader) as

positionWorldSpace = model*vertex

and model is computed (in the c++ code) as

model = objectModelMatrix * TranslationMatrix(-CameraEyePoint)

If anyone finds a cheaper method to compute the eyeRay in the convex rendering passes, please let us know.

Cheers,

Sergi

[Xmas]

I may be missing something here, but why not use view space?

In the first pass, simply store view space Z in the depth map. In the lighting pass, interpolate view space position of the light volume, and calculate:
float depth = tex2d(depthMap, screenPos);
float3 vsPos = float3(interpVsPos.xy * (depth / interpVsPos.z), depth)
You could of course also save one interpolator and calculate the second line based on screenPos (without the division), but it may be less efficient.

[sergi.gonzalez]

I may be missing something here, but why not use view space?

The phong/lambert lighting equations can be resolved in view space but we need to recover world space position (wsPos) because it's mandatory in other algorithms. For example, in shadow mapping, we need the wsPos in order to project the point in light space.

PD: BTW, in my previous post, I forgot to say that

float num = dot(plane,origin)

is a constant scalar, and it does not need to be computed per pixel (saving 1 dot)

[Xmas]

The phong/lambert lighting equations can be resolved in view space but we need to recover world space position (wsPos) because it's mandatory in other algorithms. For example, in shadow mapping, we need the wsPos in order to project the point in light space.
You can just as easily transform the view space position to light space.

[sergi.gonzalez]

You can just as easily transform the view space position to light space.

Yes, you are right. If I'm not wrong, it would be

positionLightSpace = (inverseViewMatrix * lightViewProjectionMatrix) * vsPos

instead of

positionLightSpace = lightViewProjectionMatrix* wsPos

However, the world space position is also useful for i.e cube reflection mapping. If you have static cubemaps, the lookups must be rays in world space. If you have position in world space, the lookup can be done with wsPosition - eyePos (this is just the distance vector that I already computed).

Is there any advantatge (when using vsPos) that I am forgetting?

Cheers,

Sergi

[Xmas]

Yes, you are right. If I'm not wrong, it would be

positionLightSpace = (inverseViewMatrix * lightViewProjectionMatrix) * vsPos

instead of

positionLightSpace = lightViewProjectionMatrix* wsPos
If you're using M * v, it should be (WorldToLightClipMatrix * ViewToWorldMatrix) * vsPos.

However, the world space position is also useful for i.e cube reflection mapping. If you have static cubemaps, the lookups must be rays in world space. If you have position in world space, the lookup can be done with wsPosition - eyePos (this is just the distance vector that I already computed).
If you have world space cube maps then using world space eye direction and normals to calculate the reflection vector might make sense. It depends on what else you need to calculate.

Is there any advantatge (when using vsPos) that I am forgetting?
The advantage of view space is that the viewer is at the origin looking along the Z axis, which simplifies most calculations involving the eye position or direction.

[sebbbi]

If you want to render meshes directly to the buffer without extra stencil pass, I suggest doing it like this.

- Implement basic projected texturing pixel and vertex shaders first. When rendering volumes, you need the screen space texture coordinates in all g-buffer reads. The only specific note to implement this is to do the perspective divide in the pixel shader, because triangle edge linear interpolation really messes it up. Test it with passing through your g-buffers first (you should get a pixel perfect image with no wobbling artifacts).

- After this you have x,y texture coordinate pair in [0,1] range in the pixel shader. You use this value to access all g-buffers (depth texture in your particle example).

You do not need anything more than this in your particles. You just compare the depth texture value with your pixel depth. I use world space z in my depth texture, and it's simple to pass the world space z from your particle vertex shader (and other shaders), as world space z is linear (it intepolates nicely).

For lighting (and anything that needs the pixel 3d-position), you need additionally:

- Pass the four camera view cone edge vectors as constants to the shader, and calculate the pixel position using the perspective divided texture coordinates.

float3 upper = lerp(viewVecUpperLeft, viewVecUpperRight, texcoord.x);
float3 lower = lerp(viewVecLowerLeft, viewVecLowerRight, texcoord.x);
float3 viewVector = lerp (upper, lower, texcoord.y);
float3 pixelPosition = pixelDepth * viewVector;

Now you have the position in world space. Actually you have position-cameraPosition, but that is the preferable way to do lighting calculations in most deferred shaders. If you use world space coordinates directly, you'll end up with increasing floating point inaccuracy artifacts when you move further from the world origin.


This method can also be optimized like the stencil version with some z-buffering tricks.

When you are rendering your volume, determine the range from the volume to your camera. For simple spheres (point lights) is is trivial. And:

A) If the volume is near camera render only it's backfaces and select z-comparison that rejects all pixels with depth less than the z-buffer depth. Now your volume only affects pixels that are inside it, or more near (in screenspace) than it. You get almost 100% of the efficiency of the stencil method with no stencil reads/writes and multipassing needed.

B) Otherwise (if the volume is further away from camera) render only it's frontfaces and select z-comparison that rejects all pixels with depth more than the z-buffer depth. Now your volume only affects pixels that are inside it, or more far (in screenspace) than it. This way all geometry (pixels) in front of the volume are free (and nicely block away the volume area). If the volume is behind a wall for example, it's completely z-culled.

Adjust the swap distance from these 2 modes so that you choose the mode B at a range where you have approximate more pixels in the front of the volume than in the back. Also note that the mode A is needed in all cases when the volume can intersect the camera: distance(camPos - volumePos) < volumeRadius + nearPlane. But this is not a problem in real word situations, as you want to switch to mode A much earlier (usually a good distance is half of a room length in your game).

[MJP]

- Pass the four camera view cone edge vectors as constants to the shader, and calculate the pixel position using the perspective divided texture coordinates.

float3 upper = lerp(viewVecUpperLeft, viewVecUpperRight, texcoord.x);
float3 lower = lerp(viewVecLowerLeft, viewVecLowerRight, texcoord.x);
float3 viewVector = lerp (upper, lower, texcoord.y);
float3 pixelPosition = pixelDepth * viewVector;



This isn't necessary, at least if you only want to recover eye-space position rather than world-space. All you need is the position of the vertex in eye space, and the distance to the view frustum's far clip plane (or whatever else you've used to normalize your linear eye-space depth):

Vertex Shader:

float4 eyePosition = mul(IN.position, worldViewMatrix);
OUT.screenDirection = eyePosition.xyz * (farZ/eyePosition.z);

Fragment Shader:

float4 eyePosition = float4(tex2D(depthTexture, texPos).x * IN.screenDirection.xyz, 1.0f);

[Graham]

B) Otherwise (if the volume is further away from camera) render only it's frontfaces and select z-comparison that rejects all pixels with depth more than the z-buffer depth. Now your volume only affects pixels that are inside it, or more far (in screenspace) than it. This way all geometry (pixels) in front of the volume are free (and nicely block away the volume area). If the volume is behind a wall for example, it's completely z-culled.


For light volumes, in this case I usually render both a front and back face culled sphere/whatever to the stencil buffer, with z-pass for both. Use stencil bit invert (only needing 1 bit stencil). Then draw the light where the bit is set, (where only one has rendererd) - and reset the bit while you do it.
This covers both pixels too close and too far away from the light point to be lit.
Of course inside the sphere the bit check needs to be swapped.

This is just off my head, it's been a while... So it may be wrong somewhere.

[sebbbi]

For light volumes, in this case I usually render both a front and back face culled sphere/whatever to the stencil buffer, with z-pass for both. Use stencil bit invert (only needing 1 bit stencil). Then draw the light where the bit is set, (where only one has rendererd) - and reset the bit while you do it.
This covers both pixels too close and too far away from the light point to be lit.
Of course inside the sphere the bit check needs to be swapped.

This is just off my head, it's been a while... So it may be wrong somewhere.

Actually I was describing this technique in my previous post in this same tread. (I used a different stencil operation setup to make it work also on cases where the light volume intersects the camera front plane)

The one pass technique without stencil is just an alternative way to optimize it, if you don't want to use stencil at all.

[sebbbi]

This isn't necessary, at least if you only want to recover eye-space position rather than world-space. All you need is the position of the vertex in eye space, and the distance to the view frustum's far clip plane (or whatever else you've used to normalize your linear eye-space depth)

That's true. I was just quickly writing stuff without thinking much about the most optimal implementation. Must have been a late night at work :). Actually if you want world-space coordinates in your pixel shader, just multiply the vertex position with world matrix in the vertex shader and pass that directly to the pixel shader. To calculate world space camera direction vector in pixel shader, you set camera world space position to a constant and do:

float3 cameraDir = (vsInput.pixelPosition - cameraPosition) / vsInput.zDepth;
float4 gBufferWorldSpacePosition = tex2D(depthTexture, texCoordinate).x * cameraDir;

[Wysicon]

The value you store in the rendertarget should be the distance from the camera plane, not the camera position:

depth = dot(vEye, objectPos - eyePos)

where vEye is the direction the camera is pointing and has length 1.0/zFar (you can make this a separate factor if you want). This is constant across the screen, and of course all the parameters above are in the same space. I think you have the same thing in #1 of your post.

When recovering the position, you need to interpolate the worldspace location of the far plane that you used above. Basically, imagine where the fullscreen quad would be in worldspace if it was located at the far plane. The position of that quad will be your eyeRay.

Then you just do:
wsPos = eyePos + eyeRay * depth

I dont understand how this would work. I'm assuming eyePos is the camera position in world space and eyeRay is the interpolated ray to the frustum corners in world space, right? Now as you move your camera around, all the parameters change but they wont change "linearly" with respect to each other!

For example, let's image the camera is initially at (0, 0, 0) with a far clip distance of 2 units and the object is at (0, 0, 1) (let's assume a simple point). Now eyeRay at the center would be (0, 0, 2) in world space.

When you are calculating the linear eye-space depth, it would be 1 / 2 = 0.5 and during reconstruction, the world space position becomes:

wsPos = (0, 0, 0) + (0, 0, 2) * 0.5 = (0, 0, 1). All is fine and rosy so far...

Now say the camera is repositioned to (0, 0, 0.5). So the eye ray at the center now becomes (0, 0, 2.5) and the linearized depth would become 0.5 / 2 = 0.25, right (since your object is now relatively 0.5 units from the camera)? Now following the previous calculations, the world space position becomes:

wsPos = (0, 0, 0.5) + (0, 0, 2.5) * 0.25 = (0, 0, 1.125)!

That's not correct! I get similar results when I use the above method in my deferred rendering app, I must be missing something here! Are the spaces I'm considering correct, or are eyePos and eyeRay not in world space?

[Wysicon]

To get the correct world space position, the equation needs to be modified to the following:

wsPos = eyePos + (eyeRay - eyePos) * depth

(Of course, eyeRay - eyePos can be computed in the vertex shader and passed down to the pixel shader).