Ambient shadows on nVidia - do nVidia cards need an extra render pass?

[daveb]

I have shadows and reflections code working on ATI cards and now need to extend to support nVidia cards also. But nVidia cards seem to not support ARB_shadow_ambient, and the sample code I have seen uses an extra render pass to get around this. I read somewhere that this is not a problem because it can be done via ARB_texture_env_combine. Can someone show me or point me at an example of this somewhere? Not obvious what to do...

The sample code in OpenGL SuperBible uses an extra render pass when the ambient shadow extension is not available (listing 18.2). When it *is* available (as on ATI cards) then just one line of code is added when setting up the texture state, see the if()... below:

// Set up some texture state that never changes
glGenTextures(1, &shadowTextureID);
glBindTexture(GL_TEXTURE_2D, ShadowTextureID);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP);
glTexParameteri(GL_TEXTURE_2D, GL_DEPTH_TEXTURE_MODE, GL_INTENSITY);
if (ambientShadowAvailable)
	glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_FAIL_VALUE_ARB, 0.5f);
glTexGeni(GL_S, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR);
glTexGeni(GL_T, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR);
glTexGeni(GL_R, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR);
glTexGeni(GL_Q, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR);
RegenerateShadowMap();

Elsewhere in the sample, there is the code

if (!ambientShadowAvailable) {
   ...
   // Because no support for an "ambient" shadow compare fail value, 
   // we'll have to draw an ambient pass first
   ...
   DrawModels();
}

ambientShadowAvailable is set by testing GL_ARB_shadow_ambient and this is true on ATI cards but not on nVidia cards it seems. Surely it's not necessary to do an extra rendering pass on nVidia cards? I saw a fleeting reference to the use of ARB_texture_env_combine in a post somewhere, and another to ARB_fragment_program_shadow but I can't figure out how to write a simple replacement for the one liner above!

So my question is, can one do something with other GL extensions to replace the COMPARE_FAIL_VALUE line above and still avoid the need for that extra rendering pass in this sample code?

 

[Bob]

You should just be able to use glTexEnv() on a second texture stage which just multiplies the incoming color by 0.5f.

Something like:

glActiveTexture(GL_TEXTURE1);
glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_COMBINE_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_RGB_ARB,   GL_MODULATE);
glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_ALPHA_ARB, GL_MODULATE);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_RGB_ARB,   GL_PREVIOUS_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_ALPHA_ARB, GL_PREVIOUS_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE1_RGB_ARB,   GL_CONSTANT_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE1_ALPHA_ARB, GL_CONSTANT_ARB);
glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_COLOR, 0.5f);

 

[daveb]

I gave this a try (I hope I followed the suggestion accurately) but the shadows are all black still, just as they would be with ARB_shadow by itself (failing depth comparisons give black). From what I can gather the ARB_shadow_ambient behaviour is to make the shadowed areas only dim by the value of GL_TEXTURE_COMPARE_FAIL_VALUE_ARB (eg 0.5) so you see red surfaces at low ambient as dark red, not black. Does this code really replace the functionality of ARB_shadow_ambient ?

 

[Bob]

My bad. I assumed you wanted the non-shadowed parts to be at half intensity, instead of the shadowed parts.

If you don't care about saturation, you can use this:

const GLfloat ambiant = 0.5f;
const GLfloat ambiant_vec[]  = {ambiant, ambiant, ambiant, ambiant};

glActiveTexture(GL_TEXTURE1);
glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_COMBINE_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_RGB_ARB,   GL_ADD);
glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_ALPHA_ARB, GL_ADD);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_RGB_ARB,   GL_PREVIOUS_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_ALPHA_ARB, GL_PREVIOUS_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE1_RGB_ARB,   GL_CONSTANT_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE1_ALPHA_ARB, GL_CONSTANT_ARB);
glTexEnvfv(GL_TEXTURE_ENV, GL_TEXTURE_ENV_COLOR, ambiant_vec);

If you change the ambiant value often, you'll want to put it in the primary color instead of a constant, to avoid reprogramming the combiners.

If you do need to avoid saturation (for example, if you're rendering to a floating point buffer) then you'll need more complicated code.

 

[daveb]

Still cannot make the shadowed areas change from black. I may be placing the code in the wrong place or not enabling the extra texture unit I thought, so I tried putting in some extra enables like this...

glActiveTextureARB(GL_TEXTURE3_ARB);
glEnable(GL_ALPHA_TEST);
glEnable(GL_TEXTURE_2D);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_MODE, GL_COMPARE_R_TO_TEXTURE);
glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_COMBINE_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_RGB_ARB,   GL_ADD);
glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_ALPHA_ARB, GL_ADD);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_RGB_ARB,   GL_PREVIOUS_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_ALPHA_ARB, GL_PREVIOUS_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE1_RGB_ARB,   GL_CONSTANT_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE1_ALPHA_ARB, GL_CONSTANT_ARB);

GLfloat ambient = 0.5f;
GLfloat ambientvec[4] = {ambient, ambient, ambient, ambient};
glTexEnvfv(GL_TEXTURE_ENV, GL_TEXTURE_ENV_COLOR, ambientvec);

then disable when I disable the shadow map's texture unit.

This chunk of code is sitting just after the point where I set up the shadow map which also does glEnable(GL_TEXTURE_GEN_S) T etc steps ... do I need to do all these steps for the extra texture unit too?

Also am I correct in assuming I need to allocate an extra texture unit for this step for each of the shadow maps? In the code when there is more than one light I assign a texture unit to each light and generate multiple shadow maps. On the ATI there are 8 texture units, and that means I can easily support 3 or 4 lights at once with different shadows happening. With the nVidia I think there are only 4 texture units and I need at least one for other purposes (blending the textures onto materials) leaving 3. If I need two per shadow map, that really means I cannot even get a second light happening. Well at this stage I'm having trouble getting the code to do one light - to start with that would be good, so this is a separate issue.

 

[Bob]

do I need to do all these steps for the extra texture unit too?

No, you don't.

You want to place that block of code (minus the extra enables you added) immediately after the texture stage that does the shadow mapping lookup.

It's not clear to me why it's not working for you. Does it affect the output at all? Are you really already using texture stages 0-2?

 

[daveb]

OK, after looking at this for some time I now realize that the glActiveTexture(GL_TEXTURE1) in your original code snippet refers to continuing use of the texture unit already enabled and in use for the GL_COMPARE_R_TO_TEXTURE step. SORRY!!! this is why I was so confused and also explains why I was asking questions about using up the finite number of texture units available and why I was trying to enable the extra unit. Now that the penny has dropped, I have gone back and redone both your original suggestion and the second one. Still not right but dramatically different to what was happening (basically nothing!) as one might expect with hindsight.

So here are the four situations...

First, here's the ATI image using ARB_shadow_ambient. Notice the shadowed area is not jet black but paler. This is more obvious when you rotate the scene and look at the shadowed green surfaces etc. Very simple scene here just to illustrate.

= = =
Second, if we remove the ARB_shadow_ambient step and do nothing to replace it, here is the result. All the shadows are jet black.

= = =
Third... here is the effect of the first suggestion made, ie this code

glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_COMBINE_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_RGB_ARB,   GL_MODULATE);
glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_ALPHA_ARB, GL_MODULATE);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_RGB_ARB,   GL_PREVIOUS_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_ALPHA_ARB, GL_PREVIOUS_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE1_RGB_ARB,   GL_CONSTANT_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE1_ALPHA_ARB, GL_CONSTANT_ARB);
glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_COLOR, 0.5f);

= = =
And finally here is the effect of the later suggestion, ie this code

const GLfloat ambiant = 0.5f;
const GLfloat ambiant_vec[]  = {ambiant, ambiant, ambiant, ambiant};
glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_COMBINE_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_RGB_ARB,   GL_ADD);
glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_ALPHA_ARB, GL_ADD);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_RGB_ARB,   GL_PREVIOUS_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_ALPHA_ARB, GL_PREVIOUS_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE1_RGB_ARB,   GL_CONSTANT_ARB);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE1_ALPHA_ARB, GL_CONSTANT_ARB);
glTexEnvfv(GL_TEXTURE_ENV, GL_TEXTURE_ENV_COLOR, ambiant_vec);

It's all making a lot more sense than it did at the start for me, thanks heaps, but not obvious what the correct transformation really is to simulate the ARB_shadow_ambient, or even if this can in principle work. The thing I am noticing is that (ignoring the top foreground surface which is a reflection render step) both attempts at the GL_COMBINE_ARB fail to make the shadowed and nonshadowed areas different. Look at the vertical red face in the first two images - partly in shadow - then look at it in the later images and there is no different between these two regions. Ignoring the fact that the colours themselves are wrong (perhaps just a wrong scaling or whatever), the serious concern is that the distinction between shadowed and non shadowed regions is getting lost. Perhaps this is something elsewhere in the pipeline - but thought I'd post at this point as a sanity check to get some comment back and see whether this is on the right track now. Thanks!

 

[KimB]

According to the spec, ARB_shadow_ambient is a setting whereby if a pixel fails the shadow test, it will be assigned a pre-defined color, instead of set to zero.

So, here's what you want the algorithm to do:
1. Draw the scene with ambient lighting only.
2. Draw each light individually with its own shadow test.

With ARB_shadow_ambient, you can wrap the ambient lighting into the calculations for the first light (the color would be ambient + light for pixels that pass the test, just ambient for pixels that fail).

So, you need to either do another rendering pass and use blending to combine the results for the ambient light with the other lights, or you need to use a shader that systematically goes through each light in the scene, starting with the ambient light.

 

[daveb]

The sample code in OpenGL SuperBible uses an extra render pass when the ambient shadow extension is not available (see code extracts+refs at start of this post). So I'm comfortable with that as an option. However it does require that extra pass. Now at the moment, when there are 7 lights, on the ATI there is still only one render pass per frame as you zoom and pan around the scene. That's because the ATI card has 8 texture units and after using one for the materials blending there are 7 left for shadow maps. On the nVidia it seems there are 4 texture units so can do 3 lights without another geometry pass per frame, still pretty handy. [There is an initial setup pass per light to calculate the shadow map, but when you are zooming and panning, the scene/lighting geometry isn't changing and the texture units do all the work effectively in parallel - just one geom pass per frame].

So coming back to the original question... is it possible to do one pass per frame on the nVidia where there is no ARB_shadow_ambient ? I haven't used shaders yet (have a great looking Orange book by Rost on OpenGL Shading Language and I use it as a pillow sometimes hoping the info will seep into my brain during sleeptime) so I am a bit unsure about the last response... "you need to use a shader that systematically goes through each light in the scene, starting with the ambient light". Actually I have skimmed through the Rost book but haven't taken the leap into trying samples for real. I'll give it a go if some guru out there claims that this will solve the problem (ie no extra geom pass per frame), but would appreciate some initial advice or pointers. Or is there still a solution somehow using GL_COMBINE_ARB which at last I have "working" but obviously not with the right algorithm?

 

[ERP]

You'd have to write a shader to do it in one pass, but it is possible.

 

[KimB]

Or is there still a solution somehow using GL_COMBINE_ARB which at last I have "working" but obviously not with the right algorithm?

According to the ARB_shadow_ambient extension specs, they don't claim that you need another pass, just another texture stage. But I don't really understand the fixed-function pipeline that much, so I don't know for sure. If you actually want to do 3D graphics programming, though, I'd highly recommend you bite the bullet and go shaders. Since you're doing this in OpenGL, definitely use either Cg or GLSL.

If you have nVidia hardware, I'd make use of Cg, as it's easier to get working on ATI hardware (since the nVidia and ATI GLSL compilers are different). If you're developing on ATI hardware, using GLSL is safer, as most programs that compile on ATI's GLSL should also compile on nVidia's GLSL.

 

[daveb]

OK, looks like I have to jump in the deep end. I'm gathering from responses that using GL_COMBINE_ARB alone will not be adequate to replace the functionality of ARB_shadow_ambient with no extra geometry pass. But shaders will do it, right?

So the next question is, where does the shader step click into place? Is it already too late if the shader tries to pick up the result of the GL_COMPARE_R_TO_TEXTURE step? That is, once the ARB_shadow logic has done the compare, and in the absense of ARB_shadow_ambient support just drops black pixels into the shadowed region, is it already too late to recover the correct pixel data and simulate an ARB_shadow_ambient step?

In other words, is the idea to replace the [ARB_shadow + ARB_shadow_ambient] steps with a single shader, or does the shader only need to replace the last step, the missing ARB_shadow_ambient step? Also, I'm sure there are lots of samples one can download but can anyone point me to a relevant one to help a shader newbie?

PS On the question of tools, I have ATI hardware on my dev machine, have been testing this stuff by simply not using the existing ARB_shadow_ambient and can run the exe on an nVidia based machine. So I guess I should use ATI's GLSL from other comment...

 

[KimB]

I'm pretty sure that you have to write the shader instead of the entire pixel rendering pipeline.

That is, from the vertex shader you calculate the screen-space positions of the vertices, as well as any values that you want to interpolate across triangles, which would include texture coordinates and colors for vertex lighting. You may be able to do this with the fixed-function pipeline, unfortunately I'm not certain.

Once you have done this, you just need to understand how these interpolated values are created as inputs to the pixel shader. Once you understand the input semantics, the pixel shader mostly becomes just a C function with a single color as the output. There should be demos available that show you how to perform all of the operations you want to perform within the shader. Check your favorite IHV's developer website.