Doom III Referesher

[nbohr1more]

I have been enjoying "The Dark Mod" quite a bit recently but the performance for certain maps, especially "Return to the City" is pretty taxing on my old CPU (and even newer CPU's from what I gather)... I recall that Doom III's major pitfall with regard to shadowing and geometry was that a large portion of the Stencil Shadow algorithm in done on the CPU...

1) Is that correct, do Stencil Shadow calls still need the CPU for a significant portion of their work?

2) Does "Ultra-Shadow" move that work to the GPU?

3) If "2" is "yes", could ATI implement an Ultra-Shadow equivalent via SM 3.0 (or higher)?

4) The last I heard, Ultra-Shadow was not really functional in Doom III except in internal testing (no external Game code uses it???)?

5) If there is no way to avoid the "CPU tax" for Stencil Shadowing, would AMD's Bulldozer CPU approach have the opportunity to to improve this problem by passing the stencil calculation to the integrated GPU anyway or would the extra FLOPS go to waste as the Stencil algorithm doesn't use them (only CPU speed and efficiency help?)???

6) Does the whole Doom III renderer have some significant CPU bounding due to it's dynamic nature...?

7) Is all the CPU dependence concern a red-herring as the real Polygon limits are due to the number of dynamic MRT's???

 

[CarstenS]

Stencil Shadows use bounding volumes, requiring the generation of triangles, which classically is CPU work. With DX10 however, stencil shadows can essentially be done (almost?) entirely on the GPU. IIRC there was an SDK sample illustrating this.

 

[nbohr1more]

http://http.developer.nvidia.com/GPUGems3/gpugems3_ch11.html

Like that???

I guess for Doom III this would require an OpenGL equivalent to the Geometry Shader (OpenGL 4.0???)

 

[nbohr1more]

yep... OpenGL 3.2 would work as well... So maybe when Doom III goes open-source (if ever...) DX10 and up will not be as Geometry bound... (...well I hear the Geometry shader is still slow for DX11 hardware so maybe this won't help much till DX11v2 or DX12 class hardware???)

 

[MDolenc]

1.) Depends on exact implementation of stencil shadows. If 3d engine is trying to find the silhouette of an object and extrude triangles for the shadow from that then this has to be done on CPU (or geometry shaders in DX10 world).
2.) No, nothing just moves from CPU to GPU.
3.) & 4.) The most significant part of Ultra Shadow is the ability of the GPU to draw pixels without color value (only z/stencil) much faster then pixels with color. This has been in GPUs since GeForce FX and Radeon HD 2900. This part is functional as soon as you turn of color writes (eg when rendering stencil shadow volumes). The second part is depth bounds optimization which has to be actually coded in the game (basically telling the GPU "ok I'll render only from zmin to zmax depth"). Weather this second part was in D3 or not I'm not sure.
5.) Bulldozer is just a CPU, it won't have an integrated GPU. If you mean fusion then yes, you'd profit a bit becouse you wouldn't have to transport mesh to GPU over PCI-Express. But as said you can move stencil shadowing entirely on a GPU now and preaty much everyone has moved away from stencil shadows anyway.
6.) No, why?
7.) What?

 

[nbohr1more]

Yeah... for question 5) I was presuming that part of the Bulldozer strategy is to off-load anything that needs FLOPS to a GPGPU (or integrated GPU as with Fusion)... I guess it would be a waiting game of whether:

A) Id adds GS support to Id Tech 4 for shadows

B) Open-Source Community adds GS to Id Tech 4

C) Bulldozer's compiler team lends Id Tech 4 a hand and off-loads Stencil Shadows calcs to the Fusion GPU (or other available "free" GPGPU cycles...)

D) The Dark Mod moves to the Prey version of Id Tech 4 for better Multi-Core support (not sure how much better Stencil Shadows render on this...)

E) CPU clocks move up a couple of Ghz again

 

[Novum]

Id Tech 4 doesn't use stencil shadows anymore (nor does any other modern engine), so there is no need for GS based stencil volume generation.

 

[nbohr1more]

Was that as of QW:ET, Wolfenstein (2009) or before?

 

[nbohr1more]

Wolfenstein 2009's shadows don't appear to be fully dynamic??? I have yet to see an alternate dynamic shadow method in my Id tech 4 searches... (Looks like Id reserved GPU-centric shadows for Id Tech 5 as far as I can tell???)

 

[Novum]

Was that as of QW:ET, Wolfenstein (2009) or before?

QW:ET uses stencil shadow. Wolfenstein uses prebaked lighting with perhaps additional dynamic shadow mapping - but certainly no stencil shadows.

 

[nbohr1more]

So far that says to me that no shipping Id Tech 4 title uses any dynamic shadow method other than stencil shadows. I would say possibly Brink may be a different story but it appears to have mostly static lighting as well...

After a little research I see that John Carmack was about to implement "Shadow Buffers" but didn't feel enough hardware was ready for this approach...

I am confused, though, as to why this path was never added at a later date. If I read my literature correctly, as far back as the Geforce 3 we've had the hardware for this technique...

Are Shadow Buffers still too expensive?
Are the IHV drivers for this feature still too buggy?
Is Carmack waiting for ALL integrated GPU's to have enough oomph and trying to keep the feature requirement as ancient as possible to accommodate late-comers?

I would buy the idea that there are complex engine dependencies preventing this path being added if you couldn't disable shadows. But why let your engine be so bound by CPU? Id could've easily kept Id Tech 4 in the game license-wise by removing the geometry restrictions caused by Stencil Shadows. It would easily be competitive with UE3 on most fronts and would've had a head-start. What is the deal? Is Geforce 2/Geforce 4MX support so important?

 

[MDolenc]

Why are you assuming that same limitations from 2004 still exist today in 2010? Shadow buffers or shadow mapping (what ever you want to call it or in what ever form you like) is the dominant technique for shadowing today. I also think you're paying to much in the whole idea about stencil shadows being the cause for all that CPU boundnes. And I don't remember exactly what the benchmarks were for Doom 3 but what exactly are you basing your "CPU bound" claims anyway?

 

[nbohr1more]

I thought it was common knowledge that the low-poly models in Doom III are due to the CPU-heavy stencil shadows...???

From my above correspondence it appears that no "Real-time Dynamic" shadow method is currently in any shipping Id Tech 4 game.

If Shadow Buffers is the same thing as Shadow Mapping then it is not "Dynamic" (unless it has some pseudo-dynamic capability like Humus's volumetric texture mapping demo..???).

For "The Dark Mod" dynamic lighting is a key ingredient so I don't think they could switch to Wolfenstein 2009's static lighting method...

So I am really trying to clarify if there is another "Real-time Dynamic" shadow method native to Id Tech 4 that doesn't require the CPU to generate triangles. If so, both the level and model quality will not be bound by the CPU's geometry though-put and either bigger levels or more detail will be possible (or both).

I don't really care about Soft-Shadows (the current obsession at Doom3world) I just want lots of triangles that don't need a 10ghz single-core to render (the CPU Doom 3 was planned for)....

 

[nbohr1more]

Ok, I see that Shadow Buffers can be used for dynamic light-sources so the only remaining question is if they can be called in Doom III's iteration of Id Tech 4 and what caveats...??

 

[nbohr1more]

So I did some further digging into JC's statements about Shadow Buffers vs Stencil Shadows and found the real reason for the decision. It seems that most Video Cards did not have enough RAM to store decent quality buffered shadows. So, essentially, Doom III would've had the ugly jagged-edged shadows (we've come to deride) all over the place. Even now this is an expensive proposition but with 1GB and up we're mostly out of the woods. With Id Tech 4 going open-source maybe the community will be able to graft in shadows that don't use up valuable CPU cycles or too-much GDDR.

 

[Davros]

Forgive the noobishness of this question
but instead of generating shadows, why not start with a black level and generate the light, if that makes sense ?

 

[MDolenc]

That's esentially what deffered renderers are all about. However you still need to know where not to apply light (e.g. in shadows).

 

[Rodéric]

Forgive the noobishness of this question
but instead of generating shadows, why not start with a black level and generate the light, if that makes sense ?

What's the difference ?
You still need to know what's visible from the light point of view, call that light map or shadow map, but historically light map was precomputed lighting, so shadow map/buffer was used for the dynamic rendering from light PoV.
Some make distinction between shadow map (planar projection, no depth), and shadow buffer (depth buffer)...

Anyway, in the end you need to know where to apply lighting, which information comes from what we commonly call "shadow map".

 

[nbohr1more]

...so just for the speculative value. Would the Geometry Shader based Stencil Shadow approach eat as much GDDR for it's Render Target (if it has one???) as a high quality Shadow Buffer?

Does anyone have an opinion about which shadow technique (other than baking) is most friendly for high-poly models and dynamic lighting?

(yeah I know, loaded question).

 

[MDolenc]

Stencil shadows don't eat additional memory. You can also render them completly on GPU even without geometry shaders (cast backfacing geometry to infinity with w = 0).
And why do you think one method is friendlier to high-poly models then the other? In every case, you'll have to redraw the model for every light. After that performance is independant of original model complexity.