Tim Sweeney on Shadow Maps, stencils and unified lighting!

[XxStratoMasterXx]

Well, here's another interesting email from Tim Sweeney which you guys might find interesting! It's regarding stencil shadows v.s. shadow depth buffers, and his views on Unified Lighting and shadowing!

Shadow maps definitely have a longer shelf-life than stencil shadows. Stencil shadows lead to hard edges and scale worse with scene complexity. Many of the Pixar movies use very high-res shadow maps for non-realtime rendering, while nobody uses stencil shadows in that context. So clearly shadow maps have a future.

I think it will be 10+ years before a single unified shadowing algorithm is practical in realtime. Even if you look at non-realtime rendering, where performance isn't as much of a constraint, the artists still choose between a large set of shadow options for individual objects, rather than running everything through a single pipeline.

-Tim

So it seems to me he believes that shadow maps are the way of the future and stencils are going to die.

Also, I think he may have a point with the unified lighting and shadowing. Maybe artists just want to choose from different types of shadowing for different effects, rather than just having the same kind of shadowing everywhere? This explains why Tim wanted to include stencil shadows AND shadow maps in his engine.

So yeah, I hope some interesting discussion may come from this!

 

[Ostsol]

Re: Tim Sweeney on Shadow Maps, stencils and unified lightin

Also, I think he may have a point with the unified lighting and shadowing. Maybe artists just want to choose from different types of shadowing for different effects, rather than just having the same kind of shadowing everywhere? This explains why Tim wanted to include stencil shadows AND shadow maps in his engine.

I think that artists don't care what method is used to produce the shadows so long as the result is similar in appearance across multiple situations. As such, the interface should be completely transparent to artists; they should not need to choose what type of shadowing is used. Instead, the engine should use the method most appropriate for the given situation.

 

[AlNom]

So... perhaps have a global lighting solution, but allow for specific situations? outdoor lighting versus indoor lighting, and even then.. indoors can change a lot for the effect the artist may want to convey.

*shrug*

 

[XxStratoMasterXx]

Well, what about John Carmack's solution...shadow maps globally? Won't that solve the problem quite nicely?

 

[squarewithin]

Well, what about John Carmack's solution...shadow maps globally? Won't that solve the problem quite nicely?

Remind me exactly what you mean by shadow maps globally.

Personally, as an image/sampling person, I'd agree with Tim on this one. With high enough resolution shadow maps, they are indestinguishable from stencil shadows. Plus, there are some decent projection tricks [1] [2] [3] (to name a few) that can project things to minimize the error. Combine this with smoothies (again, my sampling preference) or the like and you pretty much can encompass everything you could possibly expect these days with shadow maps.

Of course, everything gets harder when you start using those pesky area lightsources. If you want anything unified, you have to oversample at every point to get an appoximation of its hemisphere. nvidia has a demo doing this in a branching VS that breaks out at covergence for the shadows based on distance to a texture projector. It runs at 640x480 at 60hz for a single light and a teapot. Try doing this everything to everything and you'll see why we will be waiting 10+ years

 

[Laa-Yosh]

Unified lighting and shadowing are two different things.
UE3 uses unified lighting as far as we know, but 3 different methods for shadowing.

 

[JF_Aidan_Pryde]

I thought UE3 uses precomputed lighting information?

 

[XxStratoMasterXx]

No, Unreal Engine 3 uses pre-computed shadow maps for static lights, but all lighting is computed in real-time.

 

[Diplo]

No, Unreal Engine 3 uses pre-computed shadow maps for static lights, but all lighting is computed in real-time.

How do you pre-compute shadow maps before you calculate lighting? Perhaps I don't understand, but that doesn't seem to make sense to me.

 

[Ostsol]

No, Unreal Engine 3 uses pre-computed shadow maps for static lights, but all lighting is computed in real-time.

How do you pre-compute shadow maps before you calculate lighting? Perhaps I don't understand, but that doesn't seem to make sense to me.

Shadowmaps just determines how a shadow will be cast. "Lighting" refers to the equations in the pixel shader; the diffuse and specular equations, for example. If it's a depth shadowmap, you can precalculate it by saving the depth-buffer. It has nothing to do with "lighting" so can be done entirely separately.

 

[Dave B(TotalVR)]

I dunno if its just HL'2 implimentation, but I havent seen shows being cast on the object casting the shadow and ive seen (which is hl2) plenty of occurances with shadows in the wrong place (mostly in CS:S).

Ive always been the biggest fan of stencils because they are the mathematically correct answer and I think a move advanced stencil buffer is the answer to allow for soft edges.

plus, even though the edges are hard on the shadows in Doom 3, the whole game and especially the lighting looks amazing.

 

[KimB]

Ive always been the biggest fan of stencils because they are the mathematically correct answer and I think a move advanced stencil buffer is the answer to allow for soft edges.

But they're not the only way to produce exact shadows. Rendering the entire boundary of a shadow really is an extremely inefficient way of displaying shadows. This is why shadow mapping is so appealing: the performance statistics involved in generating the maps are much more similar to normal rendering.

But personally, I'd still really like to see a hardware implementation of an irregular z-buffer. Provided it is well-optimized, it could really be a general-purpose shadowing solution that has performance closer to that of shadow maps, but with the exactness of stencil shadowing.

 

[andypski]

Ive always been the biggest fan of stencils because they are the mathematically correct answer and I think a move advanced stencil buffer is the answer to allow for soft edges.

Being pedantic I wouldn't say they are mathematically 'correct' as much as mathematically exact. The hard edge on the shadows certainly isn't mathematically the 'correct' shadowing, unless your light source is an infinitessimal point.

What advance to the stencil buffer did you have in mind to allow for soft shadow edges?

 

[XxStratoMasterXx]

Ive always been the biggest fan of stencils because they are the mathematically correct answer and I think a move advanced stencil buffer is the answer to allow for soft edges.

Being pedantic I wouldn't say they are mathematically 'correct' as much as mathematically exact. The hard edge on the shadows certainly isn't mathematically the 'correct' shadowing, unless your light source is an infinitessimal point.

What advance to the stencil buffer did you have in mind to allow for soft shadow edges?

Well, shadow buffers have hard edges that aren't even straight in their raw form. It's a jagged mess which looks awful. It's the pixel processing which makes it look good.

The question is, could the same filtering and stuff be applied to stencil volumes to allow for soft shadows which are superior to pixar's? Since stencils are mathematically correct?

 

[KimB]

That's a pretty ridiculous question, since when rendering offline, it is no problem to make the shadow buffers of an appropriate size that there is no pixellation. And it is much easier to soften shadow buffer shadows through texture filtering-like operations.

Edit: You can also always touch-up the images afterwards to further eliminate artifacts.

 

[Ostsol]

Using texture filtering to soften shadows is probably the fastest method for dynamic shadows, but pregenerated shadowmaps could also receive much more accurate softening without having to resort to that. The shadowmap itself would simply represent the furthest extents of the shadow and an 8-bit alpha texture could be used to define the "softness" of the edges.

 

[KimB]

Hell, if you're going to do that, you might as well just make projective textures out of 'em. But personally, I feel the more interesting solutions are those that can be applied to realtime calculations.

 

[Ostsol]

Hell, if you're going to do that, you might as well just make projective textures out of 'em.

I was actually refering to that. I tend to think "projected shadowmaps" whenever I read "shadowmaps". *shrugs*

 

[KimB]

Oh, right, duh....brain fart.

 

[MfA]

Plus, there are some decent projection tricks [1] [2] [3] (to name a few) that can project things to minimize the error.

A lot of game developers dont seem to think those are nearly robust enough for the general case ... and are relying on the rather expensive use of multiple shadowmap Z-buffers, sectioned according to depth from view.