Shadow techniques in the new generation

[idsn6]

Stencil shadows are dead..for so many different reasons that it would be better to open a new thread just to discuss that.

Actually there's one situation where I have been recently thinking stencil could still be a win, but it's pretty game specific, and it requires mixing of both approaches. Maybe we should really open a new thread for that though.

Seems like as good a time as any to get some updated perspectives.

Dynamic shadows are almost a given in titles these days. At the same time, though, shadows are one of the most obvious graphical complaints if they're not done perfectly. The debate between shadow volumes and shadow maps seems to have gone quiet, with the latter technique seeing the most use and research of late. Even id Software, who used volumes so prominently in id Tech 4 (Doom 3 et al.), has switched to maps for iT5 (Rage) going forward.

If they are indeed dead, what factors have combined to kill the feasibility of stencil shadow volumes between the last generation and this one? Is it the explosion in vertex count hindering silhouette extrusion? The inability of fillrate to scale with the rest of the scene? The inherent limitations for content creation?

Or are volumes still a contender? There have been papers extending the algorithm to work on non-manifold meshes; to include transparent shadow casters; to cast soft shadows using penumbral wedges or screen-space filtering. These near feature parity with shadow maps (at a cost!) while retaining volumes' advantage: they don't have the pixelization problem that plagues maps.

Most research in shadow maps seems to have gone toward reducing the shadow aliasing the technique invariably produces, with varying degrees of success. Try as they might, the resolution never seems to be enough. Are maps of some flavor really the best foreseeable shadowing technique for this generation? If so, which shows the best promise, in your eyes?

And what about looking further? Is there a hardware irregular Z-buffer in our near future? Will shadows be one of the first applications for a hybrid rasterize/raytrace part? Or do you see something more exotic or mundane?

 

[Mintmaster]

Even last generation you could only get away with stencil shadows in certain situations.

Silhouette calculation isn't that's bad, especially with recent API improvements that don't need CPU intervention. The problem is much more fundamental than that: stencil shadows give you a cost for the boundaries of shadow volumes, which are invisible! Why should you pay for something you can't even see?

Light comes through a window, and you see it illuminate things with, say, a shadow of the window frame. Put blinds on the window, and you're totally screwed. Parts of the scene that are seemingly simple now have 50x overdraw. Performance is very unpredictable.

Imagine a jungle scene from Crysis or Uncharted, with the leaves from above casting shadows down on the ground. Just think about how many screen space pixels are touched by the shadow volume from every damn leaf (which, BTW, must use polygons to cast a shadow instead of an alpha texture). Moreover, the skinny triangles in the volume are not the friendliest thing to rasterize. Suddenly 50GPix/s stencil fillrate on G80 looks woefully inadequate.

You have to severly constrain your game's art to use shadow volumes in practice.

 

[joker454]

What the heck, I'll throw my hat in the ring. I'm not an expert on shadows so maybe I'll learn a thing or two I'll repost part of something that I posted in another thread.

We're using a mix of shadow techniques in our sports game, but we are still using shadow volumes for stadium structure. A shadow volume for a given stadium is small, typically < 300 verticies which means all we need is < 2k of overhead to get stadium shadows. With such a small vert count, extrusion in realtime is dirt cheap which is good, since we are outdoor (and hence a moving sun) so the shadows gradually change quite nicely over the course of a game.

Items receiving shadows are the players and the ground, which are far away from the stadium structure at all times. So, stadium structure is effectively at "infinity" and hence we can do a uniform blur on the shadow, and the shadow cast on the ground/players by stadium structure looks correct.

We can do some cheats on player-player shadows, but regarding stadium structure, all of it must cast a shadow, even structure that is very far away, so this made shadow volumes even more appealing because we could get every bit of structure to cast a good looking shadow at all times.

So, it came up in the other thread that stencil shadows are dead, which may very well be true for all I know. But in the above case for our particular situation, we can use shadow volumes to get a correct look that takes barely any memory, runs fast, and is simple to implement which is important in a sports game which has a yearly deadline set in stone. So why would shadow mapping be better than shadow volumes in this case? What would they give me that would render shadow volumes dead in comparison?

 

[Mintmaster]

joker, typically with shadow volumes lighting is done with multipassing. With your high poly counts and MSAA enabled in your game, how are you acheiving it? Hacking it by just darkening the shadowed areas? Feeding the shadowing term into the shader with a texture and not worrying about MSAA incorrectness? How do you blur a stencil shadow? In screen space?

EDIT: For a stadium, it seems like DX7-style projected shadows are enough. Render the shadow opening into an 8-bit grayscale texture, blur it, and project it on everything. Though I guess self-shadowing of the stadium is probably an effect you'd want, so we can scrap that idea.

 

[joker454]

joker, typically with shadow volumes lighting is done with multipassing. With your high poly counts and MSAA enabled in your game, how are you acheiving it? Hacking it by just darkening the shadowed areas? Feeding the shadowing term into the shader with a texture and not worrying about MSAA incorrectness? How do you blur a stencil shadow? In screen space?

EDIT: For a stadium, it seems like DX7-style projected shadows are enough. Render the shadow opening into an 8-bit grayscale texture, blur it, and project it on everything. Though I guess self-shadowing of the stadium is probably an effect you'd want, so we can scrap that idea.

The answer can get pretty convoluted, since in total we're using 4 different shadow methods. Stadium's casting on players is handled one way, stadium self shadowing another way, player to player and player self shadow a third way, and players casting on the ground yet a fourth way. Some of these are precalculated, some are realtime but I can't get any more specific than that. But this combination of methods (which include shadow volumes, shadow mapping, and others) lets us get changing shadows over a 24 hour period at 60fps.

Using 4 shadow methods sounds crazy at first I agree! Until you look at the specifics of a baseball game and realize that a lot of cheating can be done. What we do would not work for most games, but baseball lets us get away with it. I guess that's why I hesitate to call shadow volumes dead at this point since you just never know, sometimes there are particular games which can still benefit from it. I don't really see offhand how shadow mapping on it's own could be faster for all the shadows we need on baseball compared to what we're currently doing.

 

[Mintmaster]

I guess that's why I hesitate to call shadow volumes dead at this point since you just never know, sometimes there are particular games which can still benefit from it. I don't really see offhand how shadow mapping on it's own could be faster for all the shadows we need on baseball compared to what we're currently doing.

If you handle stadium shadows on players separately, then the method I described is really cheap. I guess it depends on how soft you want the shadow edge, though. A slightly soft edge (i.e. less than 1/1000th the width of the opening) is hard to do that way without a huge texture.

Anyway, how did you incorporate it into the lighting equation? Did you multipass or do something else and accept the drawbacks?

 

[joker454]

Anyway, how did you incorporate it into the lighting equation? Did you multipass or do something else and accept the drawbacks?

It ultimately ends up as a 2d texture lookup (screen space) which gives a 'scale' value to the diffuse color. After that it's run thru the lighting equations as normal. Our pixel shaders are still more instruction heavy than sampler heavy, so the texture sample for the shadow 'scale' value gets absorbed, leaving the only overhead as a single multiply.

The drawbacks aren't that bad really. Reviews have complained about other little graphic tidbits here and there, like eyes not looking quite right or a skin shader being a touch off, but they haven't complained about the shadows at all. So we've left them as they are to focus more on the graphic setbacks people have been mentioning, such as not being at 60fps.

 

[Mintmaster]

It ultimately ends up as a 2d texture lookup (screen space) which gives a 'scale' value to the diffuse color. After that it's run thru the lighting equations as normal. Our pixel shaders are still more instruction heavy than sampler heavy, so the texture sample for the shadow 'scale' value gets absorbed, leaving the only overhead as a single multiply.

Okay. So then AA gets messed up a bit. For example, if a 4xAA pixel was 2 samples fully lit red and 2 samples shadowed green (20% lit), you'd get a resolved pixel of (0.3, 0.3, 0) instead of (0.5, 0.1, 0). I guess the artifacts wouldn't be too noticeable. I'd imagine UE3 games on the PC have similar problems with AA even with DX10.1.

Do you do a Z prepass? If not, how can you figure out the shadows before you run your lighting shader? Also, how are you blurring the stencil shadow? Screen space sound like it would blur the shadow into the wrong areas.

 

[Fafalada]

Even last generation you could only get away with stencil shadows in certain situations.

Actually last generation stencil shadows were a lot more feasible to use globally then this one.

With the right data-structures, silhouette calculations are really cheap - or better yet, if you have topology access available in your vertex processors(ala PS2), they are effectivelly free - you'll never be vertex limited at all.
Fillrate always remains a problem, but there are ways to optimize that (on last generation easier for static static volumes, but regardless)

Screen space sound like it would blur the shadow into the wrong areas.

As Joker noted, if you're assuming caster at infinity from receiver, the blur becomes uniform, in which case even simple 2d filter will generally look correct - or at least perceptually a lot more real then a sharp or aliased shadow.
Ultimately it's not the math correctness to worry about but fooling the eye.

A shadow volume for a given stadium is small, typically < 300 verticies which means all we need is < 2k of overhead to get stadium shadows. With such a small vert count, extrusion in realtime is dirt cheap which is good, since we are outdoor (and hence a moving sun) so the shadows gradually change quite nicely over the course of a game.

Actually this is a lot like what I was getting at - the background shadows is where volumes can still have their place. The comment I made was referring to more complex geometries, and with possibility of precomputed volume against volume collision (or have the same collision placed and do it in realtime - on current consoles it's feasible).
Optimizing a volume in that manner yields giant fillrate savings (just for reference, we had stages on PS2 where volumes consumed as much as 50-100k polygons per frame, but didn't impact performance significantly thanks to such optimizations), and storage requirements are still fairly minimal (an indexed strip typically yields 4bytes/vertex with 16bit compression).

If you can get away with lightsource changing position only at begining of a level so you don't even deal with extrusion memory overhead and stuff in realtime, the tradeoff performance/result seems pretty good.
Additionally you can then play around with more optimally allocated shadowmap resolution for local stuff like characters etc.

The issue remaining is always making them nicely soft, but like you noted, there are areas where you can get away with it. And for that matter lots of globally shadowmapped games get away with far worse, and still not all that soft shadows.