Unreal Engine 3 shadowing?

[Intel17]

I was reading through some old B3D posts (because I didn't want to repost or anything), and I found an interesting tidbit from Daniel Vogel on Unreal Engine 3 shadowing...

Static light, static object:
precomputed light occlusion term either per vertex or texel (if mesh is uniquely unwrapped)

Static light, dynamic object:
per object shadow depth buffer, resolution based on the screenspace size of the shadow bounding sphere

Dynamic lights:
stencil shadow volumes

The different approachs work together nicely as they only contribute the light occlusion at a given pixel which is then used by the lighting shader so the only visual difference you see is the amount of fuzzyness in the shadow. The reason for mixing the three approachs is that neither of them alone would be sufficient for the kind of vivid environments we want to create.

-- Daniel, Epic Games Inc.

Now, if a static light on a dynamic object (such as a character) casts a soft-shadow, but a dynamic light on a dynamic object casts harsh-edged stencil shadows, wouldn't that look pretty odd? In fact, what if in a scene a static light is shining down on a character, casting a soft, fuzzy shadow, then a player with a flashlight shines it on that same character, wouldn't that look out of place?

Also, didn't Tim Sweeney at this year's GDC clearly say something like (paraphrasing here) "This is going to be a hallmark in next generation games, every object in the world has the ability to cast soft-fuzzy, dynamic shadows"?


Anyone have any information or speculation on this topic?

 

[Razor1]

shadows are very tricky, I would blow a gasket if all objects cast volumetric soft shadows, right now with the polycounts they are using it would be impossiable for them to do it. Possible to combine both volumetic and shadow maps. But with the shadow maps specially for very good looking ones vram usage is expensive, a 256 mb card is not enough, 512 might be able to handle it, but that too might have tough times. Maybe using different types of shadows depending on distance.

Like soft volumetric upclose, then use shadow maps for a little farther away, then use hard edge volumetic for far off objects would be a good solution.

 

[ShootMyMonkey]

Well, in the presentations, he did mention things like pixel shaders that used around 50 instructions or less. If UE3 can afford to be that loose with instruction count on pixel processing, then there are some cheap tricks you can do with stencil shadow volumes to fake soft shadows. I know about the paper on penumbra wedges, but for speed's sake, you'd really want to think about looking at the Z-buffer and testing how close you are to pass-fail and fudging in a fade-out of lighting calculations (branching out of lighting if you're in the "no-fade" zone). Also, for good measure, the width of the fade region can be determined based on the pixel's distance from the lightsource. Inaccurate, all right, but at least may not look so garrish against other soft shadows.

 

[Reverend]

I don't think the engine is as hard-coded as it sounds as per Daniel's descriptions. Or maybe I'm wrong to read a little more into what Tim Sweeney said :

Tim Sweeney, to Rev via email[/url], ]Stencil shadows are the most general solution, and are used for interactions between moving lights and static geometry. Shadow buffers are used for static lights shadowing characters and other dynamic objects. UE3 is structured so that all shadow types and all light types are fully compatible and can be mixed arbitrarily within a scene.

 

[Razor1]

Well for static objects if PRT, or LPRT (which are definitly shown off in the first recorded demo at GDC 2 years back as Intel quoted precomputed light term) is used that will take care of all thier shadowing. All thats left is well animated objects, best case scenario, penumbra shadow volumes for dynamic light sources, and ambiant oculusion for static lights and animated objects.

I wouldn't see why they would use soft shadow maps for animated objects with static lights, its a waste of vram. Specially since texture detail on the meshes need to be very high for the level of detail they are going for.

 

[Intel17]

I don't think the engine is as hard-coded as it sounds as per Daniel's descriptions. Or maybe I'm wrong to read a little more into what Tim Sweeney said :

Tim Sweeney, to Rev via email[/url], ]Stencil shadows are the most general solution, and are used for interactions between moving lights and static geometry. Shadow buffers are used for static lights shadowing characters and other dynamic objects. UE3 is structured so that all shadow types and all light types are fully compatible and can be mixed arbitrarily within a scene.

@Reverend-Well by what Tim says (interesting tidbit, by the way), I think he means more that they all look great together, but I don't think you could specify special cases (without touching the renderer code). Note on the UE3 website it says the techniques are visually compatable, which is, what I believe, Tim is saying.

Also, I have gotten confirmation of my theory but due to an agreement I'm not really allowed to be more specific

Well for static objects if PRT, or LPRT (which are definitly shown off in the first recorded demo at GDC 2 years back as Intel quoted precomputed light term) is used that will take care of all thier shadowing. All thats left is well animated objects, best case scenario, penumbra shadow volumes for dynamic light sources, and ambiant oculusion for static lights and animated objects.

I wouldn't see why they would use soft shadow maps for animated objects with static lights, its a waste of vram. Specially since texture detail on the meshes need to be very high for the level of detail they are going for.

They don't use pre-computed lighting, just pre-computed shadowing for static-light interactions...

This is a single-component, 8-bit-deep texture map applied to surfaces in a similar way to a lightmap (though lightmaps store 3-component RGB values). A shadow mask's greyscale values indicate the percentage occlusion from the light to each texel on a surface, independent of the light's attenuation, color, and brightness. The shadow mask stores the shadow part of a lighting computation in a precomputed way, while the per-pixel normal-mapping, specular lighting, and attenuation are done dynamically.

 

[Razor1]

They don't use pre-computed lighting, just pre-computed shadowing for static-light interactions...

This is a single-component, 8-bit-deep texture map applied to surfaces in a similar way to a lightmap (though lightmaps store 3-component RGB values). A shadow mask's greyscale values indicate the percentage occlusion from the light to each texel on a surface, independent of the light's attenuation, color, and brightness. The shadow mask stores the shadow part of a lighting computation in a precomputed way, while the per-pixel normal-mapping, specular lighting, and attenuation are done dynamically.

Well that is PRT lighting. Which is per vertex anyways, then the per pixel lighting is done for normal mapping or displacement mapping, so it all ends up calculted for the surface lighting. There has to be some precomputed things in there. Some of the shots that were shown off like the light dispersion, and radiance threw a prizm type material, is just impossible to do in real time right now. the SH values are precomputed, what ever is done after that yes is not pre calculated


PRT saves the shadow data and light data either per pixel or per vertex. This data can be put into a texture. Would possible think a 2D texture will work well to some degree, which you noted when you stated full unwrapped meshes. Although it might get tricky

But for lighting it isn't necessary to do this per pixel, per vertex is more then enough since there are other calculations to do be done, which then this data can be stored in the vertici's themselves, much like the way per pixel lighting is calculated with regular lighting in Doom 3, or most engines for that matter.

 

[Mate Kovacs]

Well that is PRT lighting.

"Are you sure about that?" I mean, "Nope".
IMHO what Sweeney mentions is not precomputed radiance transfer, just an occlusion-ratio map.

 

[Razor1]

Well that is PRT lighting.

"Are you sure about that?" I mean, "Nope".
IMHO what Sweeney mentions is not precomputed radiance transfer, just an occlusion-ratio map.

IF it wasn't used the effects they showed are not possible on todays hardware and would be very slow in the next line of hardware too.


Adding up the total number of passes, you have shadows at least 2, you have light scattering thats 7 passes http://www9.informatik.uni-erlangen.de/Persons/Stamminger/Research/Research/tsm/tsm.pdf (this is one of the more less stressful ways of doing light scattering without some sort of precalculation) , you have HDR, thats 2 passes. Then we know that HDR will be doing 3 passes on the r4xxx's, So you have between 12-13 passes, then if you have reflections in the screen, thats another extra pass if its real time, then if you have any sorting going on.

That does't seem very logical for all the geometry to go through all these passes specially for static geometry. It would be more feesable to do only the animated objects as you said with occlusion-ratio maps (I'm guessing those are something similiar to occulusions maps, I have never heard of ratio maps before well I did only in cases of a voxels but thats compeletly different) but even then thats not going to happen.

Opps sorry thats 14 passes for light scattering with out PRT for an entire scene.

 

[see colon]

some scenes in DooM3 use what amounts to lightmaps (or shadowmaps, however you want to look at it, for grate textures, and some fans, for example) instead of the standard stencil shadows and generaly speaking, in motion, everything looks fine. if you are looking you will notice the projected textures are soft edged while the stencils are hard edged, but they really don't jump out at you as much as one might think. at least they didn't to me.

then again, they are more noticable at higher resolutions, and i played the game through at 800*600 (2xAA) for a more consistant framerate.

 

[JD]

I use both hard stencil shadows with cube projected textures and scene looks great so this mixin is not a problem in my view. Actually, the projected textures add atmosphere to the scene because of their softness. You can make them hard edged as well as soft depending on filtering of the texture so artists can go wild with them. They can make them dark or gray and thus change their intensity. I use 80% black I think in doom3 like scenes.

The thing to understand with stencil shadowing is that in presence of many lights sources the shadows brighten up so the contrast with a wall isn't as strong as if you had a single light source.

 

[Mate Kovacs]

IF it wasn't used the effects they showed are not possible on todays hardware and would be very slow in the next line of hardware too.

I did not say they aren't using PRT. I just wanted to point out that the above-mentioned single-component, 8-bit-deep texture is incapable of storing PRT information at all. So it has hardly anything to do with the PRT effects in the engine.

 

[Razor1]

IF it wasn't used the effects they showed are not possible on todays hardware and would be very slow in the next line of hardware too.

I did not say they aren't using PRT. I just wanted to point out that the above-mentioned single-component, 8-bit-deep texture is incapable of storing PRT information at all. So it has hardly anything to do with the PRT effects in the engine.

Then why would you want to do shadows using occlusion maps if PRT takes care of that for you?

 

[Mate Kovacs]

Because PRT is limited to low-frequency lighting environments (so no point lights for example).

 

[Razor1]

Because PRT is limited to low-frequency lighting environments(so no point lights for example).

High frequancy lights create hard shadows not soft shadows....

 

[Mate Kovacs]

Let me just quote the original conversation:

They don't use pre-computed lighting, just pre-computed shadowing for static-light interactions...

This is a single-component, 8-bit-deep texture map applied to surfaces in a similar way to a lightmap (though lightmaps store 3-component RGB values). A shadow mask's greyscale values indicate the percentage occlusion from the light to each texel on a surface, independent of the light's attenuation, color, and brightness. The shadow mask stores the shadow part of a lighting computation in a precomputed way, while the per-pixel normal-mapping, specular lighting, and attenuation are done dynamically.

Well that is PRT lighting.

PRT is for dynamic lighting. The above-mentioned occlusion map is for static lighting. So the occlusion map technique mentioned above by Sweeney can not be PRT. If you still can't understand, I give up.

 

[Razor1]

Let me just quote the original conversation:

They don't use pre-computed lighting, just pre-computed shadowing for static-light interactions...

This is a single-component, 8-bit-deep texture map applied to surfaces in a similar way to a lightmap (though lightmaps store 3-component RGB values). A shadow mask's greyscale values indicate the percentage occlusion from the light to each texel on a surface, independent of the light's attenuation, color, and brightness. The shadow mask stores the shadow part of a lighting computation in a precomputed way, while the per-pixel normal-mapping, specular lighting, and attenuation are done dynamically.

Well that is PRT lighting.

PRT is for dynamic lighting. The above-mentioned occlusion map is for static lighting. So the occlusion map technique mentioned above by Sweeney can not be PRT. If you still can't understand, I give up.

it stores it in a precomputed way, this is very similiar to what is done in photon mapping, which PRT is very similiar to, if you use a texture to save the shadowing data which was mentioned earlier.

If you were using psuedo global illumination why would you want to not use the same SH values for static objects, lighting also, you just repleace the vertex lighting with the premade SH values, keeping the light similiar would be avantagious for the level desigers, and this has almost no slow down compaired to regular vertex lighting. Its not much more complex once those values are pre computed and stored.

Maybe they found a limitation to PRT shadowing since those shadows aren't going to be interactive to other animated objects that come into the scene so they use both?

 

[KimB]

PRT is useful for dynamic lighting with static geometry. Since UE3 has no listed lighting model for this situation, you can be sure they're not using PRT.

 

[Razor1]

The lighting and shadowing pipeline is 100% uniform -- the same algorithms are supported indoor and outdoor. Various kinds of lights (omni, directional, spot) are provided for achieving different kinds of artistic effects.

All lighting is supported 100% dynamically. There is no static global illumination pass generating lightmaps, because those techniques don't scale well to per-pixel shadowed diffuse and specular lighting.

http://www.beyond3d.com/interviews/sweeneyue3/index.php?p=2


There are no static lights. So even if a light is "static" not moving, its still concidered dynamic. Very similiar to the Doom 3 engine.

In the videos that were recorded at GDC last year, they mentionsd interrefractions between objects and light reflections with in certain objects, thats not possible with some sort of precalculations before hand. And right now PRT is the easiest to implement to get those effects to work with no overhead. Making a radiace transfer engine or a pseudo global illumination engine will take months to make just the lighting system.

 

[Ostsol]

I seem to recall, perhaps in one of the GDC videos, mention that some lighting can be burnt into static lightmaps. . .

EDIT: Nope, not in the GDC videos. I don't know where it was, but I'm still quite sure of it. It may not have been traditional lightmaps, either, but some other form of static data to hold lighting information.