Spherical Harmonics Lighting in videogame

[Farid]

*I know the term Spherical Harmonics Lighting is used "loosely", but since there's no other (widely accepted) term, i'll stick with this name.

Is there another game besides Conker Live and Uncut that use SHL*? I've also heard some people saying that Silent Hills may use the effect, somebody knows for sure?

I'm asking that since lot of people think that SHL is one of the next big thing in the industry;
for my part, i quite agree with this point of view, since SHL is a cheap and impressive effect which is exactly what real time 3D is about...

Links about SHL:

"Spherical Harmonic Lighting, the Gritty Details" by Scea research

A demo (PC)

A recent .pdf about SHL

A .pdf by ATi about... SHL

SHL in OpenGL

 

[Guden Oden]

and real-time spherical harmonic lighting creating perhaps the most realistic shadows I've ever seen

This is BS. No current realtime 3D lighting process creates shadows, the shadows are always rendered separately or pre-rendered in the form of lightmaps and applied with multitexturing.

The guy's confused about what he is seeing, or actively misled by Rare about what it actually is they're doing.

 

[Farid]

and real-time spherical harmonic lighting creating perhaps the most realistic shadows I've ever seen

This is BS. No current realtime 3D lighting process creates shadows, the shadows are always rendered separately or pre-rendered in the form of lightmaps and applied with multitexturing.

The guy's confused about what he is seeing, or actively misled by Rare about what it actually is they're doing.

Indeed the great reporter from IGN is not the sharpest tool in the shed, but you know how thoses public presentations work, the devs (or worse the PR guys) threw some words "that sounds cool", the journos wrote them down, and... voila!
Anyway, we're sure, like you said, that the Ign guy didn't come out with a word such as SHL by himself, he just don't have a clue about what it's about.

 

[MrWibble]

Hmm. None of the screenshots look to me like they've got particularly impressive lighting.

The talk (in the IGN article) of "realtime SHL" amused me - seeing as the whole point of the technique is to encode expensive lighting calculations in kind of advance look-up table for realtime usage. Not a lot of point using it if it's not realtime...

I've yet to see anything that I think looks like it's using a PRT technique. Howver I'm a little susprised at that, because it's pretty achievable on todays hardware ("even" a PS2 can manage it, if you can squeeze in a lot of data and don't mind fairly extreme tesselation). Maybe I'm just looking at the wrong titles...

I kind of agree that PRT is probably going to be a big feature of rendering engines in the near future, but I'm not sure that SH's are the way to go - they're pretty poor at doing high-frequency stuff (i.e. details and sharp edges) and already a lot of the research is moving towards different basis functions, like wavelets.

They also have some big issues - mostly centered on the fact that as it's a pre-computation reliant effect, it can't cope with dynamic scenes. So objects can't move or be animated unless the lighting effects are very localised... Fine for a rigged tech-demo, but a bit of an issue for a game.

I also think they're only a stop-gap until GI is a little more practical in real-time, and maybe thats not so far off...

(and Guden - PRT lighting techniques *do* include shadowing as an integral part of the process, but then it's also arguable that it's not really "lighting" at all...)

 

[Fox5]

Why can't games just use it for static objects, and then the actual characters can use some other type of lighting technique?

Maybe conker does?

http://media.xbox.ign.com/media/490/490304/img_2124209.html
In all the images I've seen, the characters and vehicles appear to be lit differently than the environment. They're always brighter.

Man, I hope xbox is down to $100 with a free game by the time conker comes out...gamecube already is.

 

[MrWibble]

Why can't games just use it for static objects, and then the actual characters can use some other type of lighting technique?

No reason at all - if they're doing PRT then the chances are they're doing it this way.

I just think that if I was going to all that trouble to calculate the lighting it'd have to look better than average to be worth the effort... So far, I'm unimpressed.

 

[Fox5]

I dunno, I thought conker looked pretty nice.

BTW, supposendly rebel strike had some kind of lighting special effect(simulating how light reflects in the atmosphere), but I doubt most people noticed it, so what ws the point of them doing that?

 

[Squeak]

What is the actual difference between SPH and precomputed radiosity?

 

[Guden Oden]

(and Guden - PRT lighting techniques *do* include shadowing as an integral part of the process, but then it's also arguable that it's not really "lighting" at all...)

PRT?

Anyway, I was talking about the lighting schemes that can be implemented on today's realtime polygon-based renderers in current consoles, and they don't generate shadows as an integral part of the rendering, because if they did shadow contours would have to follow the 3D mesh the characters and world is created of, and that would just look fuckin weird.

 

[Farid]

BTW, supposendly rebel strike had some kind of lighting special effect(simulating how light reflects in the atmosphere), but I doubt most people noticed it, so what ws the point of them doing that?

IIRC, Rebel Strike had a HDR-like effect, IGN, ..., made quite a fuss about it.
HDR-like effects are somewhat noticeable, even for someone who's not a "3D enthusiast", and Factor 5 bothered to ad this effect because, it's the details that make the difference between something "OK" and something "OMG11!1", people can't tell the subtilities (with respect to the effects/techs used) even in CG movies, but if you remove all of thoses at once, then the difference will become obvious for them.

...That or Factor 5 were experimenting thing with their renderer and add it... because it "looked cool".

 

[Cryect]

and real-time spherical harmonic lighting creating perhaps the most realistic shadows I've ever seen

This is BS. No current realtime 3D lighting process creates shadows, the shadows are always rendered separately or pre-rendered in the form of lightmaps and applied with multitexturing.

The guy's confused about what he is seeing, or actively misled by Rare about what it actually is they're doing.

So stencil shadows are fake? Heh they are done at the exact same time as the lighting is. And using penumbra wedge's you can get impressive looking RT soft shadows (though a few issues do exist with them).

Also not sure why SHL is exactly called realtime, since pretty expensive preprocessing required and it only works with static objects doesn't seem much more useful than any other GI schemes.

 

[MrWibble]

PRT?

Anyway, I was talking about the lighting schemes that can be implemented on today's realtime polygon-based renderers in current consoles, and they don't generate shadows as an integral part of the rendering, because if they did shadow contours would have to follow the 3D mesh the characters and world is created of, and that would just look fuckin weird.

PRT = Precomputed Radiance Transfer.

Thats the name of the overall technique of calculating the interaction of light between surfaces and storing the resulting function in some form for later evaluation. Spherical Harmonics are a popular set of functions to use for this, but they're not the only/best solution.

You can implement a PRT based rendering solution on a current console, and whether it looks weird or not depends on the level of tesselation. One of the problems of SH based calculations is the low frequency of the lighting information which makes everything kind of blurry. However that has a side-effect of lessening any tesselation based nastiness, so you kind of get away with it. You could even be cheeky and tesselate based on the potential variation in lighting to increase detail where it matters...

However on the whole I think you're right - we need more memory and better pixel-shaders to really take advantage of these techniques.

 

[ERP]

FWIW - I'm not even convinced it's a good solution for nextgen platforms.

I sat down and implemented PRT with various basis functions, and compression schemes and unless you have a relatively constrained scene the amount of memory requred gets very large very quickly.

The other technique that's often referred to as Spherical Harmonic lighting is SH evnvironment maps (used for diffuse lighting). This is in use in games today.

I really wish people would stop refering to Spherical Harmonics as a graphics technique :/

 

[Guden Oden]

So stencil shadows are fake? Heh they are done at the exact same time as the lighting is.

Um, forgive me for asking, but how can they be done at the exact same time when stencil shadows are untextured pixels drawn into the stencil buffer and not lit, textured pixels belonging to the world/object polygons?

Also, as you need to calculate shadow outlines and extrude them in the direction away from the lightsource you're by definition creating new polygons anyway so again how could it be done at the exact same time?

Just curious, maybe you got the answer.....

 

[DeathKnight]

I think he was referring to the fact that they're calulated during the rendering process, not pre-computed into some kind of shadow map beforehand.

 

[Cryect]

Yeah DeathKnight pretty much got what I wanted to say. Basically when you said shadows drawn seperately I was thinking like projected shadows. I'm thinking of stencil shadows still being lighted at the same time due to when the lighting is run you don't bother lighting the shadowed pixels where as algorithms like projected shadows just draw over those lit pixels.

Also with the penumbra wedges are applied at the same time as the polygons are lighted. You seem to really want to say if something isn't calculated at the same time as the lighting pixel shader its not being calculated at the same time while I'm considering a more broaden view of at the same time.

There isn't a magical algorithm that handles both light and shadows all at once except for ray tracing or if you calculate the light volume and clip the polygons to that and just draw the polygons making up the light volume then (and stencil shadows are just much easier). Plus by your definations sounds like since its clipping polygons before lighting must not be at the same time.

Edit: Also any various algorithm that handles the lighting equation as its correct integrals will work as well for handling lighting and shading in one go and of course not be feasible in realtime.

 

[Guden Oden]

You seem to really want to say if something isn't calculated at the same time as the lighting pixel shader its not being calculated at the same time while I'm considering a more broaden view of at the same time.

Well I didn't really mean anything else by using the words "at the same time" than what they actually mean.

I guess it would be neat if we had a shadow model that could handle having a tree standing on a hill lit by a light (like a simulated sun for instance) and then have the tree cast a shadow down the hill using only gouraud shading so it could all be done in one pass, but it would require a damn fine polygon mesh or else the shadow would just look like crap, especially with the thin branches and twigs.

Not sure there's a solution that is fast enough to do that in real-time, considering some shapes (like trees for example) have a very complex silhoutette...

 

[DeanoC]

So stencil shadows are fake? Heh they are done at the exact same time as the lighting is. And using penumbra wedge's you can get impressive looking RT soft shadows (though a few issues do exist with them).

Penumbra wedge have ugly artifacts on flat objects, I've seen no solution that would allow them to be used in these situations yet (maybe o.k. for characters but not buildings)

Also not sure why SHL is exactly called realtime, since pretty expensive preprocessing required and it only works with static objects doesn't seem much more useful than any other GI schemes.

Because diffuse irradiance mapping (what most people mean by SH Lighting) is pure real-time, no precalculation at all (you simple convert you lights in a SH basis before using them). PRT calculates the response to light, which is precalculated but isn't static in the usual sense, if you have the memory and calculation time you can account for both viewer and light changes. More commonly you can account for moving lights but no viewer changes (diffuse only PRT, no specular)

PRT adds an extra degree of freedom from standard GI techniques (the lights themself can move with PRT, whereas with GI can't) (sorry for the abuse of the term GI, I mean really static lighting).

Some shadow (low frequency soft self shadows) are incorporated into PRT, until we get higher frequencies and the ability to merge set of PRT coefficents (so one object can shadow another) we will be 'stuck' with stencil shadow or shadow maps for a while.

You can argue very well that any system using ambient occulusion is already doing very low frequency PRT. So its a technique that will definately be used but the memory cost (as ERP mentions) may mean it never getting beyond ambient occulusion.