Can anyone explain spherical harmonics to an artist?

[Matt B]

Hey guys, the title pretty much sums up what I'd like to ask here. Basically I'm a CG games artist with an interest in technical areas and for quite a while now I've never been able to get my head around what spherical harmonics are and what they are used for in game engines. The only real information I've been able to dig up is very maths heavy and unfortunately most of it goes over my head.

I understand the principles but not necessarily all the code of most rendering techniques like ambient occlusion and global illumination but my understanding of spherical harmonics is very vague. It seems like its a way of precomputing the diffuse component of global illumination of scene elements when lit with a hemisphere sky light but I have no idea what is precomputed and what information is stored and then reused to render at real time.

If anyone could explain any of the concepts to me without resorting to the pure maths implementation then I would very much appreciate it.

Thanks

 

[ERK]

Well, since no one has yet volunteered, I guess I could give it a shot...

Let's try an analogy. Imagine sound. Sound (music, voice, etc.) can be described from an abstracted point of view as a graph of air pressure (or velocity) vs time. The graph (sound signal) can have pretty much arbitrary shape, but can be described (approximated) by the sum of all the constituent frequencies (with various weights and phases, ie the Fourier transform/decomposition of the signal). In pure math terms, it is like approximating a function using Taylor series or Fourier series or some other series basis.

The spherical harmonics are a basis for doing the same thing in two angular dimensions (and one radial dimension). They are a basis of orthogonal/independent functions out of which one can describe an arbitrary angular/radial function as a series of weighted harmonics, for instance the angular distribution of the brighness of a light source.

Anyway, this was all from my distant memory, so others please correct my mistakes. Hope this was what you were asking.

ERK

Pictures of what the shapes look like.

 

[soylent]

Instead of describing all of space as a cartesian coordinate system with 3 dimensions(x y z) you can use 2 angles( theta and phi ) and a radius(r); this is called spherical coordinates.

If you drop the radius r, the angle pair will give you the coordinates on a sphere(think latitude and longitude). Spherical harmonics are functions defined on a sphere.

What it means when functions are ortogonal is something you need to understand to know what spherical harmonics are for.

When the scalar product of two vectors is 0 the vectors are said to be orthogonal and they are perpendicular. The regular cartesian coordinate system has 3 normalized mutually ortogonal vectors, usually given the names i, j and k. These 3 vectors are said to span the entire space; I can express any vector r as a linear sum: r = xi + yj + zk, where x y and z are scalars. When you have a coordinate space you usually suppress the reference to i, j and k and simply give x y and z as coordinates: r = (x, y, z). The i, j and k vectors are implicit. In a cartesian coordinate system we can express a scalar(dot) product as the sum of the products of corresponding components: ( x1, y1 ,z1) * (x2, y2, z2 ) = x1*x2 + y1*y2 + z1*z2.

The scalar product can be extended to functions. A function f(x) takes any x for which it is defined. For any normal function you'll have some continous range of allowed x values. Since it is continous we can pick an infinite number of different x values. A function is therefor kind of like a vector, with an infinte amount of components.

Each x value behaves as a basis vector with f(x) as the coordinate. If we known f(x) for any x we know the full behaviour of the function f. Any function of the type f(x) is fully described by it's 'coordinates' in this infinite dimensional space, wheter it be sin(x) or sqrt(x) or anything we can imagine that is well-defined on our interval of choice on the real axis.

As our scalar product for functions('coordinates') one the real axis('basis') we can choose something analogous to the scalar product in cartesian space. Namely we integrate the product of two functions.

For each value of x('basis vector') we add f(x) * g(x) to our sum, and we sum over all possible values for x. This would tend to infinity so we have to mutliply by the infinitesimal 'dx' value. I.e. it becomes an integration.

Two functions are said to be ortogonal on an interval(e.g. x is between 0 and 2pi) if the scalar product defined above, is 0. These functions are 'perpendicular' if you will.

If you find an infinite set of such mutually ortogonal functions, they will also span all space; instead of defining a function in terms of x on the interval [0,2pi] we can define a function in any other basis, such as the set of functions sin(2*pi*n*x) and cos(2*pi*n*x) for integer n.

Here sin(2*pi*1142*x) is a 'basis vector' in the same sense that x = 1.234135134 is a basis vector. f then becomes c0 + c1 * sin(2*pi*1*x) + d1cos(2*pi*1*x)+ c2*sin(2*pi*2*x) ... where the coordinates of our function are the parameters cn.

Now for finding these coefficients for a function of x. Since we know a function of x can be expressed as a linear sum of our new basis functions, and we now our new basis functions are ortogonal, we can take the scalar product of a basis function and the function who's coefficients we want. In the same way as multiplying (7,8,9) with (0,0,1) gives us the z-component, 9, we can get the coefficient for our basis function from a scalar product.(up to some constant that we determine by taking the scalar product of the basis function with itself).

To help visualise it: We know we can express f(x) in terms of our basis as f(x) = c0 + c1 * sin(2*pi*1*x) + d1cos(2*pi*1*x)+ c2*sin(2*pi*2*x) ... If we take the scalar product of f(x) this means integrating over the product of f(x) with our basis function. This is a sum of products and we can, we can rewrite it as a sum of integrals, one for each product. But since these basis functions are ortogonal all integrals except one will evaluate to 0; namely , if our basis function is sin(2*pi*n*x) only the integral over cn * sin(2*pi*n*x) * sin(2*pi*n*x) will amount to anything but 0. Evaluating the integral f(x)*sin(2*pi*n*x) therefor gives us cn's value directly up to a constant.

With this little trick we can get at all the coefficients by integrating with each basis function in turn. This is the essence of fourier analysis, where a function of time is expressed in terms of angular velocity or frequency.

(an interesting result of fourier analysis, that is very zen-like: A single blip on the spacial(x) axis corresponds to all frequencies in equal amounts in frequency space. In quantum physics we can relate pairs of properties as being each other's fourier transform. Position and momentum is related via a fourier transform, therefor, if we know exactly a particles position it's momentum could be anything at all and vice versa. The more frequencies we use, the more exactly we can localize the a wave in space, and the less precisely it will be located in frequency; this is the essence of heisenbergs uncertainty principle.)

Spherical harmonics then, are ortogonal functions on the surface of a sphere. We can express any arbitrary function that is defined on the surface of a sphere as a sum of spherical harmonics. This is very useful for such functions as diffuse lighting incident on an object from the scene, precomputed radiance transfer or as a solution for the angular components of electronic orbitals in hydrogen etc. Given a cube map you can transform it into spherical harmonic lighting components and vice versa analogous to fourier transforms with sinusoidal basis functions.

 

[santyhammer]

http://en.wikipedia.org/wiki/Spherical_harmonics

Ignore the maths... You can express any point in the sphere surface with the vector between the sphere centre and that point. Just see the animated GIF on the right. The white color on the spheres mean where a vector points.

The idea is to decompose any point in the sphere in the same manner you do this with simple numbers:

10 = 1 + 2 + 7

So you decompose the sphere in a set of basis vectors, then adding/subtrating/multiplying those vectors you can get any poin in the sphere surface.. .Something like:

point in sphere = level1Vector*weight1 + level2Vector*weight2 + .... + levelVectorN*weightN

hope that helps

 

[trinibwoy]

I feel stupid

 

[Mintmaster]

Hi Matt_B,

It's actually a lot simpler than these guys are making it out to be. The best way is to look at it from a linear system point of view. Imagine a black box B that takes input x and gives output y:

y = B(x)

In our case, B will take the environmental lighting hitting an object as input and output the shadowing an object has. Now suppose you know the following from precomputation:

y1 = B(x1)
y2 = B(x2)

Light is a linear system, and linear systems have the following key property for any numbers a and b (which I will later call coefficients):

B(a*x1 + b*x2) = a*y1 + b*y2

Make sense? Basically, a linear combination of the inputs gives a linear combination of the outputs. How does spherical harmonics fit in? Imagine running your model through a radiosity renderer in 3DSMax (or whatever) with each of the following lighting environments:

You could save each lighting result in a separate texture. Now there are two keys that make spherical harmonics special:
A) You can estimate any lighting environment using a linear combination of these environments.
B) Rotating the environment only requires some operations on the coefficients.

A) is important because it means you use the same coefficients to recombine the outputs, and voila, you have your final lighting. B) is important because in our 3D world, things rotate! If you have 1000 objects in your scene, you have to figure out the coefficients of the lighting environment for each object from it's own rotated point of view of the world, which would take a long time. Fortunately, B) allows us to figure out the coefficients for only one orientation of the current lighting (which is sort of time consuming), and then do this relatively simple calculation for each object.

EDIT: I glossed over some details, but hopefully this gives you the basic idea for now.
EDIT2: Fixed the linear function property.

 

[KimB]

Well, just for kicks, I thought I'd give my little crack at it.

First, spherical harmonics are defined on a sphere. You can consider any function defined on a sphere as being a sum of spherical harmonics with various coefficients.

The coefficient of the first spherical harmonics, ell=0, just gives the average value of the function: since this spherical harmonic is uniform over the entire sphere, any number times this spherical harmonic changes the average value. Providing a value for this spherical harmonic would, for example, be important for diffuse or ambient lighting.

The second spherical harmonic, ell=1, is a wave with one oscillation over the sphere, and has three possible orientations. A sum over the possible orientations of this spherical harmonic gives a function that is a smooth gradient over the sphere: negative on one end, positive on the other. The average is zero. This might be useful, for instance, for simulating directional lighting.

The third spherical harmonic, ell=2, is a wave with two oscillations over the sphere, and has five possible orientations. If you took a cross section of the sphere with this harmonic, you would see positive values in two opposing ends, with negative values shifted 90 degrees from these.

And so it goes after that. If we want to approximate a function defined on a sphere that is very smooth, then we only need a couple of spherical harmonics. The finer directional information we want, the more spherical harmonics we need.

Spherical harmonics, by the way, pop up pretty much any time you have oscillations over a sphere. They are important for 3D graphics when you want to approximate a directional function, but are particularly useful for physics and chemistry. In chemistry, the spherical harmonics define the s, p, d, and f electron orbitals (with ell=0, ell=1, ell=2, and ell=3, respectively). In cosmology, we use spherical harmonics to examine the physics that lead to the cosmic microwave background:

Examining the statistics of the spherical harmonics that make up the above map give us insight into basic physics.

 

[Mate Kovacs]

Hi Matt_B,

It's actually a lot simpler than these guys are making it out to be. The best way is to look at it from a linear system point of view. Imagine a black box B that takes input x and gives output y:

y = B(x)

In our case, B will take the environmental lighting hitting an object as input and output the shadowing an object has. Now suppose you know the following from precomputation:

y1 = B(x1)
y2 = B(x2)

Light is a linear system, and linear systems have the following key property for any numbers a and b (which I will later call coefficients):

a*y1 + b*y2 = a*B(x1) + b*B(x2)

Make sense? Basically, a linear combination of the inputs gives a linear combination of the outputs.

I guess you're saying here that B is a linear function.
B(a*x1 + b*x2) = a*B(x1) + b*B(x2), then.

(I suppose you meant "linear combination of the inputs gives the linear combination of the outputs", so not just a linear combination, but one with the same coefficients.)

 

[Droid]

I feel stupid

No worries, you are not alone.

 

[Matt B]

excellent, thanks for all the replies, looks like there's a lot of material for me to get my teeth into. I've done a quick skim through and a lot of the complicated stuff didn't click with me but I'm going to have a proper thorough read through in a bit and then hopefully come back with some more questions.

 

[Mintmaster]

I guess you're saying here that B is a linear function.
B(a*x1 + b*x2) = a*B(x1) + b*B(x2), then.

Oops, my bad. I fixed it now.

 

[nAo]

Yesteday I explained SH to one of our artists and I tried to do that cutting most of the mathematical stuff.
We had a conversion like that:

me: so..you wanna know what SH are..ok.. do you know what blendshapes are?
him: yes, of course I do
me: then you know that basicly someone models some basic facial expressions for a character
and then an animator actually animates it blending together different poses.
him : I'm perfectly confident with this concept, it's very simple
me: Cool. now you know that more basic poses you have to combine together and more chances has an animator to obtain what he wants to see on screen.
him : Yes, that's obvious.
me: That's it, then you know what SH are. We use a set of pre built lighting poses and we combine them together to obtain a lighting enviroment. More poses we use more accurate can be our lighting representation, but also more expensive to compute and to store.
The basic lighiting poses are defined by some mathematical formula, so we don't have an artist or a programmer defining them, this is the most striking difference between SH and blend shapes.
him: What? is it that simple?
me: Yep, this is the basic concept
him: I see the light!

Well..the last line was made up by me, but all the rest is more or less real

Marco

 

[purpledog]

My turn...

I'm sure you know spherical envmaps which are textures you can map onto a sphere. I you are at the center of this sphere, for any direction you can look at, you can get a color by looking into this texture.
Spherical harmonics are essentially the same thing, just a function giving you a value for any direction. They are usually much smaller than envmap thought (image a envmap of 16x8 for instance...)

As you know as well, image can represent different data: color, alpha, normal. Same thing for spherical harmonics.
When it is used for ambient occlusion for instance, it can store the amount of light which reach one given point on an object. Again, imagine you are on that point, you can look around: when you see the light (the sky) the value is hight (white). When you don't see the light (because it is occluded by another part of the object), this is black.

SH can be used for many other different things. I believe their first use in computer graphics was for radiosity (a form of global illumination).

 

[K.I.L.E.R]

Why not simplify everything and just use a Fourier Transform?

 

[KimB]

Why not simplify everything and just use a Fourier Transform?

While spherical harmonics and Fourier transforms are similar, they are not the same. Spherical harmonics work only on the surface of a sphere. Fourier transforms work only on flat surfaces.

 

[K.I.L.E.R]

Couldn't you use a flat surface and then use texture mapping to map it over an entity or will the result be wrong due to doing the FT on a different surface?

Surely you could map a FT flat surface in some manner to a curved object?

 

[KimB]

Yes, but it will be distorted. Size scales won't accurately be represented in the Fourier transform.

 

[Davros]

Ignore all these people
its actually the sound made by a round harmonica........

 

[KimB]

Ignore all these people
its actually the sound made by a round harmonica........

Well, technically, if you have a spherical sound source, the waves produced could be well-expressed as a combination of spherical Bessel functions (spherical Bessel functions are just waves in the radial direction in a spherically-symmetric system) and spherical harmonics.

 

[Mintmaster]

Yesteday I explained SH to one of our artists and I tried to do that cutting most of the mathematical stuff.

I was going to do a graphical representation of my argument, which would basically boil down to illustrating the same point that you're making, but then I'd have to generate some images, and was too lazy to do that.

But you're right. Forget all the math. If you combine the inputs, you get a combination of outputs. It's as simple as that.