How can that be possible? IIRC, the pertubation you add to the normal lies in the plane of the tangents.
Very true, but the perturbation is added to a normalized (lenght 1.0) normal, so if you were to get a gradient (rise/run rate of change) of greater than 1.0 in the direction of the normal, (because the gradient value is subtracted from the normal) it would perturb the normal to point in the opposite direction.
That is hard to understand, so let me clarify with an example:
------------
N = original smooth surface normal
N' = bump mapped surface normal
P = perturbation vector (composed of rise/run slopes, gradients, in x, y, and z)
N' = N - P
N = (0,0,1) = normalized normal pointing towards positive z
bump map gradient in x = 0
bump map gradient in y = 0
bump map gradient in z = 3
(In other words, the bump map has a rise/run slope of 3 in the direction of positive z, and 0 slopes in x & y.)
P is constructed from the gradients:
P = (0,0,3)
N' = N - P
= (0,0,1) - (0,0,3)
= (0,0,-2)
Normalize N':
N' = (0,0,-1)
N' now points in the opposite direction, 180 degrees rotated from N!
------------
A gradient of 3 is a rise/run of 3/1, which is only an angle of... atan(3/1) = 1.25 radians or 71.6 degrees.
71.6 degrees seems like a sharp slope. (It is.) But the bump map height field has the potential to produce slopes up to 90 degrees. But that is just from my extreme example. The point in which a slope has the potential to surpass the normalized original normal length of 1.0, is when the rise/run because greater than 1.0, which occurs at 45 degrees. 45 degrees in practice is uncommon, but I was hoping for a solution that would work in all cases.
I still feel like I'm missing a better solution.
What I have done for now, works, which is to catch all perturbations of the normal vector that are greater than 90 degrees, and rotate them BACK to the 90 degree point (in the same direction of rotation of course). So, whenever the perturbation causes a >90 degree rotation, I calculate where it would be if the rotation happened to be only 90 degrees.
I hope this explains my problem better!