Can you explain Multisampling?

[king_iron_fist]

Hi guys, sorry to start a dumb thread but its been bugging me for a while exactly how multisampling actually works. Supersampling is ok but my knowledge of multisampling is a bit hazy.

From what I know multisampling somehow re-uses texture data and for each pixel selects a series of adjacent points to sample and compare pixel colour/data. IIRC these points are sometimes random or sometimes grid based depending on the implementation. Am I right in thinking that theoretically multisampling shouldnt have a fill rate penalty due to it not rendering any *new* pixels? If so then its got to be mainly a bandwidth hit hasnt it?

I would appreciate any answers that would clear this up for me and if possible can you spare in depth math/programming explanations and just keep it understandable ;).

[Nupraptor]

You've basically got it right. Normal Full Scene Anti-Aliasing eliminates jaggies on both polygon edges and textures. MSAA does nothing for texture aliasing. 2X MSAA works like RGSS. 4X MSAA is more like OGSS.

It still has a sizeable performance hit, but not quite as bad.

[king_iron_fist]

Um, sorry to awkward but I dont know exactly how RGSS or OGSS work :oops:

[Nupraptor]

OGSS = Order Grid Super Sampling
RGSS = Rotated Grid Super Sampling

Sorry about that, I meant to explain that, but I forgot.

To keep things simple, RGSS > OGSS.

[king_iron_fist]

sorry, im still not completely with you....

As far as I know supersampling just means rendering the image at a higher resolution and essentially scaling it down. I dont quite see how supersampling is done on a grid basis or how rotating a grid improves IQ. Sorry to be dumb... :wink:

[Nupraptor]

If you'd like, VIA has a pdf file on their site with some diagrams and such. I'll see if I can't get you a link.

Edit ~ Here we go: http://www.via.com.tw/jsp/en/vtf/presentations/3Dfx.PDF

[Bigus Dickus]

2X MSAA works like RGSS. 4X MSAA is more like OGSS.

That's entirely dependent on the chip design. While that is true for current nVidia chips, the R300 from ATi uses a rotated grid (well... a sparse sampled or jittered grid) for 2X, 4X, and 6X modes.

[Nupraptor]

I was unaware of how the R300 performed MSAA at all. Thanks for enlightening me. :)

[king_iron_fist]

If you'd like, VIA has a pdf file on their site with some diagrams and such. I'll see if I can't get you a link.

Edit ~ Here we go: http://www.via.com.tw/jsp/en/vtf/presentations/3Dfx.PDF

Hmm, that seemt to be a dead link over here :(

[Nupraptor]

Do you have Adobe's Acrobat Reader installed?

That PDF I linked to is for nVidia's cards, regarding the stuff they got from 3DFX.

[king_iron_fist]

Do you have Adobe's Acrobat Reader installed?

That PDF I linked to is for nVidia's cards, regarding the stuff they got from 3DFX.


Odd, it just spends ages trying to load but nothing appears. yeah I have the latest version of acrobat installed. Btw why do VIA have a PDF on how Nvidia used 3DFX tech.... ??

[Chalnoth]

Basic idea behind SSAA:

Each sub-pixel element to be combined for the final pixel color has uniquely-calculated z and color data. In other words, each sub-pixel is just a normal pixel. This allows the hardware to render at a higher resolution, then scale down for the final output.

Basic idea behind MSAA:

Each sub-pixel element has its own unique z data, but the color is only calculated once, with each sub-pixel element having the same color. Except, that is, at triangle edges. The color is calculated once per pixel per triangle. Here's a quick example. Imagine this configuration for 4x MSAA:

1 1
1 2

Where the 1's denote those parts of the pixel being covered by triangle one, and triangle two just touches the one part of the pixel denoted by a 2. Here the color value calculated for triangle one will determine 3/4 of the output color, with the final 1/4 being calculated from triangle 2.

Since Multisampling hardware is generally capable of calculating z values many times each pixel (up to the maximum supported level of AA), there is often very little difference in calculation required whether AA is enabled or not. The only difference is at triangle edges. The only problem is that MSAA does require every bit as much memory bandwidth (assuming no compression) as SSAA. Since modern video cards don't have infinite memory bandwidth, they take a performance hit from enabling MSAA.

But, that's not the end of the road. Since MSAA color data is often the same for an entire pixel (with 2-6 samples for today's hardware, depending), it turns out that it is very easy to compress. In fact, both the Radeon 9700 and GeForce FX compress framebuffer data. They also compress z-buffer data, which can also often be compressed more easily with FSAA enabled.

[Tahir2]

Is MSAA and SSAA the only way to do AA?

Why did the V5 5500 have AA that is only now being matched?
Is it through (1) user nostalgia warping their memories or (2) were 3dfx really onto something?

If it is (2) why hasn't another IHV capitalised on this?

edit: typo :oops:

[Althornin]

Is MSAA and SSAA the only way to do AA?

Why did the V5 5500 have AA that is only now being matched?
Is it through (1) user nostalgia warping their memories or (2) were 3dfx really onto something?

If it is (2) why hasn't another IHV capitalised on this?

edit: typo :oops:
V5 used RGSS.
So, theoretically, up until now, there has not been an AA mode that is as good, edge quality wise. GF3/4 uses OGMS at 4x. The radeon 9700 however, as noted above, uses RGMS at 4x, and a sparsly sampled pattern at 6x.

Note - super sampling (SS) has some benefits above multi sampliing (MS) - because the image is rendered at a higher resolution, texture aliasing is also cured with SS, while MS leaves texture aliasing up to Aniso/some other filtering algorithm.

[Tagrineth]

Is MSAA and SSAA the only way to do AA?

Why did the V5 5500 have AA that is only now being matched?
Is it through (1) user nostalgia warping their memories or (2) were 3dfx really onto something?

If it is (2) why hasn't another IHV capitalised on this?

edit: typo :oops:

3dfx used a supersampling algorithm. Supersampling is by default better quality than multisampling, as MS actually misses some things like alpha textures (because they're textures, duh).

Not only that, but... most of the competition until VERY recently has used what's considered an ordered-grid method... basically the chip takes the scene, renders it scaled up, then filters it down... so basically, a GeForce doing 4x FSAA at 800x600 will very simply do the calculation at "1600x1200" and then filter down (it's a little different for MSAA but you can think of it that way, really).

This results in the sampling being very simple...

Straight rendering, X is the sample point:

_ _ _ _ _
_ _ _ _ _
_ _ X _ _
_ _ _ _ _
_ _ _ _ _

Regular ordered grid sampling:

_ _ _ _ _
_ _ _ _ _
_ _ X _ X
_ _ _ _ _
_ _ X _ X

So the result is not much more coverage, especially on near horizontal and vertical edges - which will trigger two sample points at once, so there's only one gradient.

3dfx, on the other hand, went with a MUCH better sampling pattern...

_ _ _ _ _
_ _ X _ _
_ _ _ _ X
_ X _ _ _
_ _ _ X _

Resulting in vastly improved coverage, especially on near-vertical and near-horizontal edges.

...there's a tiny bit more to it (subsamples jittered as well) but that's basically the entire reason. And the reason IHV's never capitalised: 3dfx used VERY performance-expensive supersampling. ATi, however, DOES use an identical sample pattern for their 4x MSAA now... so the edge quality is ALMOST as good (as far as anyone knows, ATi doesn't jitter). :)

[Althornin]

Tagrineth - jittered subsamples IS the same as RG.
I seem to recall SA telling you that before.

[Simon F]

Is MSAA and SSAA the only way to do AA?
No, there a other ways. It's just that Super sampling (with or without SS of textures) is an easy way of (approximately) implementing AA.

Tagrineth - jittered subsamples IS the same as RG.
I seem to recall SA telling you that before.
I think you've got some terms mixed up. Jittered sampling is a way of approximating "minimum distance poisson disc" sampling by taking (usually) regular grid points and randomly displacing them by small offsets.

AFAICS the 3dfx 2x2 rotated grid looks to be one example of a sparse (regular) sampling pattern.

[Ailuros]

I'd still say that in terms of edge quality ATI's 6xMSAA is quite superior to the T-buffer's 2x2 RGSS.

[oyvind]

Is MSAA and SSAA the only way to do AA?

Why did the V5 5500 have AA that is only now being matched?
Is it through (1) user nostalgia warping their memories or (2) were 3dfx really onto something?

If it is (2) why hasn't another IHV capitalised on this?

edit: typo :oops:
V5 used RGSS.
So, theoretically, up until now, there has not been an AA mode that is as good, edge quality wise. GF3/4 uses OGMS at 4x. The radeon 9700 however, as noted above, uses RGMS at 4x, and a sparsly sampled pattern at 6x.

Note - super sampling (SS) has some benefits above multi sampliing (MS) - because the image is rendered at a higher resolution, texture aliasing is also cured with SS, while MS leaves texture aliasing up to Aniso/some other filtering algorithm.

But there is still a need for aniso when using SS, right? To make textures look sharper?

Thanks,
Øyvind

[Ailuros]

Supersampling - as implemented so far - filters textures only to a 16-tap aniso equivalent degree.

So yes for more "defined" textures, one can combine it with a higher degree of anisotropic filtering.

[RoOoBo]

Supersampling doesn't solve the problem Anisotropic is addressed to solve, the problem when the area of the pixel projected over the texture isn't a 'circle' but some kind of deformed ellipsis. It just provides more filtering (say 4X Trlinear) for textures. The filtering is still isotropic filtering ('circle based filtering').

[andypski]

Supersampling doesn't solve the problem Anisotropic is addressed to solve, the problem when the area of the pixel projected over the texture isn't a 'circle' but some kind of deformed ellipsis. It just provides more filtering (say 4X Trlinear) for textures. The filtering is still isotropic filtering ('circle based filtering').

When performing supersampling the extra texture samples will be taken at different sample points in screen space - if you project the resulting sample footprint for the final combined sample back into texture space the resulting filter is no longer necessarily circular - the shape changes depending on the projection of the texture into screen space.

Therefore it is no longer isotropic.

In fact, the filtering you get from supersampling in screen space is superior to simply applying anisotropic filtering in texture space since it is not the texture values that are being filtered but the results of the shading calculation over the sampled positions.

As an example, currently filtering of bump maps produces incorrect results because hardware filters are designed for colour values, not normals. If you linearly filter between normals the results become denormalised and as a result inaccuracies are introduced in subsequent shading calculations.

However, if you point sample the bump map while supersampling in screen space then the resulting filter will produce the correct values.

[moichi]

At NVIDIA's MSAA:

Z-values of every sub-sample points are same.

Is this true?

[Tagrineth]

I think you've got some terms mixed up. Jittered sampling is a way of approximating "minimum distance poisson disc" sampling by taking (usually) regular grid points and randomly displacing them by small offsets.

Thanks. If I'd said it I would've sounded like an idiot :lol:

I'd still say that in terms of edge quality ATI's 6xMSAA is quite superior to the T-buffer's 2x2 RGSS.

Well, yeah, sparse sampling will do that.

But as mentioned, 3dfx also jittered the subsamples slightly to eliminate patterns. The human eye picks up on patterns VERY VERY quickly and gets sick of them almost immediately, thus losing the effect somewhat... that's why all the 3dfx fans have always said, see 3dfx's FSAA [b]in action[/i] then judge for yourself. I've seen both and granted the edge quality is brilliant in ATi's 6x but I still like 3dfx's 4x most (it's wrong to call it 2x2... it's a sparse grid).

[Ailuros]

Supersampling doesn't solve the problem Anisotropic is addressed to solve, the problem when the area of the pixel projected over the texture isn't a 'circle' but some kind of deformed ellipsis. It just provides more filtering (say 4X Trlinear) for textures. The filtering is still isotropic filtering ('circle based filtering').

I said equivalent as in output.

[Ailuros]

But as mentioned, 3dfx also jittered the subsamples slightly to eliminate patterns. The human eye picks up on patterns VERY VERY quickly and gets sick of them almost immediately, thus losing the effect somewhat... that's why all the 3dfx fans have always said, see 3dfx's FSAA [b]in action[/i] then judge for yourself. I've seen both and granted the edge quality is brilliant in ATi's 6x but I still like 3dfx's 4x most (it's wrong to call it 2x2... it's a sparse grid).

I can hear 3dfx trumpeting the superiority of T-Buffer AA, even if there appears a 10x times better sollution in every aspect in the future. What exactly makes you think that I haven't experienced both yet anyway?

I still prefer ATI's sparsely sampled 6x sollution for more than one reasons. One of them would be performance in ultra-high resolutions and with 32bpp colour depth if you don't mind.

[king_iron_fist]

Cheers guys, you've been really helpful so far. :D

I'm still not 100% on all of it though..... :o

Right, random thoughts/questions :
The difference between rotated/ordered grids is its designed to improve AA on near horizontal and vertical edges. However surely this comes at the expense of reducing AA quality on diagonal edges?

From what I currently understand Multisampling "picks up" pixels on edges and then samples adjacent or semi adjacent pixels and blends the colour value. How does it recognise edge pixels though, is it through Z checks?

Surely the Gpu has to do *some* work to do multisampling as it has to retrieve and blend a massive amount of pixel values, the performance hit can't just be bandwidth related.... Or is this where dedicated AA hardware comes in?

In very crude terms it seems that multisampling has a very similar effect as to what would happen in photoshop if you manually went over all edge pixels with a blur tool......?

One scenario I can't get my head around is if you have a polygon which is only one pixel thick i.e. an aerial. If you take samples of the surrounding pixels then it won't show up on any of them and it will only register on the existing pixel - therefore the aerial will barely exist. Surely it would get fainter the more samples you take and at 16xAA it would barely register.

Sorry to keep on asking dumb questions by the way, I just really want to make sure I've properly understood it :wink:

[Tagrineth]

The difference between rotated/ordered grids is its designed to improve AA on near horizontal and vertical edges. However surely this comes at the expense of reducing AA quality on diagonal edges?

Yeah, this is true. But it's sort of like anisotropic filtering, it just isn't really needed everywhere on the screen at once. Aliasing isn't really as visible on diagonals as near horizontals and verticals. Try it yourself.

From what I currently understand Multisampling "picks up" pixels on edges and then samples adjacent or semi adjacent pixels and blends the colour value. How does it recognise edge pixels though, is it through Z checks?

Precisely.

Surely the Gpu has to do *some* work to do multisampling as it has to retrieve and blend a massive amount of pixel values, the performance hit can't just be bandwidth related.... Or is this where dedicated AA hardware comes in?

Yep! GeForce3 and 4 have no less than four Z-test units per pixel pipeline (rather than the standard one on any older core), so the computation for 4x MSAA is for all intents and purposes, free. Sorta like bilinear and trilinear filtering - at this point the TCU's can fetch the texels and blend 'em just fine :) But there's still a slight perf. difference... due to bandwidth.

In very crude terms it seems that multisampling has a very similar effect as to what would happen in photoshop if you manually went over all edge pixels with a blur tool......?

Similar, but not really. In photoshop you'll get a solid, 50/50 blend on all edges; however MSAA uses weighted averages to blend the lines. So say you have a diagonal line that, in 4x AA, has three samples per pixel on one side of the line, and the remaining one is on the other side. The video core will give a 3/4 weight to the first side with three samples.

In PS the weight will be 1:1 no matter what.

One scenario I can't get my head around is if you have a polygon which is only one pixel thick i.e. an aerial. If you take samples of the surrounding pixels then it won't show up on any of them and it will only register on the existing pixel - therefore the aerial will barely exist. Surely it would get fainter the more samples you take and at 16xAA it would barely register.

Actually you have it backwards. If all the sample points miss, more samples won't result in a fainter picture - zero samples on the aerial means 0% aerial colour. However, having more and more sample points vastly improves the odds that at least one point will hit the aerial, and having more and more points means more samples will be on the aerial.

3dfx demonstrated this very well with some of their early AA demos, for example the biplane one - there was a lot of pixel flicker on the beams between the wings on the no-AA side, but on the antialiased side the beams were always visible... because at a given time, at least one sample point was always on-target.

Sorry to keep on asking dumb questions by the way, I just really want to make sure I've properly understood it :wink:

It's no problem at all! The only truly stupid question is the one that goes un-asked.

That isn't to say there are no such things as stupid questions, of course... :lol:

[king_iron_fist]

Hey cheers for that :D

last question (I think) so when blending the sub pixel values for a given pixel is the value of the original pixel included or not?

[king_iron_fist]

Ah yeah, can you try and explain the jittering thing used by 3dfx in their supersampling in an idiot proof way? :wink:

[3dcgi]

Hey cheers for that :D

last question (I think) so when blending the sub pixel values for a given pixel is the value of the original pixel included or not?

The subpixel values are part of the pixel so there is no "original pixel" value at this point.

[Hyp-X]

At NVIDIA's MSAA:

Z-values of every sub-sample points are same.

Is this true?

No its not true.

The only (consumer) AA solution that has uniform Z values is Parhelia's FAA.
The problem with such approach is that it cannot antialias poligon intersections.
Matrox choose that so they could achieve 16 samples per pixel. (apart from storing fragment mask - a way of MSAA compression.)

[Chalnoth]

In fact, the filtering you get from supersampling in screen space is superior to simply applying anisotropic filtering in texture space since it is not the texture values that are being filtered but the results of the shading calculation over the sampled positions.

Not when you take performance into account.

For example, if you totally ignore memory bandwidth, and just look at fillrate, then two images that are taken at the same performance levels will show that the anisotropic implementation will usually do a better job of filtering the textures.

But there are certainly benefits to be had from supersampling. I just think that if it's going to be done, it should be done on a per-primitive level, only applied where it's needed (This should be possible with the pixel shaders on the NV30, don't know about the R300). Applying supersampling globally just wastes performance.

[Tagrineth]

For example, if you totally ignore memory bandwidth, and just look at fillrate, then two images that are taken at the same performance levels will show that the anisotropic implementation will usually do a better job of filtering the textures.

Granted.

But combining AF and SuperSampling results in thoroughly incomparable IQ.

You should see such a solution in action... :)

[Ailuros]

But combining AF and SuperSampling results in thoroughly incomparable IQ.

The R200 and KYRO's (only up to 2xLevel aniso) allow a AF/SS combination. If you mean that supposed Feline implementation in VSA-100 I don't want to even hear about it again.

To all those I prefer a 4xS/32-tap combination. Still an AF/SS combination in a way.

[SA]

Multisampling can be performed in several ways.

In the case of tilers, the pixel color can be computed once and applied to the visible samples in the pixel based on the sample's z values. When the tile is done, the filter can be applied to the tile and the results written to the frame buffer.

On IMRs multisampling is best done using some version of the A buffer. In this case, triangle fragments (those portions of a triangle within a pixel) set the bits of a coverage mask with one bit per sample. A single color and z value is stored for the fragment. Since there are usually only one or two fragments in a pixel, this allows significant compression of the color and z data. For 8x AA it allows up to 4 to 1 compression without loss for two fragments.

It does assume, however, that the z values are the same for each sample, which is usually not the case. Z3 handles this by including the z slopes along with the z value at the center. Other problems occur when the number of fragments exceeds what is allocated (usually two are allocated since most pixels have one or two fragments).

There are a number of sample patterns.

Ordered grids - standard row-column pattern
Staggered grids - every other row offset half a column (close packed)
Rotated grids - ordered grid rotated by a fixed angle
Sparsely sampled grids - NxN grid for N samples, one sample per row and one sample per column
Jittered grids - a uniform grid with each sample randomly jittered in +/- x and y up to half a sample step. Requires a large number of samples to work well. Similar to Poisson distribution.
Poisson pattern - a uniformly sampled random pattern. Requires a large number of samples to work well. Difficult to implement in hardware.
Others

The important aspects of a good sample pattern are:

For small numbers of samples ( say <32) a pattern that provides approximately the same number of intensity gradations as the number of samples is of primary importance. Ideally this would be true for edges at all angles, but this is especially true for near horizontal and near vertical edges. Sparsely sampled grids is best, followed by rotated grids.

The next most important factor (especially for a small number of samples) is uniform sampling. This prevents pixel popping. Care must be taken with all the above patterns to make sure the samples are fairly uniform in their coverage of the pixel and do not aggregate in clumps or along characteristic lines.

To break up aliasing across pixels, a pattern library is often used. These patterns are interleaved across several pixels (say 4x4 pixels). To avoid pixel popping, care must be taken that the pattern boundaries do not cause non-uniform sampling. Ideally, the whole 4x4 pattern looks like one uniform pattern much like seamless texture tiles.

Of course it goes without saying the more samples the better. For IMRs coverage mask (A buffer) techniques easily allow 8x, 16x, or even 32x AA without much extra memory bandwidth since it simply requires a larger bit mask. Tilers can also easily increase the samples without incurring extra bandwidth.

[andypski]

Not when you take performance into account.

For example, if you totally ignore memory bandwidth, and just look at fillrate, then two images that are taken at the same performance levels will show that the anisotropic implementation will usually do a better job of filtering the textures.

I'm not going to argue too much about this. What you say is largely correct.

I was talking purely from a quality standpoint, the key issue being that while textures have evolved so that they no longer exclusively hold colour or luminance information, filtering techniques are still based around the assumption that the information in the texture map can be linearly interpolated. Generally this assumption is a good one for many types of data, but if you imagine the case where a normal stored in a texture flips from (1,0,0) to (-1, 0, 0) there will be a point where the interpolated normal is (0, 0, 0) - not very good ;)

[king_iron_fist]

Multisampling can be performed in several ways.

In the case of tilers, the pixel color can be computed once and applied to the visible samples in the pixel based on the sample's z values. When the tile is done, the filter can be applied to the tile and the results written to the frame buffer.

On IMRs multisampling is best done using some version of the A buffer. In this case, triangle fragments (those portions of a triangle within a pixel) set the bits of a coverage mask with one bit per sample. A single color and z value is stored for the fragment. Since there are usually only one or two fragments in a pixel, this allows significant compression of the color and z data. For 8x AA it allows up to 4 to 1 compression without loss for two fragments.

It does assume, however, that the z values are the same for each sample, which is usually not the case. Z3 handles this by including the z slopes along with the z value at the center. Other problems occur when the number of fragments exceeds what is allocated (usually two are allocated since most pixels have one or two fragments).

There are a number of sample patterns.

Ordered grids - standard row-column pattern
Staggered grids - every other row offset half a column (close packed)
Rotated grids - ordered grid rotated by a fixed angle
Sparsely sampled grids - NxN grid for N samples, one sample per row and one sample per column
Jittered grids - a uniform grid with each sample randomly jittered in +/- x and y up to half a sample step. Requires a large number of samples to work well. Similar to Poisson distribution.
Poisson pattern - a uniformly sampled random pattern. Requires a large number of samples to work well. Difficult to implement in hardware.
Others

The important aspects of a good sample pattern are:

For small numbers of samples ( say <32) a pattern that provides approximately the same number of intensity gradations as the number of samples is of primary importance. Ideally this would be true for edges at all angles, but this is especially true for near horizontal and near vertical edges. Sparsely sampled grids is best, followed by rotated grids.

The next most important factor (especially for a small number of samples) is uniform sampling. This prevents pixel popping. Care must be taken with all the above patterns to make sure the samples are fairly uniform in their coverage of the pixel and do not aggregate in clumps or along characteristic lines.

To break up aliasing across pixels, a pattern library is often used. These patterns are interleaved across several pixels (say 4x4 pixels). To avoid pixel popping, care must be taken that the pattern boundaries do not cause non-uniform sampling. Ideally, the whole 4x4 pattern looks like one uniform pattern much like seamless texture tiles.

Of course it goes without saying the more samples the better. For IMRs coverage mask (A buffer) techniques easily allow 8x, 16x, or even 32x AA without much extra memory bandwidth since it simply requires a larger bit mask. Tilers can also easily increase the samples without incurring extra bandwidth.

:o

You lost me after "Multisampling can be......" ;)
Seriously though I've never heard anything about an "A buffer" before, how does this approach differ to the normal?
Erm, what exactly is Z3?
I'm also quite baffled how a uniformly sampled random pattern would work in a poisson distribution. Does this mean a new random pattern is generated every frame and repeated?
I'm also not too sure what you mean by the term intensity graduations?
And can you explain the term 'coverage mask'?

Cheers for the help :)

[Simon F]

King iron fist,
With the possible exception of Z3, most of those topics are very well covered in a good 3D graphics book. If you can't afford to buy one, find your local technical library.
Try "Computer Graphics. Principles and Practice" by Foley, van Dam et al, or "Advanced Animation and Rendering Techniques" by Watt & Watt.

[king_iron_fist]

ok, thanks - I'll check them out.

[Chalnoth]

I was talking purely from a quality standpoint, the key issue being that while textures have evolved so that they no longer exclusively hold colour or luminance information, filtering techniques are still based around the assumption that the information in the texture map can be linearly interpolated. Generally this assumption is a good one for many types of data, but if you imagine the case where a normal stored in a texture flips from (1,0,0) to (-1, 0, 0) there will be a point where the interpolated normal is (0, 0, 0) - not very good ;)

Speaking of which, it still seems to me that normal maps should be able to be filtered and MIP mapped. It will likely require a little bit more processing power than just plain linear filtering, but it should be possible. Situations like you described above should almost never occur. If it did, the hardware should just choose one or the other ((1,0,0) or (-1,0,0)). It seems to me that you'd need to just filter the textures linearly, then normalize the result (given that bump mapping techniques assume normalized normals). MIP maps would be best pre-generated.

Another way might be to do a linear interpolation in angle space (aka spherical coords) instead of cartesian. This would require quite a bit more processing, but should still be possible.

[Hyp-X]

Another way might be to do a linear interpolation in angle space (aka spherical coords) instead of cartesian. This would require quite a bit more processing, but should still be possible.

Slerp does that.
Unfortunately it's quite expensive. :(
(Altough there could be a sufficiently precise aproximation.)

[Basic]

Filtering and PS does not commute in the general case. They only commute with very simple shaders. So it doesn't matter if you do a renormalization of filtered normals, you'll still get wrong result.

Classical example is an environment mapped rough bumpmapped surface, that get filtered to a perfect mirror instead of "smearing" the reflection.

Supersampling solves it, although in a brute force way. Better texture filtering does not. (Advanced pixel shaders can also do it, but not without some effort.)

[Dio]

Try "Computer Graphics. Principles and Practice" by Foley, van Dam et al, or "Advanced Animation and Rendering Techniques" by Watt & Watt.
... but don't expect to understand much of Foley and Van Dam. I've been working in this industry the better part of ten years now and I still don't :)

Before andypski says anything, it is probably me, not the book :D

The Watt & Watt book is much more readable, although it's not well targetted at the PC space. Of course, Foley & Van Dam is still the computer graphics bible, and looks very good on the shelf.

[Dio]

Speaking of which, it still seems to me that normal maps should be able to be filtered and MIP mapped.
Mipmapping is an extremely non-trivial issue - probably more so than filtering.

For certain lighting algorithms the mipmap is a valid approximation, but other than that you have to accept that the mipmapping process will smooth out your bumps (through information loss).

For example, if you are using the conventional Phong lighting model with a high specular power, you will only get a bright specular reflection if H (the halfangle vector, the vector evenly dividing the light incident vector and the vector to the eye) is nearly exactly the same direction as N (the normal vector from the bump map). (Simple math, H.N must be very nearly 1).

So, if a single 'mip' pixel was composed of two source angles at 90 degrees to each other, the correct result is to see two half-brightness flashes of light as the H.N crosses each of the two source angles. Unfortunately, what you get with a mipped vector is a single full-brightness flash at the halfangle of the two normals. In effect, you have flattened out your bump map.

There's no easy solution to this problem.

[jb]

sa,

thanks for the info!

[Bambers]

Speaking of which, it still seems to me that normal maps should be able to be filtered and MIP mapped.
Mipmapping is an extremely non-trivial issue - probably more so than filtering.

For certain lighting algorithms the mipmap is a valid approximation, but other than that you have to accept that the mipmapping process will smooth out your bumps (through information loss).

For example, if you are using the conventional Phong lighting model with a high specular power, you will only get a bright specular reflection if H (the halfangle vector, the vector evenly dividing the light incident vector and the vector to the eye) is nearly exactly the same direction as N (the normal vector from the bump map). (Simple math, H.N must be very nearly 1).

So, if a single 'mip' pixel was composed of two source angles at 90 degrees to each other, the correct result is to see two half-brightness flashes of light as the H.N crosses each of the two source angles. Unfortunately, what you get with a mipped vector is a single full-brightness flash at the halfangle of the two normals. In effect, you have flattened out your bump map.

There's no easy solution to this problem.

With sufficient AF wouldn't the mipmaps only be used when the bumps were so small that they were sub pixel sized?

[Basic]

Mipmaping is one way to low pass filter the texture. Bilinear and trilinear is mostly reconstruction filters (going from a texture of discrete samples to a continuous image that can be sampled again at exact pixel positions). Anisotropic is a bit of both, doing low pass filtering primarily in the direction of the anisotropy.

All of it can be called "filtering". (That's what I meant by filtering above.)

So even if AF in some cases can reduce the use of anything but the most detailed mipmap, it will itself incorrectly smooth out bumps.

[moichi]

At NVIDIA's MSAA:

Z-values of every sub-sample points are same.

Is this true?

No its not true.

The only (consumer) AA solution that has uniform Z values is Parhelia's FAA.
The problem with such approach is that it cannot antialias poligon intersections.
Matrox choose that so they could achieve 16 samples per pixel. (apart from storing fragment mask - a way of MSAA compression.)


Thanks, Hype-X.


Speaking about Multisampling,
recently "Render to Texture" become basic more and more.
Especially for HDRI
But we can't render to texture with Multisampling AA.
So I think multisampling AA will be not important in the near future.

[Ailuros]

Is MSAA really impossible with render to texture (via upsampling f.e.), or would it just cost too much in performance?

[Simon F]

Try "Computer Graphics. Principles and Practice" by Foley, van Dam et al, or "Advanced Animation and Rendering Techniques" by Watt & Watt.
... but don't expect to understand much of Foley and Van Dam. I've been working in this industry the better part of ten years now and I still don't :)

"better part of ten year"
Oh I wish it were only ten years :-)

The "Aliasing and Antialiasing" chapter is quite good because, even though it contains plenty of theory (for sad gits like me), it also has lots of diagrams, pictures, and graphs to make it easy to understand what the maths is doing (also great for sad gits like me when it's early in the morning). The corresponding chapter in Watt & Watt is also quite good.

The Watt & Watt book is much more readable, although it's not well targetted at the PC space. Of course, Foley & Van Dam is still the computer graphics bible, and looks very good on the shelf.
Then get it off the shelf and on to your desk! :-)

[Dio]

Then get it off the shelf and on to your desk! :-)
My desk won't support the weight :)

[Chalnoth]

Filtering and PS does not commute in the general case. They only commute with very simple shaders. So it doesn't matter if you do a renormalization of filtered normals, you'll still get wrong result.

Classical example is an environment mapped rough bumpmapped surface, that get filtered to a perfect mirror instead of "smearing" the reflection.

Only thing is I'm not talking about API-defined filtering. I'm talking about PS-defined filtering. That means it's customized by the programmer for the specific application it's intended for. I don't see how this can go wrong.

Yes, a properly-rendered rough surface would require supersampling, but it could all be done in the PS, which should improve performance a fair bit.

I still see definite advantages to filtering of bump maps in specific scenarios, though. A good example would be ATI's own car demo. With sufficient internal accuracy, the rendering that used the 8-bit normal map should look just as good as the higher-precision map with just a bit of filtering.

[Basic]

User defined filtering in PS aren't as trivial as it might seem. Even a bilinear filter is difficult. The problem is to know how much to shift coordinates to get the texture samples. You'd need to replicate the GPUs built in mip lod selection algorithm in the PS.

Doing supersampling in PS i possible. But with Nx supersampling, at least the whole lightning part of the shader need to be duplicated N times, one for each sample. I agree that there are other operations that work fine when relying to texture filtering. The supersampled part will be slightly slower than with usual supersampling, since you'd need to explicitly shift tex-coords, and then blend the results. The part that don't need supersampling will of course be faster. And then you get the benefit of better frame buffer compression with MSAA than SSAA. With this long shaders, the frame buffer bandwidth might not be a big limiting factor though.

Overall, yes a fair performance improvement, although it might not be huge. (Far from the usual difference between MSAA and SSAA.)

The real deal would of course be to implement something like what we talked about here (http://www.beyond3d.com/forum/viewtopic.php?t=1114&start=60). :) And yes, that's pure PS, no supersampling, and would work well all the way through the mipmap pyramid. What I said should be possible on R300, and what MfA linked to should be possible on NV30.


I'm not so sure that it's easy to make ATI's car demo look just as good with a 4x8bit normal map, just by adding better filtering. What kind of filter do you have in mind?

[arjan de lumens]

Hmm .. supersampling in the pixel shader sounds a bit cumbersome, but not impossible. You could, for each datum (gouraud color, texture coordinate set) that you wish to supersample, use the derivative instructions (DDX, DDY) to compute the offsets you need to add to the datum to sample it at a given sample grid point, and repeat this procedure (and all subsequent calculations) for each sample grid point. This should be moderately slower (about 3 to 10 instructions per sample point) than hardwired supersampling, and has a small problem with what happens at polygon edges - since the pixel shader has no way of knowing which sample points are inside and outside the polygon, you don't get any edge AA with this method (can be partially fixed with multisampling), and you risk sampling outside the polygon (giving artifacts with polygons viewed from sharp angles).

Manual texture filtering in a pixel shader program sounds *very* slow - I seem to remember a figure of about 10-12 instructions for just bilinear interpolation. For trilinear, you can multiply this figure by 2 and add instructions for mipmap level computation, which will get you something like 30-40 instructions.

[psurge]

Hyp-X :

what if the values stored in the normal map were quaternions?

OTOH I can't remember what is required to convert back to a Euclidian vector, but I thought it was just a matrix multiply...

Serge

[Chalnoth]

User defined filtering in PS aren't as trivial as it might seem. Even a bilinear filter is difficult. The problem is to know how much to shift coordinates to get the texture samples. You'd need to replicate the GPUs built in mip lod selection algorithm in the PS.

Which is where two things come in:

1. This could show quite well how modern Pixel Shaders need to be improved. User-defined filtering is only natural for complex shaders.

2. Higher-level languages should be able to help out significantly if the hardware is general enough for this sort of thing.

The supersampled part will be slightly slower than with usual supersampling, since you'd need to explicitly shift tex-coords, and then blend the results.

This is just an optimization issue. We're really not to where this sort of thing is viable anyway, but hopefully next-gen hardware will improve added routines for optimizing PS filtering.

I'm not so sure that it's easy to make ATI's car demo look just as good with a 4x8bit normal map, just by adding better filtering. What kind of filter do you have in mind?

I was thinking of just using an angle-based filtering method, essentially duplicating bilinear filtering in every form, except for computing the final weighted average in angle space. Obviously this won't provide the exact same results, but the results should be very similar. This should work almost perfectly here since we're not talking about a rough surface, just a smoothly-curved and very shiny surface, though I could imagine a couple of spots where it would look a bit different than what the higher-res bump map displays.

And obviously you would absolutely need higher internal precision than is available in the 8-bit per value normal map to get the proper result.

[Chalnoth]

The real deal would of course be to implement something like what we talked about here (http://www.beyond3d.com/forum/viewtopic.php?t=1114&start=60). :) And yes, that's pure PS, no supersampling, and would work well all the way through the mipmap pyramid. What I said should be possible on R300, and what MfA linked to should be possible on NV30.

Oh, thanks, that would be cool. Finally took a look.

While I didn't do much more than a very quick glance, I got the impression that the idea was to encode "blurriness" data into the bump map's MIP maps, as well as doing texture filtering that takes the amount of "blurriness" that should be there into account? If I'm reading it right, it sounds like an excellent idea, particularly since you only need two values to fully-define a normal map, there's lots of space left to use for other things.

[OpenGL guy]

And obviously you would absolutely need higher internal precision than is available in the 8-bit per value normal map to get the proper result.
You've lost me here: What does internal precision have to do with the input precision?

[Chalnoth]

You've lost me here: What does internal precision have to do with the input precision?

My basic idea was that the "bands" in the ATI car demo were due to lack of normal map precision. If internal precision is higher, and the pixel shaders do some sort of filtering on the normal map, then those bands should be gone.

[OpenGL guy]

My basic idea was that the "bands" in the ATI car demo were due to lack of normal map precision. If internal precision is higher, and the pixel shaders do some sort of filtering on the normal map, then those bands should be gone.
Maybe you could reduce the banding by using a "better" filter, but you are still going to be limited by the precision of the input data.