"Pure and Correct AA"

[Reverend]

A long, long time ago, I went into a kind of debate about "pure/correct AA" with a really smart guy (still at the same IHV as then). A number of things were discussed and debated, including the use of filters, which I'd objected to as having anything to do with AA (hence the thread title). The following comment from him acted as his final comment on the debate :

"pure and correct AA" blends together information (samples) from multiple pixel areas (rectangles). that is the right answer, not a hack or a blur. it's just math and science, not opinions.

I am still not convinced that is the right way to approach AA. Thoughts?

Sorry for starting another thread about AA. I guess you know where my priorities lie!

 

[MfA]

He was oversimplifying a bit to make the point short and sweet (to make it understandable to those who think of AA as blur I imagine, not the brightest lights) but yes, given a couple of "small" assumptions you can formulate a pure and correct way of doing AA.

You have to posit a footprint for the pixel samples and you have to posit the reconstruction filter you use for textures (ie. the magnification filter). After that infinite supersampling is the pure and correct way of doing what a real-life camera would do given the same scene ... approaching that as close as possible is the correct way of doing AA, and that does result in blending multiple samples together in one way or another. I've always wanted an actual benchmark which used objective quality metrics to compare images generated by the various super/multisampling and texture filtering combinations against one generated in a software renderer with very high supersampling ratio actually.

 

[silent_guy]

A long, long time ago, I went into a kind of debate about "pure/correct AA" with a really smart guy (still at the same IHV as then). A number of things were discussed and debated, including the use of filters, which I'd objected to as having anything to do with AA (hence the thread title). The following comment from him acted as his final comment on the debate :

I am still not convinced that is the right way to approach AA. Thoughts?

(Reviving rusty parts of memory)
Doesn't it have to do with this?

An ideal anti-aliasing filter is a filter that removes all frequencies above a certain frequency and leaves those below entirely intact. The impulse response of such a filter has the form of a sinc(x) function. This function doesn't have bounds. Translation: it spans an infinite amount of pixels. All other solutions are approximations of this ideal.

Does that sound about right?

Edit: Thinking about it some more: this is probably entirely impractical for a concrete implementation, so it only confirms your quote from the AA specialist. It's never a bad approach, though, to start out with a theoretical framework before descending to reality. At least it gives you a better understanding of where you're cutting corners.

 

[Ailuros]

OT: Dear Rev, that's too many AA related threads all at once to be just a coincidence; something is cooking in the background and I hope it's a good one

I've read many times recently that the next step is better filters; one less recent example would be that one:

http://forum.beyond3d.com/showthread.php?p=46390&highlight=filters#post46390

In a relevant debate I had myself, calling such a filter (such as in Quincunx) a "blur filter" was also being deemed as a too harsh comment. In any case I'd have to see something first in real time before I can say anything from my layman's perspective, but I never liked Quincunx for the specific reason that the added "blur" doesn't effect polygon edges/intersections only, but rather the entire scene.

If there are possibilities/sollutions being worked on that are times better than that, I guess I could tolerate some amount of "oversampling". For Quincunx - despite what many saw with it - I never personally had the feeling that I'm using anything more than 2xMSAA, but it might have been just me.

 

[Colourless]

The problem is you aren't sampling something that already exists. Texture antialiasing is not the problem. You could consider it mostly a solved problem. The edge of a triangle and conditionals in shaders (including alpha testing) have an infinitely high frequency (they are well, purely digital things, either there or not). You can't resolve that without sampling at a higher resoution, or including metadata into the framebuffer about what is in the pixel and the shape of the thing in that pixel and doing a fancy post process pass.

You of course have problems with high frequency geometry (such as horizontal venetian blinds that cause all sorts of problems on TV broadcasts) where on a frame to frame basis you can get completely different appearance due to insufficient sampling resolution. I guess not including such geometry in the first place though could be considered a form of low pass pre filtering.

 

[Simon F]

My recollection of this is a bit rusty but, FWIW...

(Reviving rusty parts of memory)
Doesn't it have to do with this?

Does that sound about right?

It would be if the display system/monitor used sinc functions to reconstruct the image, but I don't believe they do, so you have to adjust your filtering model somewhat.

 

[bloodbob]

My understanding of a pure and correct AA would be one that uses analytical functions for every object on a pixel and integrates across those pixels to come up with the true average value for that pixel. However it is a pain trying to art with formulas you can integrate analytically and perhaps even impossible in some cases.

 

[Rys]

Doesn't Nyquist state that supersampling is the only way to completely solve frequency aliasing? Filtering can never lead to perfection in that respect, according to my understanding of what that sampling theorem states at least.

 

[pcchen]

Sampling theory requires the filtering of higher frequencies (which is, more than 1/2 of your sampling frequency) to avoid aliasing. So, in the case of 3D graphics, you need to generate the image in infinite resolution and filtered with a low pass filter. Of course, it's impossible to use infinite resolution so generally we use a limited supersampling to mimic the effect.

As bloodbob said, it's also possible to analytically do the low pass filter. However, it's extremely costly to do so, if at all possible.

 

[Jawed]

I dare say it's trivially easy to compare a 320x240 image without AA and a 2048x1536 image (also without AA) sized-down to 320x240, to see what the "ideal" AA on the 320x240 image should look like. That's 6.4x over-sampling.

You can even play with different image-sizing algorithms on the 2048x1536 image, nearest-neighbour, bilinear, bicubic, sinc-approximation, etc.

Just make sure to do all the sizing/filtering in linear space (if you use Photoshop, this is a big deal, since it doesn't re-size images in linear space - you have to convert to 16-bit then convert to gamma 1.0 before re-sizing).

Once you've got your "ideal" 320x240 image, then you can argue about what GPU AA modes come closest... Though I think there'll have to be distinctions drawn between polygon edge-AA and texture-filtering.

Jawed

 

[Xmas]

Doesn't Nyquist state that supersampling is the only way to completely solve frequency aliasing? Filtering can never lead to perfection in that respect, according to my understanding of what that sampling theorem states at least.

In theory you'd have to use an infinite amount of samples for those infinite frequency edges, there's no question about that. But this thread is about the "correct" downsampling method.

While I formerly believed that every sample should "belong" to one pixel and only be used to determine the final color of that pixel, I've changed my opinion on that point. Yes, a sample may (and even should, usually) influence more than one pixel.
But I don't think a sinc filter is ideal either. It's not just about removing the higher frequencies while downsampling. What we have to try is make the result as pleasant to the eye as possible. And every person will judge that differently, depending on the content and even on the display hardware.
A "good" downsample filter in my opinion should follow those three rules:
- it operates in linear color space
- every sample is equal thus the sum of weights for each sample should be constant (1/N)
- sample position information must be retained in the final image, thus the footprint of a sample has to be at least one pixel in size. That is especially important for very bright thin lines when using HDR.

 

[Simon F]

Doesn't Nyquist state that supersampling is the only way to completely solve frequency aliasing? Filtering can never lead to perfection in that respect, according to my understanding of what that sampling theorem states at least.

You're confusing post-filtering, which can't solve the problem, with pre-filtering which can.

 

[silent_guy]

My recollection of this is a bit rusty but, FWIW...

It would be if the display system/monitor used sinc functions to reconstruct the image, but I don't believe they do, so you have to adjust your filtering model somewhat.

I wasn't even thinking that far.
But I don't think that you're correct: the sinc(x) function is what you need if your display shows the pixels as a series of dirac impulses (not a sinc). Basically, if you're going to operate on the data in memory without displaying it, a sinc impulse response would be ideal.

But you are correct in that you'd need some kind of equalizing filter to adjust for the display characteristics when you're sending it to the monitor.

(All this is still assuming that there is a continuous source representation of the image, which, in reality, doesn't exist of course.)

 

[Simon F]

I wasn't even thinking that far.
But I don't think that you're correct: the sinc(x) function is what you need if your display shows the pixels as a series of dirac impulses (not a sinc). Basically, if you're going to operate on the data in memory without displaying it, a sinc impulse response would be ideal.

Surely the dirac impulses relate to taking the samples of the original signal, not the reconstruction thereof.

 

[silent_guy]

Surely the dirac impulses relate to taking the samples of the original signal, not the reconstruction thereof.

They relate to taking samples of the filtered signal, not the original one.

 

[Simon F]

They relate to taking samples of the filtered signal, not the original one.

Only if you want no aliasing in the result.

 

[Geo]

Actually, maybe Rev could ask JC and TS what they think about the great "shader aliasing" debate, and how they feel about being left alone to deal with it as if somehow it was their fault/problem alone. As the leading engine makers, their comments might be interesting on what they feel would be the best way forward for the industry that would be both robust and friendly. Is it an API issue? An IHV issue? An engine-maker issue? An individual ISV issue? Some combination? Where are the relative responsibilities, and who should rightfully have the onus on them to lead the way forward? This is a question that has been bothering me for the last year, as I sense a lot of finger pointing going on, and a severe lack of leadership and cooperation to solve a growing problem that's only going to get worse. . .

 

[Davros]

"I dare say it's trivially easy to compare a 320x240 image without AA and a 2048x1536 image (also without AA) sized-down to 320x240, to see what the "ideal" AA on the 320x240 image should look like. That's 6.4x over-sampling."

This any good ?
done using paintshop pro 320x240 original size 1280x960 (monitor only does 1280x1024) not converted to 16bit
(ps: couldnt do 320x240 game doesnt support it)

1 = Bicubic
2 = Bilinear
3= Pixel Resize
4= Weighted Adverage









original 1280x960 no AA (had to jpeg it)

 

[anjulpa]

I was wondering.. Could an inverse point spread function (if one exists), or the information about the point spread function of the objects in a scene (and the camera/imaging system characteristics) be used to estimate what amount oversampling is needed for a good anti-aliased image?

 

[Humus]

Assume we have the following pixel:

"Correct AA" would then be 0.1 * red + 0.55 * blue + 0.45 * green. This of course assumes that the pixel is square and pixels are tightly packed and that the response curve of the monitor is linear. If we assume the intersecting primitives are not just single color as in the example you'd have to evaluate an integral over the area. That's hard to impossible to do for most cases. The closest thing you can do in practice in the general case is to just supersample at insane density. Of course though, the added image quality of doing this is nowhere close to proportional to the cost. Even assuming it would be cheap it's highly questionable if attempting to do it "mathematically correctly" is useful in practice. As soon as any assumption break your "mathematically correct" break as well. For instance, there are loads of monitors out there with a response curve that's a power like 1.8 up to maybe 2.6. sRGB assumes 2.2 and that's reasonable to use in practice, but no monitor out there exactly matches this, particularly not after the user has tweaked brightness and contrast settings.

IMHO, speaking of edge antialiasing alone, evaluating the shader more than once is not useful. Adding more coverage samples is useful to a certain limit. I don't think going over 16 will really add any visible improvement. If you're at maybe 8 samples or above I think matching the response curve of the monitor is more important than more samples.