Would it be wrong to say AF is the AA for textures?

[KimB]

Would it be wrong to say AF is the AA for textures?

SSAA + AF and MSAA + AF which would look better?

Texture filtering is anti-aliasing for textures. This doesn't just apply to anisotropic filtering, but to simple bilinear and trilinear as well. The intent is the same: reduce aliasing. But the algorithms are very different, which means the actual effects are different.

Texture filtering is a limited method of anti-aliasing that specifically uses the paradigm of displaying color textures. Through the examination of this display algorithm, much more effective anti-aliasing techniques have been devised other than the "catch all" supersampling method. So, texture filtering will always have better image quality/performance ratio than basic supersampling FSAA. When combined with MSAA for anti-aliasing the edges, you get a much better image quality/performance ratio overall.

But what are the drawbacks? Today's texture filtering depends on certain assumptions. Many rendering algorithms that are available today violate those assumptions. For example, one cannot filter a normal map in the way one filters a color map. These algorithms also will not cover per-pixel branching (such as the compare instructions in DX8 and DX9, or an alpha test).

So, I think that first of all, anisotropic filtering will become less and less useful as time goes on. As pixel shaders become more advanced and complex, there will be less and less room for traditional texture filtering, as other forms of aliasing will crop up. We will either need to research efficient means of filtering each and every type of texture that is used, or simply head back to using the "catch-all" AA method of supersampling. I hope that both become available to future game programmers.

Here's what I would consider ideal for a future architecture:

Have multisampling AA selected by the program. Then, for a specific surface type that produces aliasing in a way that is not easily solved through more programming, the program decides that that surface should use supersampling instead of multisampling (at the same sample frequency). As far as I can tell, this should produce no major problems such as memory reallocation or abnormal pipeline stalls.

In today's software, for example, it would be nice to select SSAA for only those surfaces that use an alpha test. Some more recent programs may choose to select SSAA for surfaces that use some sort of bump map.

 

[MfA]

Well the brute force way is always an option, I hope in the short term developers arent too lazy to look at special case solutions though ... for instance for bumpmapping you can use roughness maps which describe the subtexel distribution of normal vectors for shading, so mipmapping wont destroy the effect with minification.

 

[KimB]

Well the brute force way is always an option, I hope in the short term developers arent too lazy to look at special case solutions though ... for instance for bumpmapping you can use roughness maps which describe the subtexel distribution of normal vectors for shading, so mipmapping wont destroy the effect with minification.

I think the best way to tackle this would be to start building standardized libraries for a high-level shader language. This way, turning on the various texture filtering techniques for a number of different pixel shader algorithms would be as simple as calling the appropriate function.

I'd say that we're at the point where advancements in software are becoming more important than advancements in hardware to keep 3D graphics moving forward.

 

[OpenGL guy]

Have multisampling AA selected by the program. Then, for a specific surface type that produces aliasing in a way that is not easily solved through more programming, the program decides that that surface should use supersampling instead of multisampling (at the same sample frequency). As far as I can tell, this should produce no major problems such as memory reallocation or abnormal pipeline stalls.

Sure it would. The one pixel that requires SSAA would require much longer to render than other pixels that do not. In a quad based architecture (as most are these days) that would definitely impact performance. Also, you're assuming that these SSAA pixels would mesh well with pixels that were not SSAA'ed. A false premise, I believe.

In today's software, for example, it would be nice to select SSAA for only those surfaces that use an alpha test. Some more recent programs may choose to select SSAA for surfaces that use some sort of bump map.

And just how would you SSAA only polygons rendered with alpha test? Where do you generate the extra color samples?

 

[MfA]

In a quad based architecture a X times multisampling shader would be rendering 4 pixel-quads with 4*X samples, and a supersampling shader would be rendering 4 sample quads ... so yeah it would be X times slower. Wether it impacts performance is quite besides the point though, wether the tradeoff is worth it is upto the developer.

As for where the colour samples come from ... where they always come from with supersampling, by sampling and shading them.

 

[OpenGL guy]

In a quad based architecture a X times multisampling shader would be rendering 4 pixel-quads with 4*X samples, and a supersampling shader would be rendering 4 sample quads ... so yeah it would be X times slower. Wether it impacts performance is quite besides the point though, wether the tradeoff is worth it is upto the developer.

As for where the colour samples come from ... where they always come from with supersampling, by sampling and shading them.

But if your pipeline is currently processing triangles with multisampling, something very significant has to change in order to start doing supersampling. That's what I meant by "where do the extra samples come from".

 

[MfA]

For the shader it really doesnt matter much, it just needs to stop multisampling ... for the rest its an ordinary quad. That is not a big change in the hardware.

Hell if the sampling positions can dynamically change per quad you could just set a subpel sample position to the center of the "pixel" and drop the rest of the generated samples for that "pixel" in the backend. So you would only need an extra flag per quad and couple of small changes to setup and the backend.

 

[Simon F]

I think the question that needs to be asked is "what is AA?"

Well said Rev!

More than 16x SSAA? I've seen stochastic AA images rendered with and without AF using 16 R200 cores and you can easily see the difference.

The thing is that SSAA only removes aliasing. It can't do anything to increase the level of detail of textures at further distances.

If you remove the aliasing (and nothing else) then that is what you'd get.

also, ssaa doesn't really do anything for mipmap transitions.

nor does AF, they both will jsut push the mip transitions back (assuming mip lod is set correctly for SSAA)

Both of you are confusing particular implementations with the underlying concepts.

AFAIK aliasing includes things like pixel popping along with jaggies among other things which I have forgotten.

Aliasing is a distortion caused by a system limitation, is it not?

Aliasing is caused when you try to sample (i.e take measurements at instants in time or space of) a particular signal at an inadequate frequency.

For example, you work in an office from 9-5, but take a 15 minute coffee break at 11am every day. Your lazy colleague, OTOH, arrives in the office at 10:30 and then leaves at 4. Every day, your boss, who does not understand signal theory, walks around the office every day at 11:05. He never sees you working but "rewards" your lazy colleague with a whopping a pay rise.


Signal theory says that if you want to sample a signal that contains a maximum frequency of F (and that can be in terms of time or pixels), then you must sample at a rate > 2F. If you don't, those high frequencies will re-appear as low frequencies, i.e., they come back with a false identity, an alias.

The problem is that poly edges have infinite frequencies (but those do drop off in importance fairly quickly). Similarly with textures extending off to the distance.

 

[Xmas]

But if your pipeline is currently processing triangles with multisampling, something very significant has to change in order to start doing supersampling. That's what I meant by "where do the extra samples come from".

Hm, for the shader pipeline there should be no difference. The rasterizer unit has to switch to a different mode, and the ROPs must work differently, too. But I don't think that's too complex to be worth being implemented in hardware.

Of course an application can choose to do supersampling by rendering an object several times with multisample masking, but that's far less efficient.

 

[OpenGL guy]

But if your pipeline is currently processing triangles with multisampling, something very significant has to change in order to start doing supersampling. That's what I meant by "where do the extra samples come from".

Hm, for the shader pipeline there should be no difference. The rasterizer unit has to switch to a different mode, and the ROPs must work differently, too. But I don't think that's too complex to be worth being implemented in hardware.

And just how does the hardware know that it needs to change to this supersample mode?

 

[Xmas]

And just how does the hardware know that it needs to change to this supersample mode?

Are you referring to the lack of API support for such use or some hardware-related issues?

As long as there is no API support, a naïve driver tweak could enable supersampling if alpha test but no blending is activated. That would at least produce antialiased trees and ladders in several games. Especially some older games where performance isn't an issue would profit from that.

But I'd also be glad if MS changes the way D3D offers AA support in the next DX version, because I think the current interface with "quality levels" is crap.

 

[Tridam]

OpenGL Guy -> Basically, supersampling is calculating the color for all the samples and multisampling is calculating the color once and put the result in every sample. I think that many attributes are attached to every triangle and that some of them could be transmitted to the samples. So why can't you just add one more triangle attribute which would mean "Calculate the color for all the samples coming from this triangle" and which will become a sample attribute saying "Calculate the color for me" ?

Of course it would uses fillrate and your color compression system will be useless for these samples. But the power of the new GPU will need to have a utility

 

[OpenGL guy]

OpenGL Guy -> Basically, supersampling is calculating the color for all the samples and multisampling is calculating the color once and put the result in every sample. I think that many attributes are attached to every triangle and that some of them could be transmitted to the samples. So why can't you just add one more triangle attribute which would mean "Calculate the color for all the samples coming from this triangle" and which will become a sample attribute saying "Calculate the color for me" ?

I know what supersampling is. What I am questioning is how easy you people think it is to switch from multisampling to supersampling.

Of course it would uses fillrate and your color compression system will be useless for these samples. But the power of the new GPU will need to have a utility

There's far more to it than that.

 

[Tridam]

OpenGL Guy -> Basically, supersampling is calculating the color for all the samples and multisampling is calculating the color once and put the result in every sample. I think that many attributes are attached to every triangle and that some of them could be transmitted to the samples. So why can't you just add one more triangle attribute which would mean "Calculate the color for all the samples coming from this triangle" and which will become a sample attribute saying "Calculate the color for me" ?

I know what supersampling is. What I am questioning is how easy you people think it is to switch from multisampling to supersampling.

Hehe of course, I know that you know what SS is. I just wanted to describe the 2 AA in a simple manner so that you can say where this doesn't make sense.

 

[MfA]

The shading itself is the bulk of the hardware so lets ignore the rest ...

How does having a flag per quad which dynamically indicates which mode to use make hardware more complex? If your hardware can really use per pixel dynamic sampling positions both for supersampling and multisampling then there is no real overhead needed to support the dynamic switching.

Would having the hardware be able to grok arbitrary sampling positions for the 4 samples in a quad even during multisampling need more hardware? No, because of the orthogonality with supersampling its probably easiest to just do this even during multisampling and pass it uniform grid positions for the 4 samples.

Would having the hardware be able to only do a single centered sub-sample per sample in the quad during multisampling need more hardware? Obviously no if it can support arbitrary positions for the subsample positions anway (if necessary you could just drop all but 1 afterward).

If you combine these 2 you have the shading engine being perfectly able at any time to generate a 4 sample quad for supersampling instead of a 4*X sample multisampled quad, without any real overhead.

 

[OpenGL guy]

The shading itself is the bulk of the hardware so lets ignore the rest ...

How does having a flag per quad which dynamically indicates which mode to use make hardware more complex? If your hardware can really use per pixel dynamic sampling positions both for supersampling and multisampling then there is no real overhead needed to support the dynamic switching.

What determines when the flag is set?

Would having the hardware be able to grok arbitrary sampling positions for the 4 samples in a quad even during multisampling need more hardware? No, because of the orthogonality with supersampling its probably easiest to just do this even during multisampling and pass it uniform grid positions for the 4 samples.

In multisampling, those four samples go directly from the rasterizer to the depth unit, the pixel shader only gets a single sample.

Would having the hardware be able to only do a single centered sub-sample per sample in the quad during multisampling need more hardware? Obviously no if it can support arbitrary positions for the subsample positions anway (if necessary you could just drop all but 1 afterward).

If you combine these 2 you have the shading engine being perfectly able at any time to generate a 4 sample quad for supersampling instead of a 4*X sample multisampled quad, without any real overhead.

No you don't. That 4 sample SSAA quad only gives you a single 4x SSAA pixel, what about the other three pixels in your quad? Also, this requires significant changes to the rasterizer and depth unit. Why? Because the rasterizer has to go from computing the samples for four pixels to the four samples for a single pixel and the output from four pixels has to be combined into a single pixel's results (color and depth for each subsample). What about supporting other SSAA modes?

You folks talk about all of this stuff as though it's trivial, when it's not remotely so. Try to make a software rasterizer that does all of the above... Then imagine making a HW implementation that can be pipelined for peformance.

 

[KimB]

Have multisampling AA selected by the program. Then, for a specific surface type that produces aliasing in a way that is not easily solved through more programming, the program decides that that surface should use supersampling instead of multisampling (at the same sample frequency). As far as I can tell, this should produce no major problems such as memory reallocation or abnormal pipeline stalls.

Sure it would. The one pixel that requires SSAA would require much longer to render than other pixels that do not. In a quad based architecture (as most are these days) that would definitely impact performance. Also, you're assuming that these SSAA pixels would mesh well with pixels that were not SSAA'ed. A false premise, I believe.

I'm not sure it would be significant. I would tend to think that typically the entire quad (or multiple quads, in the R3xx architecture) would all be working under the same rules at any one time: if one pixel in the pipeline is using SSAA, then they all will be. While there would definitely be waste in doing it this way, whether or not to use SSAA would be selected on a per-primitive basis, not a per-pixel basis. Beyond the obvious performance hit, there would be a state change, but I see no reason why it needs to be any worse than that (in fact, I think it would make sense for the state change hit to be relatively small compared to the SS hit)

In today's software, for example, it would be nice to select SSAA for only those surfaces that use an alpha test. Some more recent programs may choose to select SSAA for surfaces that use some sort of bump map.

And just how would you SSAA only polygons rendered with alpha test? Where do you generate the extra color samples?

if (alpha_test=on) then SSAA=on

I'm not sure it needs to be more complex than that. After all, both SSAA and MSAA store the same amount of color and z-buffer information. While compression will break down, MSAA + color/z-buffer compression algorithms are already built to handle compression breakdown. It's necessary to handle compression breakdown to properly render triangle edges.

What I would do is have the main "switch" placed before the pixel pipeline, in the part of the engine that is calculating which pixels to send to the pixel pipelines (triangle setup engine? I think I heard another name a bit ago, but I forgot it ). As long as the pixel pipelines know the right ways to handle mixing of MSAA and SSAA, I anticipate no significant problems.

 

[K.I.L.E.R]

Chalnoth, if it was that easy then even I would be doing it.

 

[MfA]

What determines when the flag is set?

The driver, it is part of the shader.

In multisampling, those four samples go directly from the rasterizer to the depth unit, the pixel shader only gets a single sample.

For it to be quad based it has to shade 4 samples, regardless of the number of subsamples used for multisampling ... multisampling is really pretty much irrelevant for the shader. It is only relevant to us because we need a centered Z sample for the supersampling mode.

No you don't. That 4 sample SSAA quad only gives you a single 4x SSAA pixel, what about the other three pixels in your quad?

Why should the shader care about pixels? It gets a set of parameters for 4 samples, it delivers 4 shaded samples. Wether a sample is a pixel or a subpixel sample is irrelevant to the shader. The other three pixels worth of samples come as later quads, that's the job of the setup engine.

Also, this requires significant changes to the rasterizer and depth unit. Why? Because the rasterizer has to go from computing the samples for four pixels to the four samples for a single pixel and the output from four pixels has to be combined into a single pixel's results (color and depth for each subsample). What about supporting other SSAA modes?

That is small fry.

You folks talk about all of this stuff as though it's trivial, when it's not remotely so. Try to make a software rasterizer that does all of the above... Then imagine making a HW implementation that can be pipelined for peformance.

Work is amortized soon enough, the added area is what counts.

 

[OpenGL guy]

Here's why this stuff doesn't work well. Imagine doing texkill instead of alpha test. End result is the same if you use texkill to kill texels based on alpha value. Now, if you say the driver should enable SSAA in this case, then I say, "What if no pixels are killed? You're incurring a huge cost for no reason!" Of course, this applies to alpha test as well.

What I thought you folks were talking about was a dynamic (i.e. hardware based) MSAA -> SSAA transition. However, even if you assume it's the driver that makes the switch from MSAA to SSAA and back, I still think you are making some very large assumptions. That doesn't mean the problems can't be solved, but there's a lot more involved than just some driver tweaks.