msaa + alpha to mask

[DemoCoder]

This technique basically boils down to dithering alpha transparent texture edges. My question is, isn't this bound to produce horrible artifacting in some cases? Seems like you could end up with flickering stipple patterns on the edges of objects.


It seems like a nice addition to the next-gen card repetoire kinda like brilinear et al, but it doesn't seem to be a general purpose solution to the problem of aliasing in transparent textures, or under the function of fragment-kill or shader aliasing when using MSAA.

Also, GL_SAMPLE_ALPHA_TO_MASK has been in OpenGL for a long time. Have cards supported this functionality prior to G70/R520? And if so, why has no one ever used it?

 

[Xmas]

This technique basically boils down to dithering alpha transparent texture edges. My question is, isn't this bound to produce horrible artifacting in some cases? Seems like you could end up with flickering stipple patterns on the edges of objects.

It could be compared to dithering, but at the sub-pixel level.
Yes, you can get artifacts in some cases. The most obvious one being banding when you get close to alpha edges and the texture is magnified. Alpha to coverage does not result in a sharp edge with texture magnification, unlike alpha test. You get a blurry "edge" like with alpha blending, but only with 4 (or # of MS samples) intermediate steps.
Then you can get some weird effects with two overlapping layers of transparency, where parts seemingly have the wrong level of transparency. I don't think this is much of an issue, though.
I'm not sure what kind of flickering stipple pattern you mean.

It seems like a nice addition to the next-gen card repetoire kinda like brilinear et al, but it doesn't seem to be a general purpose solution to the problem of aliasing in transparent textures, or under the function of fragment-kill or shader aliasing when using MSAA.

It seems like NVidia is very aware of that. That's why they offer a second, higher quality setting (using sparse grid supersampling).

Also, GL_SAMPLE_ALPHA_TO_MASK has been in OpenGL for a long time. Have cards supported this functionality prior to G70/R520? And if so, why has no one ever used it?

It's supported since NV20. I know, because I did use it. Though only for my own experiments.

My program doesn't work correctly on R300 though. I don't know why, I simply don't get multisampling at all though I successfully choose a multisampled pixel format.

NV20 however has some weird behavior which makes it more useful for screen door transparency, but less useful for edge AA - it dithers not only at the subpixel level, but at the pixel level, too. This way you get 16 intermediate steps with only 4 samples per pixel, but well, except for the 25%, 50%, 75% and 100% opacity steps, the dithering is visible.
I guess they changed this with G70.

 

[Ailuros]

It seems like NVidia is very aware of that. That's why they offer a second, higher quality setting (using sparse grid supersampling).

Across the entire scene

 

[DemoCoder]

If so, I've been sold. I have been waiting for something like this for a long time! Bout time too, since next-gen cards have enough heavy lifting power to run SS on older games at ok resolutions, especially if 2x sparse supersampling mode is available.

 

[Xmas]

Across the entire scene

In this context, no.
Only for objects that have alpha test enabled.

 

[Ostsol]

Damn. . . Spent half an hour searching for support on the R300 before realizing that GL_SAMPLE_ALPHA_TO_COVERAGE_ARB == GL_SAMPLE_ALPHA_TO_MASK_EXT. So, yeah, it's supposedly been supported on ATI cards since the R300. Whether it actually works or not is another matter entirely. . .

 

[DemoCoder]

Across the entire scene

In this context, no.
Only for objects that have alpha test enabled.

Is there specific HW support, or is there some driver trickery going on?

 

[Humus]

Whether it actually works or not is another matter entirely. . .

It works, I just tried it. I guess I could make a demo of it ...

 

[ERP]

Across the entire scene

In this context, no.
Only for objects that have alpha test enabled.

Is there specific HW support, or is there some driver trickery going on?

This was possible on even NV2x level cards, but I was never clear what the cost of changing the AA mode was. It's possible it was a slow operation (synced the pipe).

Purely speculation, but if they've done anything they may just have made the state change cheap.

 

[OpenGL guy]

Also, GL_SAMPLE_ALPHA_TO_MASK has been in OpenGL for a long time. Have cards supported this functionality prior to G70/R520? And if so, why has no one ever used it?

All Radeon products from the 9500 and up have such functionality. I don't know about older products.

 

[Ailuros]

Across the entire scene

In this context, no.
Only for objects that have alpha test enabled.

Wonderful Thanks.

 

[Simon F]

This technique basically boils down to dithering alpha transparent texture edges. My question is, isn't this bound to produce horrible artifacting in some cases? Seems like you could end up with flickering stipple patterns on the edges of objects.

The technique is known as screen door transparency and has been around for "donkeys' years". Here's a recent (1998) paper looking (I think) at using better patterns.

Also, GL_SAMPLE_ALPHA_TO_MASK has been in OpenGL for a long time. Have cards supported this functionality prior to G70/R520? And if so, why has no one ever used it?

As I said, it has been around for quite some time on older workstation HW (I'd have to go searching through my collection of Siggraphs (or on the ACM website) for actual devices), but the trouble is that it looks very dithered.

 

[Xmas]

Whether it actually works or not is another matter entirely. . .

It works, I just tried it. I guess I could make a demo of it ...

Hm, could you tell me what your init code looks like? I use wglChoosePixelFormatARB to get a pixel format that is multisampled (and verify that with wglGetPixelFormatAttribivARB), but when I use that with SetPixelFormat, i don't get MS at all. But the same code worked on my GeForce3.

 

[Ostsol]

Here's the NeHe tutorial. Not the greatest, but it shows you what need to be done.

 

[Xmas]

NV20 however has some weird behavior which makes it more useful for screen door transparency, but less useful for edge AA - it dithers not only at the subpixel level, but at the pixel level, too. This way you get 16 intermediate steps with only 4 samples per pixel, but well, except for the 25%, 50%, 75% and 100% opacity steps, the dithering is visible.
I guess they changed this with G70.

I was wrong...
http://www.ixbt.com/video2/images/g70/aa-aa4x-taa-ms.jpg

Still it looks quite good in this case, and for some reason TSAA doesn't work here. I'm not really sure why TMAA doesn't work well with HL2 chainlink fences, but I'm suspecting something...



Ostsol, thanks. I guess I'll do some tracking what I'm doing different, and why it works on NVidia cards.

 

[DemoCoder]

Ok, the riddle of why alpha-to-mask hasn't been used for transparency MSAA in the past has been answered: because it sucks. (e.g. it doesn't do much)

Transparency supersampling (apply SS only to alpha-test/alphatextures) looks frigging amazing. The different is immediate and startling. To me, this is the single best addition to a card since MSAA and AF were introduced.

I have been incredibly annoyed at aliasing in games like HL2 and BF2, and most outdoor games that make use of alpha textures for shrubbery, wires, fences, railings, etc. You'd expect games like HL2 or BF2 to replace detail like railings and fences with real geometry, but they don't, and it seems alphatextures are here to stay for awhile.

I was really taken aback by the 8xS + TSAA shots on HardOCP tho. Simply amazing. I'd love to see those HL2 shots in motion.

The next step is to apply TSAA to selectively antalias shaders.

Do I dare say that the G70 seems to have finally bested ATI IQ?

 

[Ostsol]

Ok, the riddle of why alpha-to-mask hasn't been used for transparency MSAA in the past has been answered: because it sucks. (e.g. it doesn't do much)

Yeah, I just found that out. It really didn't do anything noticable at all unless you're analyzing the image really closely. The tiny bit that it does do is also in an ugly stippled pattern.

 

[Xmas]

Ok, the riddle of why alpha-to-mask hasn't been used for transparency MSAA in the past has been answered: because it sucks. (e.g. it doesn't do much)

I'm not sure that's true. All I've seen yet are HL2 chainlink fence shots where it doesn't do much, and this FarCry shot from the ixbt review where it actually does a lot (it's dithered, but still far better than no AA, IMO).
I suspect the reason it doesn't do much in HL2 is that it doesn't disable/adjust the alpha test, so the intermediate steps simply get killed by the alpha test.

Transparency supersampling (apply SS only to alpha-test/alphatextures) looks frigging amazing. The different is immediate and startling. To me, this is the single best addition to a card since MSAA and AF were introduced.

Agreed.

The next step is to apply TSAA to selectively antalias shaders.

But which selection criteria would that be? I guess that would have to be part of a profile.

 

[DemoCoder]

I see many ways it can be done.

1) heuristically. Some operations are known to cause aliasing. Pow(), step functions, branching. You can afford to be conservative in this, since if you get it wrong (false positive), you just apply a little more supersampling than neccessary, but the image looks good.

2) dynamic runtime profiling. Have the driver, on a random basis, for each shader, render some quads of varying sizes, perhaps making use of gradient instructions. Analyzing the resulting data for aliasing. Over time, as you play the game, the driver will more and more learn which shader/pipeline state needs TSAA, and keep a cache. This is a technique to automatically "learn" game profiles. This technique is way too complicated tho, when a simpler technique which will work for most consumers exists:


3) Driver downloads profiles from repository at NV. NV labs run internal profiling on games and determine which shaders/pipeline state need TSAA. The driver uses these profiles to detect when to switch on TSAA.

4) In my dreams, the driver compiler would contain a symbolic math library. The driver performs an analytic FFT on the shader. J/K

 

[Clootie]

All I've seen yet are HL2 chainlink fence shots where it doesn't do much, and this FarCry shot from the ixbt review where it actually does a lot (it's dithered, but still far better than no AA, IMO).

But have you looked at this page?