Type of AA most beneficial for 360/PS3

[Rolf N]

I won't mind Quincunx AA. Is it almost free without much complication?

Its cost is equal to 2xMSAA, because it builds on 2xMSAA, but it destroys quality. It destroys texture detail and edge resolution. It's shit.

 

[nAo]

Its cost is equal to 2xMSAA, because it builds on 2xMSAA, but it destroys quality. It destroys texture detail and edge resolution. It's shit.

There are already titles using is and ppl think are using 4X AA (no, I'm not talking about HS, we use 4X rotated grid MSAA), so can't be so bad.

 

[Shifty Geezer]

Are these games where people also complain about the texture quality and say PS3 has lowres textures due to memory limits?

 

[nAo]

Are these games where people also complain about the texture quality and say PS3 has lowres textures due to memory limits?

To be honest I don't know, I don't keep track of these 'details'!

 

[Rolf N]

To be honest I don't know, I don't keep track of these 'details'!

Whatever it is (as you don't seem willing to tell), if it's still in production go fire the team an email and tell them it's shit and they should just use 2xMSAA. The stuff that Quincunx does in addition to its AA component has nothing to do with AA. It's a purely destructive process that reduces information, which is the exact opposite of what AA mechanisms do (they integrate additional information).

 

[nAo]

I'm aware of this since it partially works as a low pass filter, nonetheless it can be an interesting trade-off in some games, especially if you factor in the extra flexibility you have on a closed platform..

 

[Rolf N]

I'm aware of this since it partially works as a low pass filter, nonetheless it can be an interesting trade-off in some games, especially if you factor in the extra flexibility you have on a closed platform..

I could see a use as a quick-and-dirty DOF solution if it could actually be applied selectively. I don't think it can do that however (on PC hardware anyway), and because of that I must insist that in this world there is no place and no reason to exist for Quincunx.

 

[Xmas]

Quincunx uses too few samples with poor placement (wrt the filter kernel), but the general idea of not using a box filter is right.

 

[Simon F]

Whatever it is (as you don't seem willing to tell), if it's still in production go fire the team an email and tell them it's shit and they should just use 2xMSAA. The stuff that Quincunx does in addition to its AA component has nothing to do with AA. It's a purely destructive process that reduces information, which is the exact opposite of what AA mechanisms do (they integrate additional information).

If you use that reasoning, moving all the original pixels to random positions would be just as correct.

Quincunx, because it uses a tent filter of 5 samples, should produce better results than a box filter with just 2 samples. Having said this, a tent filter will begin to attenuate some frequencies you would like to keep a bit more than a box filter but it does a much better job of removing illegally high frequencies that you need to eliminate. Of course, with only 2 MSAA to start with, neither filter is going to produce outstanding results.

As for "destroying" information, I doubt a tent filter destroys any more information than the box filter. I don't have time to prove that, but it seems likely to me.

 

[AlNom]

Is it difficult to extend Quincunx to more samples?

 

[Mmmkay]

Whelp, I was bored and downright curious as to how Quincunx affects image quality. Having an nVidia graphics card, and arming myself with 3dMark03 I went to work:

This is my reference image (everything was rendered at 1024x768 with default settings, having AA forced via the control panel. A section was then cropped, and magnified nearest neighbour 200%):
(*)http://photos.digitalrave.net/images/92200x.png

Difference images are in comparison to the reference 0x screenshot:

The file size difference alone should tell you how much Quincunx is modifying the image (17k vs 43k). It certainly degrades texture quality, but is it really perceptible at native resolutions? I understand the principle that AA absolutely should not interfere with texture quality, I'm just wondering if the impact it has is really more of a theoretical criticism.

The original 1024x768 images are here if anyone wants to play around with them (warning 600kB images):
0x - http://photos.digitalrave.net/viewer.php?id=73390xoriginal.png
2x - http://photos.digitalrave.net/viewer.php?id=37682xoriginal.png
2xQ - http://photos.digitalrave.net/viewer.php?id=20312xQoriginal.png
4x - http://photos.digitalrave.net/viewer.php?id=92754xoriginal.png

 

[rounin]

I don't see image degradation myself, but I certainly don't have good eyes to see the difference. Given these recent examples, I think perhaps Quincunx is a good thing for PS3 games, since the system does not seem to lend itself to 4xAA. I think the degradation should not be too noticeable on a bigscreen TV for a person sitting a few feet away.

From what nAo has said, maybe we can deduce the games that use Quincunx from the list of games we think uses 4xAA on PS3 right now. I'm thinking of Resistance : Fall of Man, but thats all I can come up with now. I would also not be surprised if some Xbox games used this method too, since it uses nVidia hardware. Is Quincunx only available for nVidia hardware?

 

[Shifty Geezer]

Whelp, I was bored and downright curious as to how Quincunx affects image quality. Having an nVidia graphics card, and arming myself with 3dMark03 I went to work:

Nice idea, but it really needs a comparison with straight 2xMSAA as that's what it replaces. I've had a quick look for comparisons, but the pics I've seen seem to demonstrate very little AA from Quincunx! I've an ATi 9600 now so can't test it myself.

If Quincunx does degrade texture sharpness, it can still have a use (assuming it's edge AA is an improvement over 2xAA, which I haven't seen demonstrated yet) in some titles where a soft look is okay. Something like LocoRoco PS3 without lots of sharp textures. Even RnC. Not every game wants the sharpness of highres textures to fit it's art direction.

 

[Shifty Geezer]

I don't see image degradation myself, but I certainly don't have good eyes to see the difference.

It's not a very texture-heavy example pic Mmmkay's chosen (a nice chequered pattern anywhere?), but if you look at the close-ups, you can see, specially obvious on the darker pixels, blurring of the texture in Quincunx. If you load the pics into a pic viewer/editor and switch between them, it's clear. 4x has no affect on the texture.

 

[Mmmkay]

It was just a quick comparison really. That being the default snapshot frame 3dMark offers, etc. Here's the 2x and difference images:

Original
2x - http://photos.digitalrave.net/viewer.php?id=37682xoriginal.png

I didn't include 2x because I thought 2xQ was seen as a low cost alternative to 4x. If it didn't filter edges better than 2x then it wouldn't make sense using it. Anyway, the point I think still stands, the texture softening it seems to cause isn't really something I would perceivably notice. Not something that would make me question texture resolution

 

[Shifty Geezer]

Depending on the texture type and resolution, it could have a marked effect. On my monitor though, the results are much better 'gamma corrected' AA samples. The 2x smoothing is too dark to be noticeable. The 2xQ sits about halfway in visible intensity between light and dark. Thus AA is obvious in the 2xQ example above. In the 2x example, I'd think there wasn't any AA at first.

As the texture smearing isn't too bad in many cases, 2xQ is certainly an option. I wouldn't discount it out-of-hand like some people!

 

[Reverend]

Type of AA most beneficial for 360/PS3?

I recommend "None".

 

[nAo]

I recommend "None".

Why?

 

[Rolf N]

If you use that reasoning, moving all the original pixels to random positions would be just as correct.

I don't follow.
A physical pixel is a rectangle with extents. To determine the color of that pixel, finite samples should be taken to make the best representation of the pixel interior. Approximating the sum of all surface areas*surface colors inside the pixel's extents (compensated for gamma) is the target.
If you have one sample, the point at the center of the pixel rectangle is your best bet.
If you have multiple samples, rotated patterns, and later sparse (~n-queens) patterns are the most attractive because they can differentiate more edges that might run through the pixel at angles where the impact on the final color is large. It has been proven often enough in theory and practice that 4xRG >> 2x2OG and 6xSparse >> 4x4 OG.
I can see the value in having multiple carfefully selected sample patterns to choose from on a per-pixel basis, but I'm certainly not a proponent of fully randomized sample grids. I don't see where I implied that.

Quincunx, because it uses a tent filter of 5 samples, should produce better results than a box filter with just 2 samples. Having said this, a tent filter will begin to attenuate some frequencies you would like to keep a bit more than a box filter but it does a much better job of removing illegally high frequencies that you need to eliminate. Of course, with only 2 MSAA to start with, neither filter is going to produce outstanding results.

Contrast ratio between neighbours is all that Quincunx can reduce. There will still be signals with the frequency inherent to the pixel grid, just like before (I'd argue that you construct a grid of rectangles, but sample a cloud of points (just like textures), might throw more words at you if you want).
Try something like a dense chain-link fence purely implemented with alpha test. No matter what post-process you apply, until your filter is so brutal that all you see is constant grey, it will alias like hell, at the same frequency to boot with all wavering blobs of wrong colors and gaps at the same places, just with dropped contrast ratio between neighbour pixels.

As for "destroying" information, I doubt a tent filter destroys any more information than the box filter. I don't have time to prove that, but it seems likely to me.

If you hurl a filter on your entire screen it is not possible anymore to produce a sharp 100% edge between neighbours. But that's exactly what should happen (and does happen with MSAA, no matter the sample count) if e.g. the left pixel is fully covered by a white surface and the right pixel by a black surface.

 

[Fox5]

I won't mind Quincunx AA. Is it almost free without much complication?

Quincunx was horrible! Equivalent jaggy removal to 4x AA at the cost of 2x (or so nvidia's PR said), but it also severely blurred the image.