X1800/7800gt AA comparisons

[L233]

think the problem is simply ati assumes you have 2.2 gamma and your lcd doesn't use that, like it should.

I've adjusted gamma accordingly, doesn't even make the slightest difference.

Look at the zoomed image. There you have your problem. It looks like a dotted line because it is a dotted line and no amount of gamma tweaking will change that.

ATI's gamma corrected AA may work miracles on most edges but when it comes to fine lines it seems to produce results that don't look good.

 

[ChronoReverse]

Someone already recalibrated his LCD to 2.2 and the pattern still shows.

 

[Moloch]

I've adjusted gamma accordingly, doesn't even make the slightest difference.

Look at the zoomed image. There you have your problem. It looks like a dotted line because it is a dotted line and no amount of gamma tweaking will change that.

ATI's gamma corrected AA may work miracles on most edges but when it comes to fine lines it seems to produce results that don't look good.

What does the gamma ramp look like on your lcd?

 

[neliz]

Hi Guys,

Thanks everyone for participating in this thread! Thanks also to Sharkfood for more dramatically showing the effect that I was talking about. If any of you guys have a plasma or other high luminance display mind checking out those images?

Jawed:

The LCD display I'm viewing with here at work appears to be calibarted to 2.0. ATI drivers by default assume a gamma-space of 2.2?

Nite_Hawk

I checked on my 42" samsung ppm42m5s (contrast ration 10.000:1) http://product.samsung.com/cgi-bin/nabc/product/b2c_product_detail.jsp?eUser=&prod_id=PPM42M5S/XAA

yes, on that screen the checkerboard is obvious, very obvious.. but what's also obvious is the fact that the jpeg compression is able to screw up the red letters 8's look like there are x's in them... there are black dot's in the F etc.

unless someone gets us uncompressed screenshots, it's useless going into detail on this...

 

[Nite_Hawk]

I checked on my 42" samsung ppm42m5s (contrast ration 10.000:1) http://product.samsung.com/cgi-bin/nabc/product/b2c_product_detail.jsp?eUser=&prod_id=PPM42M5S/XAA

yes, on that screen the checkerboard is obvious, very obvious.. but what's also obvious is the fact that the jpeg compression is able to screw up the red letters 8's look like there are x's in them... there are black dot's in the F etc.

unless someone gets us uncompressed screenshots, it's useless going into detail on this...

Hi Neliz,

thanks for doing this. I'll try to get a hold of Brent and see if we can get uncompressed images.

Nite_Hawk

 

[neliz]

Hi Neliz,

thanks for doing this. I'll try to get a hold of Brent and see if we can get uncompressed images.

Nite_Hawk

NP,

But do you see what they made us do? take a part of a screenshot, say 3x7cm, magnify it 300% and display THAT on 107cm (42") high contrast plasma to see if it's friggin obvious.



Anyway, the ati line looks checkered. the nv line looks like it hardly received aa at all.. (I'm talking about the original pictures.)

 

[Nite_Hawk]

NP,

But do you see what they made us do? take a part of a screenshot, say 3x7cm, magnify it 300% and display THAT on 107cm (42") high contrast plasma to see if it's friggin obvious.



Anyway, the ati line looks checkered. the nv line looks like it hardly received aa at all.. (I'm talking about the original pictures.)

How does the non blown up version look? Also, how does the 8xs mode look? Hopefully I'll be able to test on a plasma in a little while as well. Hey, atleast this is any interesting discussion, better than yelling back and forth about which card was marginally faster in far cry.

Nite_Hawk

 

[OpenGL guy]

ATI's gamma corrected AA may work miracles on most edges but when it comes to fine lines it seems to produce results that don't look good.

4x AA can't do miracles, you know! The result is the best that can be expected with 4 samples. Try 6x and you'll get better results.

 

[Nite_Hawk]

4x AA can't do miracles, you know! The result is the best that can be expected with 4 samples. Try 6x and you'll get better results.

Aha! Now we know you are reading this thread and can pester you for answers!

Any ideas about the effect that we are seeing? I am still thinking luminance, but perhaps we will know more when/if we get uncompressed images.

Nite_Hawk

 

[OpenGL guy]

Any ideas about the effect that we are seeing? I am still thinking luminance, but perhaps we will know more when/if we get uncompressed images.

Thin lines at particular angles can cut off all, sample or no samples. If all samples are cut, then you see a gap. If some samples are cut then you see a light line. If none are cut, then you see a heavy line. Looking at the result, the line is about 1 pixel in width. This can give odd results as some pixels will be heavy (all samples are in the line) while neighbors can be thin (some samples in the line). The average value of the samples is the same, however, so things are good: Gamma correction is doing the right thing. I don't see any gaps in this case, so that's good as well.

As I said, 4x AA can't do miracles, you really need many more samples to remove all aliasing. 6x AA should give better results, but 12x AA on CrossFire would probably look very nice indeed as the sample pattern is very good and 12 samples is quite a bit more than 4.

 

[EasyRaider]

The "checkered line" effect was immediately obvious to me when reading the article, before it was pointed out to me. Overall though, AA seems like a toss-up to me. I can't really conclude anything from compressed images, on this portable with quite poor monitor.

 

[neliz]

How does the non blown up version look? Also, how does the 8xs mode look? Hopefully I'll be able to test on a plasma in a little while as well. Hey, atleast this is any interesting discussion, better than yelling back and forth about which card was marginally faster in far cry.

Nite_Hawk

Okay, 6x vs 8x (the fence)
again, AD against a light background leads to dissolving of the texture in the background on a light background (right part of the screenshot) but there is clearly more detail in the ati shot against the dark (red brick) background of the left side of the shot.
also, the (barb)wire supposedly running throught the spikes on top of the fence seem to be "gone" or very hard to spot on the 6xAD shot.. and two white pixels on the closest fence texture :|

The screenshots with the branches and foliage show nv leading, again, ati's AD seems to dissolve textures against the background too easily.
There was one thing I noticed with the nv shots, and that is that some vertical foliage (bottom of the screenshot) against a white background exhibit the same problem, one leaf that shows on the ati and default image is hardly visible on the nv shots, only the top part exists... and you know why? the lowest branch of the tree (dark) portrudes through the leaves, there is clearly no Z check done before applying the TRAA and the branch just seems to push the leaf to the background. again, this just seems to lead back to the way nv handles gamma.

There is nothing more I can add to the single fence screenshot.. it just looks like ati blurres the whole fence into the background.

but what is the performance hit for the r520 6xADmsaa vs. the g70 8xTR SSAA?

 

[neliz]

The "checkered line" effect was immediately obvious to me when reading the article, before it was pointed out to me. Overall though, AA seems like a toss-up to me. I can't really conclude anything from compressed images, on this portable with quite poor monitor.

See, that's exactly opposed to what I saw when I saw the screenshots on a crt, 2xaa and 4xaa are for ati hands down while the 6/8 shots go to nvidia.

I'll have a look at work tomorrow and see what my n610c thinks of those images.. then again.. I can also check what it looks like on my 20"hitachi lcd ... and to be honest.. on that screen it looks like the ati 2x and 4x screenshots are just shot in another resolution.. that's how crappy the aa looks..

 

[Moloch]

Could sshots in general just look worse on lcds since they dont have "free aa" like crts do?
In terms of jaggies of course.

 

[OpenGL guy]

The screenshots with the branches and foliage show nv leading, again, ati's AD seems to dissolve textures against the background too easily.
There was one thing I noticed with the nv shots, and that is that some vertical foliage (bottom of the screenshot) against a white background exhibit the same problem, one leaf that shows on the ati and default image is hardly visible on the nv shots, only the top part exists... and you know why? the lowest branch of the tree (dark) portrudes through the leaves, there is clearly no Z check done before applying the TRAA and the branch just seems to push the leaf to the background. again, this just seems to lead back to the way nv handles gamma.

There is nothing more I can add to the single fence screenshot.. it just looks like ati blurres the whole fence into the background.

??? What do you expect from antialiased pixels? Some samples come from the fence, others from the background. This is absolutely correct.

 

[Nite_Hawk]

Okay, 6x vs 8x (the fence)
again, AD against a light background leads to dissolving of the texture in the background on a light background (right part of the screenshot) but there is clearly more detail in the ati shot against the dark (red brick) background of the left side of the shot.
also, the (barb)wire supposedly running throught the spikes on top of the fence seem to be "gone" or very hard to spot on the 6xAD shot.. and two white pixels on the closest fence texture :|

I'm seeing the same effect basically. Too me the ati 6xAD shot on the red brick doesn't seem to be doing as good a job eliminating stairstepping as well though. I can see how it may be more detailed though. Aren't the two white pixels on all of the shots (including the one with no AA)?

The screenshots with the branches and foliage show nv leading, again, ati's AD seems to dissolve textures against the background too easily.
There was one thing I noticed with the nv shots, and that is that some vertical foliage (bottom of the screenshot) against a white background exhibit the same problem, one leaf that shows on the ati and default image is hardly visible on the nv shots, only the top part exists... and you know why? the lowest branch of the tree (dark) portrudes through the leaves, there is clearly no Z check done before applying the TRAA and the branch just seems to push the leaf to the background. again, this just seems to lead back to the way nv handles gamma.

hrm... I did notice that the ATI shots on that leaf appear to have a very slight amount of the same effect. I wonder if maybe it is exhibiting the same problem that OpenGL guy just mentioned? it seems that the leaf may be thiner than 1 pixel at that point.

There is nothing more I can add to the single fence screenshot.. it just looks like ati blurres the whole fence into the background.

but what is the performance hit for the r520 6xADmsaa vs. the g70 8xTR SSAA?

Well, for the non-transparency AA 8xs of course should be quite a bit slower than 6x msaa. For AD though, It looks like ATI is actually using super sampling (with MSAA for polygon edges). In terms of the additional overhead caused by AD, I imagine ATI's solution is probably scales relatively linearly with nvidia's assuming that nVidia doesn't have some kind of special hardware solution.

Nite_Hawk

 

[Jawed]

Hi Neliz,

thanks for doing this. I'll try to get a hold of Brent and see if we can get uncompressed images.

Nite_Hawk

I think HL-2 outputs all screenies as JPGs.

Might have to try something like HL-1 (e.g. the Cobble map in CS has wires).

Jawed

 

[Nite_Hawk]

I checked on my 42" samsung ppm42m5s (contrast ration 10.000:1) http://product.samsung.com/cgi-bin/nabc/product/b2c_product_detail.jsp?eUser=&prod_id=PPM42M5S%2fXAA

yes, on that screen the checkerboard is obvious, very obvious.. but what's also obvious is the fact that the jpeg compression is able to screw up the red letters 8's look like there are x's in them... there are black dot's in the F etc.

unless someone gets us uncompressed screenshots, it's useless going into detail on this...

I just tested it on a Sony KE-42xS910 42" plasma display. This display is connected with component cables so a lot of detail is lost on the native shots. Both ATI and nVidia's solutions look about the same. The zoomed shot make the ATI AA look *terrible* though. Much more apparent than even the LCD screen from earlier. I'm not sure how apparent that would be on the non-zoomed shots if this were a DVI connection instead.

Anyway, I just wanted to confirm your findings.

Edit: I also want to mention that this display appears to be calibrated to 2.2 gamma, but the colors must be off because I am getting splotches of color on the three color gamma chart.

Nite_Hawk

 

[neliz]

I must say I haven't calibrated my plasma (yet) but it's connected through a dsub cable (vga 15 pins) on a dvi adapter on the dvi port of my 9800pro.. :|


Anyway, I think the whole thing with the tree and the leaf is just as interesting.
Remember when l'inq had those shots of TRAA and showing how much better it was in comparison to xfire AA? well, those shots exhibited the same things, some of the nv screenshots didn't even have the radio antenna's on them.

When you study http://www.hardocp.com/images/articles/1128280140ABTiXJphEC_8_6_l.jpg you can see that, both don't do a good job at the thin leaf, but on the nv screenshot it's basically missing a bigger piece than on the ati screenshot. .then again, I'm watching these pics on a crt now, no idea what it looks like on lcd/plasma

It's just looks like 8xTRSSAA is a bit too aggressive for dense foliage and winter trees..
I'll have a better look at it tomorrow afternoon.. I seriously need some sleep now..

later

 

[neliz]

I think HL-2 outputs all screenies as JPGs.

Might have to try something like HL-1 (e.g. the Cobble map in CS has wires).

Jawed

Mm... Fraps? Ati Tray Tools?