ATi vs. Nvidia IQ showdown at Firingsquad
- Thread starter micron
- Start date
OpenGL guy
Veteran
Thanks for the quick response.crazipper said:
Sometimes it's easier to be vocal on the proper forum...
Sorry if I missed it, I was just concerned that things were being interpreted incorrectly.Also, note that while the article has been updated, I did mention the differences between 6x MSAA and NVIDIA's 8xS implementation previously, contrary to your prior claim. That said, thanks for at least caring enough to comment
Hopefully, constructive criticism
A few unfortunate things about the article remain:
For one thing, some performance disparity is noted throughout the article, and some judgements are made by the writer, but the actual performance disparity is witheld from the reader such that they aren't able to make an evaluation themself (unless I just missed it?). The basic premise of this complaint is this: image quality and performance are intrinsically linked, and, without assuring an accurate representation of the relationship between them, an analysis of either in enforced isolation is prone to rather extreme distortion, no matter how thorough. I don't understand at all why the information was not provided.
For each comparison:
3dmark, scene 01 - seems a pretty accurate assessment of the way the screenshots compare.
3dmark, scene 04 - I don't understand the "ATI is darker" commentary, as it seems to actually be the opposite to me (perhaps someone could clarify more for me). However, the comments on the item details in foreground and background details in the screenshots seem particularly accurate, which is an important part of the comparison.
NASCAR - the context of the continuing discussion of spectator heads is still confused with the prior commentary about ATI having some sort of anomaly, and the "but the discrepancies in detail are so distracting that it’s hard to tell" seems to be wholly political commentary (Why is the writer telling readers how distracted they are? Why does more detail in one section of grass make AA harder to evaluate anywhere else?). The texture detail commentaries seem accurate, however.
IL-2 - I'm still not understanding why the writer is saying the performance for 8x AA is "just fine" instead of showing some sort of performance indication for the reader to evaluate for themself. The performance in a scene with more graphical effects (water?) with supersampling would have been informative as well.
UT2k3 - I'm more than slightly puzzled here. First, it takes a close up for AA being on, and focuses on comparing texture quality in that closeup, and categorizes the AA difference as unremarkable. At the same time, I can actually see a difference in the AA without a closeup, especially on the stair edges the the left/middle of the screen. This doesn't mean the texture quality difference isn't accurate, it means that the texture quality difference is placed in the most distorted context possible: with an enforced separation of performance and image output, with one card trying to improve texture quality and another not trying at all, with the only zoomed image presented focusing on highlighting that situation, and with the writer telling the reader that AA is equivalent while concentrating on a shot picked to focus on something else. To conclude, this statement is made: "Fortunately, the texture blurring we previously saw on the ATI card isn’t a problem at all here, and for all intents and purposes, we’d call this a tie", which just tacks on a prior mistake(?) to all of the above.
The texture filtering discussion after that is a bit odd as well, since trilinear filtering's benefit is usually primarily for aliasing in motion, and the commentary focuses solely on the still screenshots, and actively works against recognizing the issue of evaluating in motion by reaching a conclusion that ignores the issue completely.
...
For the conclusion:
I don't understand how nVidia is in a "much better position today than it was several months ago" based on the above...I'm not aware of anything that has changed for the presented information: supersampling had better texture quality than no supersampling before, 8x AA was better than 4x AA for nVidia before, and full trilinear's benefit was primarily in motion before.
Perhaps it is in reference to a prior FiringSquad article, but wouldn't the "better position" be the result of the AA and supersampling being turned up while not discussing the impact this has on performance? I do think the bandwidth of the NV35 cards allows significantly better performance with the supersampling AA than, for example, the NV30 cards, but I wasn't provided performance figures to see that, just fed a performance conclusion (I really don't understand the merit of this).
Also, is this true: "That said, in certain games, ATI texture quality has dipped below what we witnessed in our first image quality piece". I didn't understand this to be supported by the examinations, but perhaps I missed something? If so, what? Does the writer understand the significance of supersampling, and why texture detail comparing less favorably when it is added doesn't mean texture filtering got worse, or is the issue something else? Was the function of LOD bias adjustment, and the drawbacks it can offer, considered, and how this relates to why considering still screenshots alone is inadequate for the conclusions proposed in the piece? I'm worried that we'll see default negative LOD bias forever as the result of continued comparisons with these faults, or maybe even worse: encourage LOD bias adjustment as an unannounced replacement for AF.
This discussion isn't an accusation of malicious intent or dishonesty, as there are choices made and information presented that leave significant room for some other interpretation. But it does relate to how there seem to be, AFAICS, significant issues resulting from some very prominent discrepancies made evident in what was written and how some decisions were made in presenting (and not presenting) information. I hope it can be received with thorough consideration.
A few unfortunate things about the article remain:
For one thing, some performance disparity is noted throughout the article, and some judgements are made by the writer, but the actual performance disparity is witheld from the reader such that they aren't able to make an evaluation themself (unless I just missed it?). The basic premise of this complaint is this: image quality and performance are intrinsically linked, and, without assuring an accurate representation of the relationship between them, an analysis of either in enforced isolation is prone to rather extreme distortion, no matter how thorough. I don't understand at all why the information was not provided.
For each comparison:
3dmark, scene 01 - seems a pretty accurate assessment of the way the screenshots compare.
3dmark, scene 04 - I don't understand the "ATI is darker" commentary, as it seems to actually be the opposite to me (perhaps someone could clarify more for me). However, the comments on the item details in foreground and background details in the screenshots seem particularly accurate, which is an important part of the comparison.
NASCAR - the context of the continuing discussion of spectator heads is still confused with the prior commentary about ATI having some sort of anomaly, and the "but the discrepancies in detail are so distracting that it’s hard to tell" seems to be wholly political commentary (Why is the writer telling readers how distracted they are? Why does more detail in one section of grass make AA harder to evaluate anywhere else?). The texture detail commentaries seem accurate, however.
IL-2 - I'm still not understanding why the writer is saying the performance for 8x AA is "just fine" instead of showing some sort of performance indication for the reader to evaluate for themself. The performance in a scene with more graphical effects (water?) with supersampling would have been informative as well.
UT2k3 - I'm more than slightly puzzled here. First, it takes a close up for AA being on, and focuses on comparing texture quality in that closeup, and categorizes the AA difference as unremarkable. At the same time, I can actually see a difference in the AA without a closeup, especially on the stair edges the the left/middle of the screen. This doesn't mean the texture quality difference isn't accurate, it means that the texture quality difference is placed in the most distorted context possible: with an enforced separation of performance and image output, with one card trying to improve texture quality and another not trying at all, with the only zoomed image presented focusing on highlighting that situation, and with the writer telling the reader that AA is equivalent while concentrating on a shot picked to focus on something else. To conclude, this statement is made: "Fortunately, the texture blurring we previously saw on the ATI card isn’t a problem at all here, and for all intents and purposes, we’d call this a tie", which just tacks on a prior mistake(?) to all of the above.
The texture filtering discussion after that is a bit odd as well, since trilinear filtering's benefit is usually primarily for aliasing in motion, and the commentary focuses solely on the still screenshots, and actively works against recognizing the issue of evaluating in motion by reaching a conclusion that ignores the issue completely.
...
For the conclusion:
I don't understand how nVidia is in a "much better position today than it was several months ago" based on the above...I'm not aware of anything that has changed for the presented information: supersampling had better texture quality than no supersampling before, 8x AA was better than 4x AA for nVidia before, and full trilinear's benefit was primarily in motion before.
Perhaps it is in reference to a prior FiringSquad article, but wouldn't the "better position" be the result of the AA and supersampling being turned up while not discussing the impact this has on performance? I do think the bandwidth of the NV35 cards allows significantly better performance with the supersampling AA than, for example, the NV30 cards, but I wasn't provided performance figures to see that, just fed a performance conclusion (I really don't understand the merit of this).
Also, is this true: "That said, in certain games, ATI texture quality has dipped below what we witnessed in our first image quality piece". I didn't understand this to be supported by the examinations, but perhaps I missed something? If so, what? Does the writer understand the significance of supersampling, and why texture detail comparing less favorably when it is added doesn't mean texture filtering got worse, or is the issue something else? Was the function of LOD bias adjustment, and the drawbacks it can offer, considered, and how this relates to why considering still screenshots alone is inadequate for the conclusions proposed in the piece? I'm worried that we'll see default negative LOD bias forever as the result of continued comparisons with these faults, or maybe even worse: encourage LOD bias adjustment as an unannounced replacement for AF.
This discussion isn't an accusation of malicious intent or dishonesty, as there are choices made and information presented that leave significant room for some other interpretation. But it does relate to how there seem to be, AFAICS, significant issues resulting from some very prominent discrepancies made evident in what was written and how some decisions were made in presenting (and not presenting) information. I hope it can be received with thorough consideration.
micron said:
Nope, the article is terrible. You simply don't compare texture quality of MSAA and SSAA. MSAA is only supposed to remove aliasing and then be used together with AF for sharper textures. The whole point of MSAA is that it is only done on the edges.
That SSAA is improving textures is actually a bad thing because you can reach basicly the same result with much higher performance using MSAA+AF.
Edit:
The conclusion is also way of. The texture quality is exactly the same as always (with R3x0 cards) because that is the way MSAA works.
Hi crazipper,
it's nice to see you here! I hope you can filter out the positive constructive criticism you got here and just ignore the negative destructive criticism!
Here are my 2 cents on your article:
(1) When comparing a multisampled screenshot with a supersampled screenshot, it should be clear to everyone that the texture quality is better on the supersampled screenshot. The average reader of your article might be tempted to say: "Why does ATI not use supersampling? It seems to be superior in every way!". The article fails to make clear (enough) that multisampling is used for a reason - and that is a much less performance impact! Or let's say it differently: When using multisampling instead of supersampling you can do anisotropic filtering for free while getting similar performance. And the article doesn't mention this at all (or at least doesn't stress it enough), which puts ATI somehow in a bad light.
(2) Why doing seperate AA / aniso screenshots? I would have much *much* preferred, if the article would have compared only screenshots with full AA + anisono settings. The article was about maximum IQ, wasn't it? So why bother with half baken settings? The time you could have saved by only using full settings, could have been spent better on adding some performance numbers. The problem with the half baken settings is that ATI's approach (multisampling) strictly seperates edge and texture antialiasing, while NVidia's approach (mixed multisampling+supersampling) mixes both antialiasing effects. So comparing AA alone doesn't make much sense to me.
(3) I would like to see some proof for the claim "That said, in certain games, ATI texture quality has dipped below what we witnessed in our first image quality piece". That's a serious claim and it should not be stated without backing it up.
(4) You didn't seem to have understood that with Det5x.xx not only UT2003 gets no full trilinear filtering, but NO DirectX game does!! Also the differences between full trilinear and half trilinear filtering is noticable in motion more than in still screenshots.
Thanks for listening!
it's nice to see you here! I hope you can filter out the positive constructive criticism you got here and just ignore the negative destructive criticism!
Here are my 2 cents on your article:
(1) When comparing a multisampled screenshot with a supersampled screenshot, it should be clear to everyone that the texture quality is better on the supersampled screenshot. The average reader of your article might be tempted to say: "Why does ATI not use supersampling? It seems to be superior in every way!". The article fails to make clear (enough) that multisampling is used for a reason - and that is a much less performance impact! Or let's say it differently: When using multisampling instead of supersampling you can do anisotropic filtering for free while getting similar performance. And the article doesn't mention this at all (or at least doesn't stress it enough), which puts ATI somehow in a bad light.
(2) Why doing seperate AA / aniso screenshots? I would have much *much* preferred, if the article would have compared only screenshots with full AA + anisono settings. The article was about maximum IQ, wasn't it? So why bother with half baken settings? The time you could have saved by only using full settings, could have been spent better on adding some performance numbers. The problem with the half baken settings is that ATI's approach (multisampling) strictly seperates edge and texture antialiasing, while NVidia's approach (mixed multisampling+supersampling) mixes both antialiasing effects. So comparing AA alone doesn't make much sense to me.
(3) I would like to see some proof for the claim "That said, in certain games, ATI texture quality has dipped below what we witnessed in our first image quality piece". That's a serious claim and it should not be stated without backing it up.
(4) You didn't seem to have understood that with Det5x.xx not only UT2003 gets no full trilinear filtering, but NO DirectX game does!! Also the differences between full trilinear and half trilinear filtering is noticable in motion more than in still screenshots.
Thanks for listening!
THe_KELRaTH
Regular
With regards the UT2003 images I would find more useful information if screenshots were of a similar position as used in older comparative reviews as it would better show how things have moved on for both ATI and Nvidia.
Here's 2 screenshots from H (Jan 27th) using max IQ
www.jlmay.f2s.com/H-UT2003 NV30.jpg NV30 x8xSaa x 8af
www.jlmay.f2s.com/H-UT2003 9700Pro.jpg 9700Pro x6aa x16af
Here's some updated image using a 9700Pro with Cat 3.8 Max IQ settings
www.jlmay.f2s.com/UT2003 9700Pro 10-03-.jpg x6aa x16af (app)
I've included this one as it offers a better performance comparison with current NVidia drivers
www.jlmay.f2s.com/UT2003 - 9700Pro F16af.jpg x6aa x16af (Quality setting via drivers)
All 9700Pro images look almost identical - the main area of enhancement is in performance as fps is up - especially as the screenshots were taken on a slower PC. (H's test rig was an AMD XP2800+ (166) with Dual DDR333 while I used a P4-2.4b (533) Granite Bay.)
What we need now is a similar screenshot using an NV5950 with max IQ so we can see if the AA problem is fixed, performance is now at playable fps and if haze effects are now working.
Edit: I have set DefaultTexMipBias=-0.600000 rather than -0.5 to compensate for Nvidia's texture sharpening.
Here's 2 screenshots from H (Jan 27th) using max IQ
www.jlmay.f2s.com/H-UT2003 NV30.jpg NV30 x8xSaa x 8af
www.jlmay.f2s.com/H-UT2003 9700Pro.jpg 9700Pro x6aa x16af
Here's some updated image using a 9700Pro with Cat 3.8 Max IQ settings
www.jlmay.f2s.com/UT2003 9700Pro 10-03-.jpg x6aa x16af (app)
I've included this one as it offers a better performance comparison with current NVidia drivers
www.jlmay.f2s.com/UT2003 - 9700Pro F16af.jpg x6aa x16af (Quality setting via drivers)
All 9700Pro images look almost identical - the main area of enhancement is in performance as fps is up - especially as the screenshots were taken on a slower PC. (H's test rig was an AMD XP2800+ (166) with Dual DDR333 while I used a P4-2.4b (533) Granite Bay.)
What we need now is a similar screenshot using an NV5950 with max IQ so we can see if the AA problem is fixed, performance is now at playable fps and if haze effects are now working.
Edit: I have set DefaultTexMipBias=-0.600000 rather than -0.5 to compensate for Nvidia's texture sharpening.
madshi said:Hi crazipper,
it's nice to see you here! I hope you can filter out the positive constructive criticism you got here and just ignore the negative destructive criticism!
Here are my 2 cents on your article:
(1) When comparing a multisampled screenshot with a supersampled screenshot, it should be clear to everyone that the texture quality is better on the supersampled screenshot. The average reader of your article might be tempted to say: "Why does ATI not use supersampling? It seems to be superior in every way!". The article fails to make clear (enough) that multisampling is used for a reason - and that is a much less performance impact! Or let's say it differently: When using multisampling instead of supersampling you can do anisotropic filtering for free while getting similar performance. And the article doesn't mention this at all (or at least doesn't stress it enough), which puts ATI somehow in a bad light.
(2) Why doing seperate AA / aniso screenshots? I would have much *much* preferred, if the article would have compared only screenshots with full AA + anisono settings. The article was about maximum IQ, wasn't it? So why bother with half baken settings? The time you could have saved by only using full settings, could have been spent better on adding some performance numbers. The problem with the half baken settings is that ATI's approach (multisampling) strictly seperates edge and texture antialiasing, while NVidia's approach (mixed multisampling+supersampling) mixes both antialiasing effects. So comparing AA alone doesn't make much sense to me.
(3) I would like to see some proof for the claim "That said, in certain games, ATI texture quality has dipped below what we witnessed in our first image quality piece". That's a serious claim and it should not be stated without backing it up.
(4) You didn't seem to have understood that with Det5x.xx not only UT2003 gets no full trilinear filtering, but NO DirectX game does!! Also the differences between full trilinear and half trilinear filtering is noticable in motion more than in still screenshots.
Thanks for listening!
Thanks madshi,
I'll just go one at a time...
1) Performance impact of various modes will be explored in the next article I work on. The reason it wasn't explored this go 'round is because a majority of the requests I received after the first piece asked for max'ed out settings. The only reason I was able to talk about playability is because I enabled the settings and played around in the game. I didn't take frame rate numbers; rather, I checked for fluidity "just to see what I could see." Granted, not scientific, but performance wasn't the aim of my weekend at the lab.
2) I did seperate AA and AF screenshots to explore the two topics seperately. Considering there are plenty of combined shots in there as well, I don't see any reason not to do the "half baked (though I might use another term for it)" shots. I agree that performance numbers are useful, and I'll be working on that next. Hopefully, with the feedback I'm getting here, it will be a better article as well.
3) In the first article, I found ATI's texture quality, specifically in NASCAR, to be consistent through the baseline, 4xAA, and 8xAF settings, with the exception of the AA shot, which throws the logos out of focus toward the end of the track. This time, there was a significant deviation from the baseline, 6xAA (both very similar), and 16xAF settings. This was taken on a different map, so I realize it isn't scientific, but there seems to be a discrepency there. I'll continue looking into this as well.
4) In 3) you're upset that I don't have proof for something and am making a claim and in 4) you are asking me to make a claim without proof. I read HardOCP's 5950 piece and saw that the new Det's play with the filtering settings on at least the D3D apps tested there, however, I didn't have any way of testing that, and therefore, according to the logic you used previously, wouldn't be qualified to make the claim this time either, right? After all, it's pretty serious to say a driver is messing with filtering settings for every D3D app when I don't think ANYONE here has tested every D3D app, right?
Nevertheless, thank you for wording your comment in such a way that I feel welcome to voice my opinion. I certainly don't see any way I can debate my way out of some of these other posts...[/i]
Hi crazipper,
first of all thank you for taking the time to respond to my comment!
If Det52.16 doesn't produce full trilinear filtering with this test tool, then it probably won't do full trilinear filtering in *any* Direct3D program. Can you agree to that? Or do you think that NVidia intentionally makes itself looking bad by cutting the trilinear filtering in this test application - while in games they use full trilinear filtering? That doesn't make any sense. Also you can check the filtering IQ tests in 3DMark2003 to check for full trilinear filtering.
I mean your article was about maximum IQ. So full trilinear filtering is something which is important, or is it not? Maybe NVidia's "quasi trilinear" filtering is invisible to lots of people in UT2003. But there claim to be some people who do see a significant difference (can't check that myself). And UT2003 doesn't have that hard mip map breaks. So it's quite probable that in some other games the IQ tradeoff will be more significant than in UT2003, don't you think so?
first of all thank you for taking the time to respond to my comment!
Could you please name the two topics you are talking about? That's where the problems begin. I mean I see where you're coming from. You have one AA slider and one anisotropic filtering slider on both NVidia's and on ATI's control panel, that's where you make two topics from. But please think about what effect those two sliders have: They effect (1) edge AA and (2) texture AA. Now SSAA effects both. While MSAA effects only edges. And anisotropic filtering effects only textures. Do you see what I mean? Comparing SSAA to MSAA is not unfair per se, but MSAA was always meant to be only one piece of the puzzle. It was always meant to be companied by anisotropic filtering for maximum IQ. So I don't see much sense in comparing SSAA vs MSAA quality *without* anisotropic filtering. Man, hard to express what I mean. I hope you understand my point...crazipper said:
Right.crazipper said:
No, that was not what I was asking for.crazipper said:
Why were you not able to test that? Look here: http://www.3dcenter.de/downloads/d3d-af-tester.phpcrazipper said:
If Det52.16 doesn't produce full trilinear filtering with this test tool, then it probably won't do full trilinear filtering in *any* Direct3D program. Can you agree to that? Or do you think that NVidia intentionally makes itself looking bad by cutting the trilinear filtering in this test application - while in games they use full trilinear filtering? That doesn't make any sense. Also you can check the filtering IQ tests in 3DMark2003 to check for full trilinear filtering.
I mean your article was about maximum IQ. So full trilinear filtering is something which is important, or is it not? Maybe NVidia's "quasi trilinear" filtering is invisible to lots of people in UT2003. But there claim to be some people who do see a significant difference (can't check that myself). And UT2003 doesn't have that hard mip map breaks. So it's quite probable that in some other games the IQ tradeoff will be more significant than in UT2003, don't you think so?
You're welcome! I never understood the sense of negative destructive comments.crazipper said:
crazipper said:
Why are you tackling this from the standpoint of whether you can "debate your way out of" something, and that you're not being able to debate what someone said makes criticism "destructive"? Where did this as a "construcitve" option disappear to: taking on board things you can't argue against as places where you just might actually need to reconsider and maybe reach a modified conclusion (about what is necessary for achieving an accurate image quality comparison article) from what you did before? Did you misspeak, or do you just not believe there is a problem with that phrasing?
Taking my discussion as an example...was there something rude about my wording, other than you didn't feel you could "debate your way out of it"?
Again, if you misspoke with that phrase, or didn't mean to include me and didn't respond elsewhere for some other reason, please clarify...as it stands, it strikes me as a bit "inflammatorily" concerned with something besides what I understood to be the stated purpose: passing on good and accurate information to the reader...and, for this thread, of having some informed readers offer their opinion on where an article succeeds and fails in achieving that.crazipper said:
Perhaps this is so, but has anyone seen a D3D app where the current NVidia drivers don't hack the trilinear quality? We know that the drivers hack trilinear on UT2003, Serious Sam SE and in synthetic tests. Do we have any indication that they do actually produce proper trilinear filtering in any application?
It may be wrong to assume that the drivers hack trilinear in all D3D applications but there is at least some evidence towards this end. I've not seen any evidence to back up the NVidia PR viewpoint that this is just a 'bug' which doesn't affect actual applications.
Just something that should have been mentioned in the IQ article, I feel.
Hey Chris - hows it going? Have you settled the bet with Doug yet?
One thing...
You don't really need to do this, in fact yoiu can do the reverse. All you need is an application that hasn't been seen before by the drivers that shows off the filtering and then you can see what the drivers are doing by default. this is more or less what we have done with the Aniso tester program that we used in the 5700 preview here.
Cheers,
Dave
One thing...
crazipper said:
You don't really need to do this, in fact yoiu can do the reverse. All you need is an application that hasn't been seen before by the drivers that shows off the filtering and then you can see what the drivers are doing by default. this is more or less what we have done with the Aniso tester program that we used in the 5700 preview here.
Cheers,
Dave
Simon F said:
With high levels of AF there are very little difference between the texture quality with MSAA and SSAA. Comparing the following screenshots shows the difference in texture quality with and without SSAA.
http://www.firingsquad.com/media/article_image.asp?fs_article_id=1366&pic_id=18 (no-AA 8AF)
http://www.firingsquad.com/media/article_image.asp?fs_article_id=1366&pic_id=22(8xAA 8AF)
The difference in texture quality is very slim. Putting the Ati texture quality down because the textures look better with SSAA than with MSAA when AF isn’t used is unacceptable.
The only max-quality-shot from the article where I can see any difference in the texturing quality is the one from Nascar racing but the 8xAF nVidia shots proves that the low quality on the grass textures with the Ati card is caused by the angle dependent AF not by the lack of AF.
http://www.firingsquad.com/media/article_image.asp?fs_article_id=1366&pic_id=21(Ati)
http://www.firingsquad.com/media/article_image.asp?fs_article_id=1366&pic_id=22(nVidia)
Criticizing Ati based on the angle dependent AF scheme would be perfectly OK based on the screenshots in the article.
Edit: Replaced direct links.
