zurich said:
...
I mean, look at this objectively;
OK
the 52.xx performance gains are massive.
Without regard to gains being relative to achieving the same output, I do think this is a fair objective assessment.
But it seems to me that if this is done without achieving the same level of output, such "performance gains" is best characterized as "fictional". Therefore, it seems important to me to take a reasonable amount of steps to answer that question. I think AT failed.
The IQ quality compared to 4x.xx is again a huge improvement.
Hmm...I don't quite get how you propose this statement as factual, such that you make it a fundamental premise of an "objective" evaluation.
I have a problem with this statement being proposed as such due to an abundance(!) of factors:
- A concentration on the Anti-"Synthetic Benchmark" mantra, while completely ignoring the "fixed camera path problem" that is the real issue nowadays, and which is more likely to be exposed with dedicated ("Synthetic", nevermind how representative the workload) benchmarks. I feel like someone is trying to sell me a bridge, and some reviewers seem to have bought it already.
- Selective presence of full screen images for reader controlled comparison. It takes work to reduce images after capture, so it isn't like they didn't have them to begin with. It also seems to me that you can "cut and paste" any "click to view original" code, so it wouldn't be more work to include them. This leaves me with the impression of selective truth to facilitate Anand's conclusion in place of the reader being given full information to come to their own, which in my experience is one effective tool of "marketing" and/or "lying". The rollover issue someone else mentioned also concerns me, as well as the idea of jpeg based comparison of filtering and AA in still screen shots...especially with so many being only available as reduced to 450x450 (as far as I could determine).
[list:49e5e70d81]
- Aquamark 3 : NO full size image.
- C&C Generals : full size images.
- F1 Challenge : NO full size images (with an AA/AF comparison, even!).
- Gunmetal : NO full size images.
- Halo: full size images (NO AA/AF comparison...no evaluation, illustration, mention, or consideration of shader effects at all, which you think might be warranted with the large amount of confidence Anand speaks invests in the 65% performance gains. Ah, but even though they can relate to graphics workload, cut scenes don't represent gameplay anyways...but do the screenshots? What about representing compiler technology or card performance, as Anand uses them for (especially in his conclusions about compilere technology)...which relates to the fixed camera path issue I'll cover in more detail later.
- Homeworld 2 : NO full size images (with an AF comparison...AA didn't work for the nVidia cards, though resampling to 450x450 seems likely to make that a bit irrelevant for the screenshots).
- JK- Jedi Academy : full size images (and finally with an AA/AF comparison. But hold on, they recommend you set them in game, but then went and set them in drivers, though in game settings are exactly where a difference should be most evident between ATI and nVidia...if it weren't a jpeg still screen shot and there were more than one texture layer used in the game (?). It is the only full size image AA/AF comparison that it seems possible for the reader to evaluate effectively, yet their selection of how to test it doesn't seem logical or consistent with even their own statements, nor with issues surrounding nVidia and ATI in general.
- NWN : NO full size images (with another AA/AF comparison).
- SimCity 4 : NO full size images (with another AA/AF comparison).
- TR-AOD: The most competent and thorough explanation in the review, AAICS, and full size screenshots offered for examining PS 2.0 output. What disturbs me is the lack of an actual thorough PS 2.0 performance comparison between ATI and nVidia (from any benchmark, it wouldn't have to have been TR-AOD) to go with what they use TR-AOD to show. Least logical comment in the discussion : "I know everyone will want to take these numbers and say that they universally describe the DX9 performance hit on NVIDIA hardware, but we have had plenty of other benchmarks today that show very different results." In contradiction to his discussion of how specific to TR-AOD his analysis is, it is even used to validate every other shader benchmark in a way inconsistent with the data shown.
- UT2k3: NO full size images (and Yet Another AA/AF comparison at the reduced size). No controversy around this game and filtering...let's just show the full size images for JK instead.
Did I just miss the full size images being offered for this? That would make more sense than what I think is the case now.
- WC3: NO full size images (with an AA/AF comparison).
- Wolfenstein-ET: NO full size images (with an AA/AF comparison).
- X2 : Not sure what the screenshots really mean. Looks like something in the balancing act I've thought AT has tried before, where (IMO) the "alternate" IHV is shown in the best possible light in one targeted instance to shield from accusations of bias, even when that requires doing something that doesn't even seem to make sense. ATI's AA is better, but what does showing us a screenshot of a high contrast logo demonstrating this really tell us, when you propose it represents a game? Why not a screenshot from the models/graphics for the rolling demo?
OK...maybe I'm looking at old pages, or a complete collection of full sized images were offered elsewhere and I missed it. Perhaps someone could correct me if such is the case, or I made some errors in the above. As it stands, I'm just a bit boggle by the inconsistencies, and that still leaves the following...
[*]This next list is made especially poignant In the context of the above, and conclusions like this: "The very large performance gains we saw in Halo, Aquamark3, X2 and Tomb Raider can be attributed to the enhancements of NVIDIAs compiler technology in the 52.xx series of drivers", when this relationship is made up on the spot in the conclusion.
A lack of answer to the some of the specific questions (borrowed from Valve's presentation, and related outside of that context as appropriate) seems a gross oversight:
- Camera path-specific occlusion culling.
Independtly confirmed to have been done by nVidia before. On the camera path, image quality is (usually) identical, but completely unrepresentative for performance evaluation or comparison. Big problem when this image quality comparison doesn't even seem to take the rudimentary step of custom timedemos to begin to try and avoid this happening. 
- Visual quality tradeoffs (lowered filtering quality, disabling fog).
Hmm...I'm pretty sure I've seen "4x.xx" drivers without filtering issues, though the fog question is still unanswered for me. I think Anand could have answered it, but I don't recall him trying. This could, however, be something that the new drivers improved.
- Screen-grab specific image rendering.
This remains unconfirmed outside of Valve's statements, AFAIK. Does seem to throw another monkey wrench in limiting the selection of AA/AF comparisons that users can effectively evaluate for themselves, while at the same time proposing they are universal indicators of "improved image quality" and skipping consideration of other AA/AF issues in your conclusion. As if there weren't enough wrenches in that already.
- Lower rendering precision.
Big problem, where was the investigation of this? Perhaps they meant B3D to be used for this, but the comparison they linked to doesn't seem to answer that question. Nor do they seem to consider it for their conclusions.
- Algorithmic detection and replacement.
Actually, with the HLSL compiler SINCOS issue, I think there is a valid case for nVidia to do this at the moment. Unfortunately, I don't think nVidia would limit themselves to validly addressing such problem when they did do shader replacements. Of course, if you use Cg and don't get performance gain compared to HLSL, this should remove that concern, AFAIK. In any case, see the above issue with where I think AT went wrong with evaluating the concerns this brings into it.
- Scene-specific handling of z writes.
I'd say this is a big problem in reality, but not a big problem for AnandTech failing to check this (at the moment, with the lack of tools we have, or at least that I know about).
- Benchmark-specific drivers that never ship.
I find certain displayed "logical conduct" regarding driver usage policy at AT some reason for major concern. However, this is only a big problem if the drivers used by Anand are actually never released, despite "assurances". This could end up being moot, however, if they do so...except, of course, for the "HL 2 performance update" blog entry (which, however, wasn't actually in an article, though the conclusion to this one reminds me of the same mindset).
- App-specific and version specific optimizations that are very fragile.
I think this is more of a "big problem" for a game that is going to be updated very frequently, when performance will vary greatly as these optimizations "break", unless the developer agrees to freeze their shaders for the convenience of an IHV making custom replacements . I know this is/was a big problem for HL2, but I don't know how much of an issue it is for other games...except as far as having to wait for nVidia to get around to your game, and making patches a possible nightmare for developers. Also, the uncertainty as to whether nVidia will be able to have success in creating an equivalent shader (outside of those games and timedemos used as popular benchmarks) or not.
[/list:u:49e5e70d81]
I don't really think much of this article, overall...quality over quantity. There seems to have been a good opportunity and frame work for some really good work here, but with spotty follow-through, though a lot of time seems to have been spent. The use of reduced size images seems ridiculous to me (I'm still wondering if I just missed something?), and even if that wasn't an issue, there is a complete failure to addresse even the simplest of the last list of concerns, as well as a significant lack of coherency in the "logic" present in the conclusion.
Where there is full sized IQ analysis, the choices of methodology and performance comparison presentation seem to be reasonably characterized as being chosen selectively to prevent showing nVidia in a bad light, and then applied as a universal basis for evaluating all other information, even where the same things were specifically
not evaluated in what it speaks about (AA/AF and the relative performance of the "PS 2.0 compiler" between nVidia and ATI).
The (absent) performance comparison (not the same thing as "performance hit relative to itself comparison") between ATI and nVidia in TR-AOD would seem an excellent opportunity to test the idea of nVidia's ability to "take the crown" from ATI as they propose in their conclusion, yet its not even clear to me if this comparison is without shaders completely, or with minimum settings. Not that I could tell anything from the screenshots anyways (way too dark), but I'm assuming it is due to my monitor that I'm replacing.
This, to me, is large scale deception and/or significant incompetence. Perversely, the strength of what they did show with TR-AOD (I don't have an issue with it, I have an issue with what it is used in place of and the information that ends up being witheld due to that) seems to argue against the one of those I personally find the least disappointing.
Yes, ATI is still ahead slightly, but that never caused anyone to 'hate'/witch hunt them when they were behind in the DX8 days.
Where do you get the idea that nVidia is "hated" because they are behind? Maybe some diehard nVidia fans "hate" them for that (?), but I think the hostility towards nVidia I'm most familiar with is for reasons besides that.
It's just pretty messed up. 52.xx shows large improvements. Anandtech praises these impovements (while still giving the final nod to ATI), yet gets accused of being on the NVIDIA payroll.
The issue isn't as simple as "the reported fps gains are greater" and "Anand says the image quality is improved".
Pretty scary times these days.
That conclusion does not seem even remotely objective to me, nor does it seem to have been determined based on objective evaluation.
To be fair:
This isn't so much an obervation on Anand's article, but rather on the responses I've read around the net regarding it.
I know there are some people who go to extremes in hostility, but they happen to have been given a rather large amount of cause. For instance: I didn't generally find it "scary" that people condemned ATI vehemently for "Quack", and there was quite a bit less behind the issues going on then. I did, and do, find some of the "logic" surrounding both issues a bit "deficient", "lacking", and therefore "surprising" to me in that way...but that has to do with (some of) the reasoning, not the hostility itself.
My concerns above relate to both matters.
