John Reynolds said:That's a nice way of putting it, but even with a gamer-oriented perspective Kyle is using an illogical piece of thinking (because a synthetic's DX9 shader performance fails to equate with an OGL game scores using vendor specific code paths it must be trash) to promote a blatantly obvious anti-Futuremark agenda.
I agree with your general thrust, and agree that it is tantamount to being anti-FutureMark (if we can use such a phrase without laughing at how ridiculous it is...)
However, I think the anti-FutureMark stance is merely a sideline to the real agenda which is to help nVidia promote itself by way of discrediting any software which nVidia finds inconvenient from a commercial perspective. Note that the latest nVidia attack is on "fraps" (for goodness' sake) and that certain [H] staffers have taken to adding perplexing and baffling signatures such as "The Benchmark Slayer" to their posts.... So, FutureMark is just one of many demons nVidia feels it must attempt to slay because it views such software as diametrically opposed to the ends of selling its hardware.
I think the chances of nVidia's success in this endeavor is roughly equivalent to the probability of us seeing a manned mission to Mars set foot on the Red Planet by this Christmas... For that reason it is odd indeed to see [H] staking the entire credibility of the site, something it has taken years to build, on such a silly set of notions like "frame-rate benchmarks are dead" and so on. [H] is likely to share whatever lack of success comes to nVidia by way of this campaign. It surely will not succeed.
Instead, what has clearly transpired is that the nature of benchmarks is definitely changing, but the change has nothing to do with eliminating frame-rates; rather, it has to do with adding as much emphasis on IQ as heretofore has been placed mainly on frame-rate counts alone. Yes, benchmarks are changing, but they are evolving into "frame-rate & IQ" benchmarks instead of being mere frame-rate counters. And that's what I see that is troubling nVidia--they'd just as soon everything revert to frame-rates only or else no one run any benhmarks at all. I can't see any difference between the position nVidia has taken and [H]'s position, unfortunately. Both are equally illogical, and equally improbable.
I also find it distasteful how it's never mentioned that those Doom 3 scores were derived from workstations supplied/built by Nvidia. So he's not only using a very illogical argument but he's also building it on a potential house of cards.
By far the site most impressed with the "Doom 3 preview" is [H.] It comes up in Kyle's editorial comments again and again. Most everyone else understood it for what it was: a contrived, carefully controlled PR event the main purpose of which was to promote nVidia's nv35 hardware. It was neither open nor fair nor objective in any sense of the word, and Kyle's pretensions otherwise have not taken root in sites other than [H.] His position that a pre-release game *demo* utilizing a vendor-specific path (and therefore itself not representative of the bulk of 3D games) is "A-OK", but a benchmark using generic DX9 shader code is an abomination, is really too bizarre to understand.
But so is nVidia's anti-benchmark crusade, which is just dumb. If nVidia's hardware was as it is represented by nVidia then the company would have nothing to fear from people using such software. IMO, nVidia is simply afraid of people using that type of software to discredit its marketing efforts. Either Kyle doesn't see that he's being used as a dupe, or else he intends it deliberately. What on earth would make anyone imagine that the only thing people are interested in, and should be interested in is how well a piece of hardware runs a pre-release Doom 3 preview demo that few people on earth have had any access to. If he thinks that this is the only avenue of interest people have in looking at 3D chips he is of course very much mistaken.