Reply from nVidia about the 3DMark anomaly

[Doomtrooper]

You are correct about 'two wrongs don't make a right' yet what Website just recently accused ATI of cheating again with anisotropic filtering from a 6 month old article on Digitlife that is trying to show ripmapping

It seems websites have no problem acccusing ATI of wrong doing yet when sufficient evidence was presented about Nvidia (Nvidia even admitting to it by email) only two websites I know of posted Anything about it...3DGPU and Matt Burris and Mike Chambers of Nvnews which I applaud which are ironically Nvidia fan sites.

 

[Xmas]

It seems websites have no problem acccusing ATI of wrong doing yet when sufficient evidence was presented about Nvidia (Nvidia even admitting to it by email)

What exactly do you mean here?

 

[jb]

Crusher there is no reason why you have to be an A$$ about this. I know full well what the difference is between a game and a benchmark. YOU said thay the same optimizations could be used in games powered by the same enigine. I only offered the case of the K2 which shows that may not be the case at all. BTW thoes cases you brought up dont count towards the scores only the game test.

 

[Althornin]

There's no reason for people to be printing stories about NVIDIA's drivers in this case, because there's no proof. Find some proof, and I'm sure they'll be glad to post some news about it.

Heh, the problem is, in ATI's case, people are willing to dig about for information (even when it is obviously wrong).
In nvidias, they just ignore it, and dont even bother looking for said proof.

 

[tamattack]

All indications are that NV is optimizing for 3DMark.

Which indications would that be? The only information to come out of Mad Onion or NVIDIA has been that it is a bug in the driver. The only fact regarding the situation is that the scores differ between having the title screens enabled or disabled. There is still no proof that it is a bug or an optimization, just proof that there is a difference. Even the ammount of the difference has varied greatly between the people reporting to have witnessed it.

Crusher, not meaning to be rude or anything, but how can you argue with 'indications' of optimizing? I know there is no conclusive 'proof' (which in your eyes seems to rely on nv fessing up, which they will never do if it's true), but one can make inferences based on the facts available (ie: 'indications').

As you state, the fact is that the scores are faster with splash screens on vs off. This implies that there is a separate codepath in there. Separate codepath likely = optimization.

I find it logical to conclude that there is a bug in the optimization (much like ATI had a bug in their q3 optimization at the time of quack).

 

[Dio]

When is a separate codepath an optimisation, and when is it a cheat?

 

[Simon F]

When is a separate codepath an optimisation, and when is it a cheat?

That probably depends on which graphics card was purchased.

 

[Tahir2]

 

[jb]

Good one Simon, and that seems like a good place to stop as this thread will not have much more usefull debate.

 

[Entropy]

When is a separate codepath an optimisation, and when is it a cheat?

That probably depends on which graphics card was purchased.

Apparently true. :/
And a fitting stop to that discussion.

However, there is a larger reason why it would be good if this occurence was discussed more in the general media - it might make a larger percentage of consumers more suspicious vs benchmarks in general, and less inclined to place too much of a value on small differences in results in any particular benchmark. To my mind, that would be a good thing. This thread has mostly been about nVidia vs ATI, and questions of guilt. Perhaps it would have been more useful if we had discussed 3DMark and benchmarking in general.

I always feel a bit sad/nauseated/bothered (my English fails me) when I see kids/youth having their 3DMark score in their sigs, and reading how they spend their (parents?) meager funds to climb a little higher in the 3DMark pecking order.

Reviews of video cards on the web and in print consist to 99.9% of benchmarking, and generally benchmarking of depressingly low quality. The reason is simple. Running a few benchmarks, putting the numbers into Excel, and getting graphs posted with some text requires relatively little time and equipment, and next to no skills or understanding whatsoever. It has become what readers expect, and web-sites produce.(*) And it colours our peception strongly. Mine included.

I'll say it again - nVidia can't cheat in 3DMark.
3DMark has no rules against utilising specific code paths for rendering, nor any criteria for correct output.

If everyone who who writes video card reviews gave 30 seconds of thought to that before reviewing and publishing the conclusions they draw from their benchmarking, I'd say we had taken a significant step forward. If they also considered that while this is embarrassing for a benchmarking product, the same is true for all application benchmarks that are used, we would have gained a little more (Quake/Quack).

If reviewers gave another 30 seconds of thought to whether their tests actually produced any transferable information, or if they had simply produced a number for their particular setup/program that couldn't be used for predicting anything else with useful accuracy, we would have changed computer product reviewing forever.

But while we wait for Utopia to come, a somewhat more critical and suspicious attitude would be healthy for consumers.

Entropy