ExtremeTech Article Up

[gkar1]

I really was not surprised by the blatant cheating, its the hypocrisy of the company, first attacking the benchmark and now cheating to get a better score. Complete lack of ethics. Sad thing is the fanbois(reviewers and consumers alike) will turn a blind eye to it.

 

[nelg]

Perhaps this shows why 3Dmark03 is a good bench mark. If you have the pro version you can see who's been naughty and who's been nice. If you did this on a game time demo what tools could a reviewer use to catch such a cheat ?

 

[Joe DeFuria]

I really was not surprised by the blatant cheating, its the hypocrisy of the company, first attacking the benchmark and now cheating to get a better score. Complete lack of ethics. Sad thing is the fanbois(reviewers and consumers alike) will turn a blind eye to it.

I'm not so sure.

There is a point where, particularly web reviewers, will turn the tables. That point is: if nVidia makes them look bad.

Everything is all well and good if nvidia is giving them Doom3 tests and making them look "good." However, if it turns out that web reviewers have to go back and modify thier recommendations to their readership...because "nVidia pulled a fast one"...that will really piss reviewers off, and rightfully so.

Readers put some trust into the web reviewers for an honest assesment. Without that trust, readership goes down. And if an IHV does something to undermine that trust....it can get ugly. Whether or not this incident turns out to be something like that remains to be seen, but my point is, web reviewers won't put up with everything.

 

[Solomon]

When is Aquamark 3 coming out so we can just forget about 3DMark03?

 

[Josiah]

doesn't anyone else think 3dMark is silly. its becoming less and less representative of actual real-world performance, and it's vulnerable to this sort of cheating.

 

[dream caster]

8)

the one million dollar question:

Can this "bug" be related to GF5900 advantage in Doom3 demo?

 

[jjayb]

doesn't anyone else think 3dMark is silly. its becoming less and less representative of actual real-world performance, and it's vulnerable to this sort of cheating.

As is every other timedemo out there. Did you read the article? The cheat works by knowing where the camera is going to move. Last I checked almost all websites use the same demo's for benching. Ut2k3, Serious Sam, Splinter Cell etc.... And they all use the same time demos for these games. The same thing can be done with all of these.

 

[Solomon]

It would be a hard sell to sell that the Doom III scores could of been inflated because of this anamoly with the 44.03. The reason ID supervised the results. So if it did in fact happened in Doom III. It wouldn't make ID very credible in some aspects. Just my two cents.

 

[RussSchultz]

8)

the one million dollar question:

Can this "bug" be related to GF5900 advantage in Doom3 demo?

No. The demo was recorded by iD, the code drop was a "random" one (chosen by iD _i think_). It would be either a) impossible to do the same sort of cheating alleged in the extremetech article or b) perform so well at reducing workload that it isn't cheating but an honest to goodness optimization.

Unless, of course, you'd like to believe that it was rendering with tons of artifacts that neither Kyle or Carmack noticed.

 

[Clashman]

Perhaps this shows why 3Dmark03 is a good bench mark. If you have the pro version you can see who's been naughty and who's been nice.

Or who's better at covering their tracks.

 

[tamattack]

Doom III Scores

I agree, very unlikely that this situation with 3DMark03 applies to the Doom III timedemo.

However, that's not to say that it's impossible... ID did provide it to NV for transport to the selected outlets, therefore giving NV some "alone" time with it before it got to Anand/[H].

 

[BenSkywalker]

I know nVidia's driver team it thought of very highly, but do people really consider them so good that this couldn't possibly be a bug? Not saying that it is, but why would they screw up GTA Vice City in the same fashion? Doesn't have a bench that I'm aware of.

 

[Althornin]

I know nVidia's driver team it thought of very highly, but do people really consider them so good that this couldn't possibly be a bug? Not saying that it is, but why would they screw up GTA Vice City in the same fashion? Doesn't have a bench that I'm aware of.

its a mighty big coincidence, eh, that the driver bug just happens to not screw up really while the camera remains "on track", but the minute you leave it, it looks like you are playing doom2 with "noclip" and are running around outside the level.
Too big of a coincidence to really swallow, imo.

 

[nelg]

It would be a hard sell to sell that the Doom III scores could of been inflated because of this anamoly with the 44.03. The reason ID supervised the results. So if it did in fact happened in Doom III. It wouldn't make ID very credible in some aspects. Just my two cents.

Inlight of the fact that that we could not see screen shotsof the Doom 3 test we can only speculate that they were being honest in regards to the driver settings, percision, etc. With all the mistrust Nvidia has brewed for themselves, any, and all results should be questioned.

 

[V3]

I know nVidia's driver team it thought of very highly, but do people really consider them so good that this couldn't possibly be a bug?

If this is indeed a bug, Doom3 tests should also be put under the same scrutiny for other possible anomalies.

But if this cheating, than its unlikely to effect Doom3 tests.

 

[Arun]

Okay, so let me get this clear...

That's ALL there is to it?
There better be more, otherwise the hype about those cheats is vastly unjustified

Let's face it, the sky problem WILL be gone in the next driver revision. And I mean, really gone. It's so darn fricking easy to fix it isn't even funny anymore. As I explained in another post, it should be possible to *cache* a DIP call and draw it after everything else. And then, it's perfectly undetectable.

The buffer clearing problem surprises me most however. From the looks of the screenshot, it seems like they're still clearing Z, but not color. Odd, considering the NV3x got Fast Color Clear. So yes, they're cheating, but I'd be surprised if they got more than a 3% boost from that.

Oh, sure, there may always be other things, but I guess I was right. I'm not particularly shocked by all this. The performance boost they're getting is probably lower than 10%, which would indicate they might still be able to be on par with the Radeon 9800.

While it does proof nVidia is cheating, and it would indeed be better if they stopped, this ain't THAT much when you think about it. And heck, anyone drawing the sky before everything shouldn't expect GPU companies not to cheat IMO. It's just too unoptimal.


Uttar

 

[antlers]

I know nVidia's driver team it thought of very highly, but do people really consider them so good that this couldn't possibly be a bug? Not saying that it is, but why would they screw up GTA Vice City in the same fashion? Doesn't have a bench that I'm aware of.

its a mighty big coincidence, eh, that the driver bug just happens to not screw up really while the camera remains "on track", but the minute you leave it, it looks like you are playing doom2 with "noclip" and are running around outside the level.
Too big of a coincidence to really swallow, imo.

Exactly. The clipping is tied to the camera path--it can't be an accident. The only respect in which it is a bug is that they don't turn it off when the camera leaves the expected path. As OpenGL guy said, I'm sure they will fix that one shortly.

 

[ZoinKs!]

And heck, anyone drawing the sky before everything shouldn't expect GPU companies not to cheat IMO. It's just too unoptimal.

But what if it's deliberately drawing the sky first in order to create a more stressful benchmark? Sure, there probably are better ways to do that, but that's irrelevant. 3dmark was created to run in a certain way in order to provide repeatable, comparable results across a variety of vidcards. Cheats which circumvent those methods are inexcusable.

 

[Ostsol]

The buffer clearing problem surprises me most however. From the looks of the screenshot, it seems like they're still clearing Z, but not color. Odd, considering the NV3x got Fast Color Clear. So yes, they're cheating, but I'd be surprised if they got more than a 3% boost from that.

I don't really think that not clearing the colour buffer is really critical. . . UT by default doesn't clear it, at least in OpenGL. That was one of the problems I encountered while trying to create a "wallhack" cheat (never actually used online). I think it's the same for a lot of engines and demos, actually. . .

 

[Mark]

It will be interesting to see if older drivers (and the 5800) have the same "enhancements"