I am totally stunned by this. . . After all NVidia bashed 3dMark03, saying that it wasn't a valid benchmark. . . why go through the effort of adding such optimizations -- or more appropriately, hacks -- to their drivers? My guess is that they realized just how many users rely on 3dMark to measure video performance.
DetonatorFX on Nature in 3dMark03
- Thread starter Ostsol
- Start date
Hanners said:
Now that's the REAL problem.
I'm just reviewing a FX 5800 and of course the 3Dmark03 results just went out the window.
But can I trust the other results? Shall I throw away the 2001 scores also?
Can I trust the UT2003 and Quake3 scores? The Shadermark scores? etc.
This just makes my life as a reviewer totally miserable. :/
Evildeus said:And as you are of Ati? As i said, that can be the case for every chip makers, that doesn't mean they all do it, but they can
Right now I don't have any suspicion of ATi.
Since the Quake 3 debacle (which was fixed with increased performance AND quality, and IMHO always looked to damn obvious to be a cheat but I might be wrong) I haven't ever seen anything that looks "strange" on Ati boards.
ATi also addressed the Quack 3 issue pretty much imedtiately
Once nvidia steps up and ask for fogiveness and some time passes I will of course be less wary of them. But the whole fact that they've pretty damn pestering about this is kind of making me worried.
The minute people discover the precision cheating they go and do this.
this normal plan of action would be:
"Ok we're very sorry guys but yes: we cheated. It won't happen again." and then it didn't happen again.
But when they just go "naaah we don't know what you're talking about" then they replace the cheat with another one (which in itself is almost the same thing a confessing)...
well yeah, seems pretty strange
Well the point is: Did they or didn't they? I don't know, do you? Sure nothing did get out but does that mean they didn't?
And if this new fonctionnality of 3DMark wasn't available, unfortunately, nobody would have ever known :/
Finally, i agree with you, Ati has for the past year a far better reputation (concerning drivers or hardware) than Nvidia.
but does that mean they didn't?And if this new fonctionnality of 3DMark wasn't available, unfortunately, nobody would have ever known :/
Finally, i agree with you, Ati has for the past year a far better reputation (concerning drivers or hardware) than Nvidia.
Evildeus said:As i said, that can be the case for every chip makers, that doesn't mean they all do it, but they can
Unless the given benchmark gives you a possibility to check for cheats or hacks - which is exactly what being on the 3dMark03 betaprogram does. This is also why I strongly support the move for those on the betaprogram to come forward with these findings.
Evildeus said:
I made posts on various websites on visual anomalities in 3Dmark03 with DetFX a few days ago so even though I didn't have the means to reveal the exact nature of this I at least saw the "sideffects" of this optimization.
So as I said: unless there's anything to raise suspicion I don't see why we should be worried. Anything that doesn't effect output is IMHO pretty much an ok optimization.
O fcourse doing uneccesary work in your code just to increase workload on the GPU open up for cheating as well. And if I'm not mistaken GT2 has some elements in the code which is basically just reruns that increase the workload but doesn't effect output. (Hey I'm a layman don't kill me if I'm not technical enough for B3D
)Of course cutting those out in a game if possible would only be a smart thing. In a benchmark it's a whole different matter though since the testing there isn't of game performance but rather "how well does card a run this code", especially when it comes to testing shaders.
Oh well, I just hope nvidia doesn't just lock up and go into denial mode now... that's the thing that'll hurt them the most me thinks.
Ante P said:O fcourse doing uneccesary work in your code just to increase workload on the GPU open up for cheating as well. And if I'm not mistaken GT2 has some elements in the code which is basically just reruns that increase the workload but doesn't effect output. (Hey I'm a layman don't kill me if I'm not technical enough for B3DEvildeus said:
Bypassing inefficient code and replacing it with functionally equivalent (in that particular benchmark) rendering is eminently possible. In fact, it was an nVidia driver programmer who pointed out that 3Dmark03 invited this, due to the way it was coded. He expressed irritation since doing such work was an effort which didn't benefit any other application.
Provided the driver produces the same end result, I don't see how this could be detected from the application/user side, unless you recode the application, and see how the results are affected - did it provide the expected change in performance using the suspect driver? Of course, you could disassemble the driver code, but that would seem a pretty thankless task.
Entropy
PS. It might be a good idea if FutureMark produced a MkII version of 3DMark which plugs some of the openings that has been pointed out over time. The benchmark haven't seen sufficient use yet that a reference database of results have accumulated yet that is anywhere near what it is likely to be until it it is time for a DX10 level version.
Just as an updated FYI on the situation. I have used this release of 3DM03 on an NV31 and both 43.45 and the WHQL 43.51 drivers. The apparant pixel bleed was present in both these sets of drivers on both GT2 and GT4, however in the 43.45's the 'clipped' areas only appeared in GT2, it wasn't until 43.45 drivers that the 'clipped' areas appeared in GT4. I've not looked at the 44.03 drivers.
I'm not here to suggest what is occuring, I'll wait for an official comment from NVIDIA (I've been with them all week and have periodically pushed for one but as yet theres nothing official).
I'm not here to suggest what is occuring, I'll wait for an official comment from NVIDIA (I've been with them all week and have periodically pushed for one but as yet theres nothing official).
No. That's silly for one of the main purpose of the benchmark is comparing similar systems.Entropy said:
If you change some, you cannot compare on standardized basis.
And if there be some inefficient codes in the program, it's for the coder to fix it with patches.
And nothing really important for it affects every vga.
Code substitution by a specific IHV driver only for its products is ridiculous at best.
chavvdarrr
Veteran
what's interesting - some people claim BIG fps improvements IN REAL GAMES (enter the matrix the one post i look now) on 5200 board... so question is:
Is this a "specific" 3d2003 hack , or a real-world 3dfx-alike HSR improvement ?
Is this a "specific" 3d2003 hack , or a real-world 3dfx-alike HSR improvement ?
DaveBaumann said:
I wouldn't expect one until Nvidia has had a chance to see just how far and wide the idea of them cheating is pushed. So far I still haven't seen Toms, Anand, H, the inquirer, the register etc even mention it. If only a few specialised sites mention I still think they will opt to ride out the storm!
Of course I'd love to be proven wrong and see the above sites add a headline or caveate to thier existing reviews. Irrespective of the sites actions hell will freeze over before nvidia admit it got caught! The driver bug however implausible is the only realistic line you'll get. I can see it now, Han tells the share owners that he'd rather they loose sales and market share after admitting cheating in bench marks so long as he could tell the truth
DaveBaumann said:
Interesting.
Well it's quite easy to assume that they worked on GT2 first in order to cover <DX8 products (GF3/GF4 etc) and then later added their own special something to GT4 in order to give the FX range a further boost. That way their entire lineup has ended up benefitting, but in yet another astonishing display of naivety they have done this a step at a time!
It'll be interesting to hear what nV have to say on the matter. I suspect they will go the "we did it to discredit 3DMark" route now as that would probably be the least damaging admission. Very sad.
MuFu.
Seiko said:
Absolutely. In fact, I would advise them to try and let it fizzle out (while preparing some good damage control just in case things got nasty), if I was their PR head man.
They have to adress it somehow sooner or later, but a 'bug fix' driver release is all really it takes. Sure, some wont forget this, but nVidia can survive loosing those 347 upset customers.
LeStoffer said:They have to adress it somehow sooner or later, but a 'bug fix' driver release is all really it takes. Sure, some wont forget this, but nVidia can survive loosing those 347 upset customers.
Actually from what I've seen at the forums I susually visit (except nvnews of course) people seem pretty damn upset in general.
And of course once/if [H], Anand, THG and the other major sites catches on this thing will be exposed to a much wider audience.
