Futuremark Announces Patch for 3DMark03

Can't quite see the fuss myself, I agree with what people have said about starting with a clean slate.

Now obviously if nvidias cheats are reconfigured to work with 340 in their next drivers then these can't be approved. Nor should 3dmark be patched to counter them, I think the ball is pretty much in the IHVs courts atm.

 

The thing is that FM want to penalise the cheating drivers merely by (they hope) defeating all the cheats and lowering the 3DMark score. They are not doing what you, me, and several others want ie, effectively disqualify Nvidia's drivers for attempted cheating.

If you think about it, FM are trying to provide accurate 3DMark scores. They don't care if it's done at the driver or at the application, though this does leave the loophole of cheats that they have missed.

For instance, imagine if Nvidia stopped cheating on 3DMark. Their 3DMark scores would go down, but there is nothing to stop Nvidia continuing to cheat on all other benchmarks. "Honest" 3Dmarks scores don't prove that Nvidia isn't cheating in other tests, and merely show that Nvidia cards are slower at the synthetic 3DMark tests.

It seems obvious that Nvidia are willing and able to cheat, and so should be considered to be cheating elsewhere. There is no way for FM to disprove this, and it is outside their remit. All FM want to do is make sure that their 3DMark 2003 scores are clean of cheats, without going as far as pointing at Nvidia and calling them cheaters - even if that is exactly what the results show.

 

Now I checked the "Optimization Guidelines Q&A" document released on September 26, but I didn't see much there that would help here. However, the "Process to Enforce 3DMark Optimization Guidelines" document released on October 31 does have something that makes you wonder...

http://www.futuremark.com/companyinfo/Enforcement_Process.pdf

Since ver 52.16 was released on October 23. Could this be the corrective action? Could possibly explain why the scores don't drop down as much as previous versions. Or maybe they didn't give any corrective action, but instead gave an explanation that Futuremark was satisfied with. Or it's intirely possible that Futuremark didn't demand anything and decided to do what others have suggestion and started with a clean slate. This would suggest that guidelines are only valid for drivers going forward. Man, what a freaking mess. Let's continue...

First of all, Futuremark has listed the driver as "Reviewed and Approved". This means that if Futuremarks suspected the guidelines had been breeched, then outcome #1 must have happened. This also means(according to the flowchart) they have notified all the members of the benchmark program as well.

Further inspection of the flowchart shows that other people's comments might be correct that the enforcement of the guidelines only affects new drivers. The flowchart states "New driver is published" at the beginning. To me it looks like that we may never get an explanation on the current NVIDIA drivers because it was "grand-fathered" in. I don't like that at all, but let's see if Futuremark makes announcement on this.

Tommy McClain

 

has anyone (Dave?) tested the 3.9 Catalysts? Given ATI's openness in dealing with this issue, I'm not expecting anything earth-shattering, but it would be nice to know how it stacks up with the new patch.

 

Cat3.9's are idential to cat 3.8's WRT to any 3DMark03 version. There don't appear to be any performance tweaks with the 3.9's.

 

Let's face it - all that Futuremark can do to ensure accurate scores is disable the optimisations in the current drivers. There is no chance that NVidia will say, "Sorry, forgot about those optimisations. We'll remove them from the next driver". To do so would undermine all the FUD they've been very successfully spreading about 3DMark in recent months.

I expect Futuremark will have to release more patches in future as new official NVidia drivers are released, once again featuring such 'optimisations'. I wonder when the first 'optimisation-free' driver will be released? Shortly after the release of the NV40, perhaps?

 

from the few FX scores that have gone up on the orb it seems the FXs score in the PS2.0 test is unchanged (and still beating 9800s).

futuremark missed one here?

 

Finally!

Thanks for the notification, worm. I understand politics may trump honesty, but it's the end result that counts, and FM seems to finally have the cheating licked.

Now let's see what nV does to regain that lost performance with the Det 57's. (I think it's inevitable they'll do something, and that'll be the real test of FM's new policy.)

 

Only one thing comes to my mind: write better cheats.

New drivers on the horizon anyone?

 

New drivers=new patch, betcha a cookie! 8)

 

Excuse me good sirs, but


-We had real world benchmarks in games allowing us to compare ATi and Nvidia performance in games.

-We had real world IQ comparisons at NUMEROUS reputable sites allowing us to compare ATi and Nvidia IQ.

-People could use this information to choose between the video cards and purchase what they wanted.

-No one had any complaints.


So, if Nvidia is running "non-standard" code (like that exists or something) in the background to achieve their results, WHO CARES? We know what we're getting when we buy, regardless of whether or not we truly know how we get it.

This is a pretty pointless thread in my opinion. 3Dmark is a pretty useless utility in my opinion. Wait for DX9 games, real DX9 performance tests using those games, and then make your decision. Sheesh. Who cares if they code their drivers differently?

 

I bet not. I believe they are probably done patching 3DMark03. If and when NVIDIA releases a new driver it won't be reviewed or approved right away. They will take their time to check it out and if it's found it still contains detections or optimizations for 3DMark03, they will demand an explanation or a corrective action. If they don't then all the guidelines, the QA and the enforcement documents were all done for not.

Still want to bet that cookie?

Tommy McClain

 

More details at NordicHardware. Thanks AnteP !

http://www.nordichardware.se/artiklar/Grafik/2003/3DMark03_340/

 

Sure! (My sister is the manager of a bakery. )

 

In that case, no NV driver after (and including) 44.03 would be approved by FM, making video card comparisons with 3DMark03 practically impossible and thus invalidates the benchmark's selling point ...

93,
-Sascha.rb

 

Or, you are missing the boat, because the 52.16 series driver DOES NOT CHEAT (has no driver optimizations that break guidelines) in 3dmark03 patch 340.

 

Joe,

Good post. Much better than mine.

I agree with this. I suspect Futuremark will probably release 3DMark04 before they release another patch.

I'm starting to believe this is the case well. Futuremark are getting what they want: valid 3DMark03 scores without having to call them cheats.

I don't agree with this. Why does Futuremark have to do this? They have already stated that they won't accept scores that use drivers that aren't approved. So everything on their site that is used officially will use the older approved drivers and show the lower, correct results. There's no need for Futuremark to have to release a new patch with every release of offending driver. If NVIDIA wants results that show their chips are faster, then they will have to release a non-offending driver.

Agreed. It will be interesting to see what path they choose.

Agreed. Now that I believe I understand what they have done. It's OK with me as well. Now, if they decide they need to release another patch with another driver release, then this process is NOT going to work and they might as well just throw in the towel.

Tommy McClain

 

Umm, that's possible I guess. I have a hard believing that considering the disparity between the results of 2 different builds. Without any comments from Futuremark I guess there is no way of knowing for sure.

Tommy McClain

 

No, he's saying that BECAUSE of the new patch, the new drivers aren't cheating specifically in 3DMark, because they can't. Without the patch to stop them, yes they're cheating - but that's irrelevant here.

 

I hope that is the correct statement that Althornin meant....