New 3DMark03 Patch 330

[martrox]

At last, that's good news.

What is good news, ED? That nVidia has officially been shown to be cheaters?

 

[Evildeus]

Interesting, any idea why the GT2 shows like 10-15% decrease and not as much on GT3? (0-5%)

Here are some GeForceFX 5200 Ultra scores :

AMD AthlonXP 2000+
512MB
DX9.0a
WinXP Pro

GeForceFX 5200 Ultra -- DetonatorFX 44.03

3DMark03 build 320

3DMark Score		1822 3DMarks
GT1 - Wings of Fury		91.7 fps
GT2 - Battle of Proxycon	10.5 fps
GT3 - Troll's Lair		8.2 fps
GT4 - Mother Nature		9.9 fps
Fill Rate (Single-Texturing)	940.7 MTexels/s
Fill Rate (Multi-Texturing)	920.1 MTexels/s
Vertex Shader		5.5 fps
Pixel Shader 2.0		9.3 fps
Ragtroll			5.8 fps


3DMark03 build 330

3DMark Score		1526 3DMarks
GT1 - Wings of Fury		75.2 fps
GT2 - Battle of Proxycon	9.1 fps
GT3 - Troll's Lair		7.9 fps
GT4 - Mother Nature		7.0 fps
Fill Rate (Single-Texturing)	940.7 MTexels/s
Fill Rate (Multi-Texturing)	920.2 MTexels/s
Vertex Shader		4.1 fps
Pixel Shader 2.0		5.5 fps
Ragtroll			5.7 fps

 

[Himself]

Version 330 eh? You guys sure you won't ever release an SE version of 3dmark03?

 

[antlers]

Futuremark will do what it can--

but I don't see how an industry-standard benchmark can exist when one of the main IHV's is hostile to it and looking to undermine it.

If NVidia wanted, they could cheat the other way, and make sure none of their cards reported more than 300 3dmarks. It would undermine the benchmark even more effectively--no review sites would bother using it-- and it would be much simpler for NVidia to do.

 

[Joe DeFuria]

I think Futuremark has done a good service to help legitimize the benchmarking process.

Agreed.

nVidia was bitching about how inconsequential 3DMark is, becauase it is prone to "optimziations". They were bitching about how it drags the industry down by having driver writers tune drivers to a "synthetic benchmark" and not games.

Quite ironically, FutureMark basically answered nVidia's call. FutureMark said "look, if ANYONE does such things, part of our job is to catch you doing it."

Just like FutureMark said in this PDF. It's not that cheating is any more or less easy with a syththetic test. It's getting CAUGHT that is easier with a synthetic test.

Now that everyone knows FM is "on the look-out" and will actually take measures to comabat such things, nVidia should no longer have any concerns.

I expect nVidia to now fully endorse 3Dmark!

 

[Evildeus]

At last, that's good news.

What is good news, ED? That nVidia has officially been shown to be cheaters?

No that FM has stated it.

You will also noticed that Ati was also doing some optimisations/cheats, whereas no information from B3D/ET did emerge before this PR.

 

[Nite_Hawk]

antlers4:

It probably would screw futuremark, but it'd hurt nvidia pretty badly too. Atleast initially (becuase so many people use 3dmark) they'd miss out on a lot of oem contracts, and potential buyers. It'd really hurt their reputation even more so than the current drivers do. Nvidia really wants to be a slippery target, not really pinned down for any specific thing, always changing it's tactics, but always looking better than the competition (atleast in terms of performance, which is what probably really matters when talking about sales). By making such a blatent stand, they'd be putting themselves in a sink or swim scenario.

Nite_Hawk

 

[Dave Baumann]

Interesting, any idea why the GT2 shows like 10-15% decrease and not as much on GT3? (0-5%)

GT3 is an indoor scene, there is no skybox there and so no clipping could be done.

 

[Joe DeFuria]

If NVidia wanted, they could cheat the other way, and make sure none of their cards reported more than 300 3dmarks.

But that is a very risky path to take. The risk is, people don't lose faith in the benchmark....they lose faith in nVidia instead.

 

[Evildeus]

Some figures on NV35/R350
http://www.hardware.fr/html/news/#5797

 

[mczak]

wow, overall score is much lower as expected, but look at the pixel shader 2.0 decrease! It's nearly half of what it was with the old driver. How is the pixel shader 2.0 test performed? Is it part of GT4?

Nite_Hawk

If you read the report, this is cheat 8 - PS 2.0 test is a feature test, not related to GT4.
I'm really worried by that - the doubling of the FX5900 fp shader power as claimed by nvidia over the FX5900 is not even due to a driver optimization, but due to a driver cheat (of course the FX5800 gets the same boost, but that's why the scores are very similar betweeen 5800 and 5900). I'm really wondering if the shadermark increases seen lately are also just because of cheats, and not due to driver optimizations - that would mean the whole FX fp shaders are really as weak as initially feared.

 

[Dave Baumann]

You will also noticed that Ati was also doing some optimisations/cheats, whereas no information from B3D/ET did emerge before this PR.

Thats because its not visible anywhere.

My suspicion is that they are replacing the shader code with one that is optimised for their Vec3 / parallel Scalar pixel shader. Because ATI only have one internal precision, unless they start ripping chunks of the shader out, its not going to be visible.

 

[Brent]

Wow, I didn't realize the extent of cheating that was going on in 3dmark03

It’s unacceptable

Whether it be a game or a synthetic test, it's unacceptable


Good job Futuremark, it was a very well laid out and easy to understand PDF.

 

[martrox]

You will also noticed that Ati was also doing some optimisations/cheats, whereas no information from B3D/ET did emerge before this PR.

Hmmmm...1.9% for ATI, and 24.1% for nVidia.... Keep grabbing at those straws, ED. I guess that just goes to prove how much better nVidia's drivers are than ATI's....

Bottom line will be that ATI is optimising, while nVidia is cheating......AND, there is a difference!

 

[Joe DeFuria]

wow, overall score is much lower as expected, but look at the pixel shader 2.0 decrease! It's nearly half of what it was with the old driver.

Which is what the PS 2.0 scores of the "original" drivers were.

How is the pixel shader 2.0 test performed? Is it part of GT4?

Yes, there is PS 2.0 shaders in the sky...which is one reason why having clip planes illegally removing portions of the sky can have a big performance impact.

 

[John Reynolds]

Bottom line will be that ATI is optimising, while nVidia is cheating......AND, there is a difference!

Don't be too quite to assume that. 8% difference in just one test is worth investigating.

 

[Nite_Hawk]

mczak:

Do you know how the test is performed though? Is it running the mother nature pixel shaders, doing something else? I agree, I think the most disappointing thing out of all the tests is probably that the PS tests are so slow after the cheats are removed...


Nite_Hawk

 

[martrox]

Wow, I didn't realize the extent of cheating that was going on in 3dmark03

It’s unacceptable

Whether it be a game or a synthetic test, it's unacceptable


Good job Futuremark, it was a very well laid out and easy to understand PDF.

Brent...IF nVidia will go to these extremes for 3DM03, then just what would they do for Doom3? Don't you think it would be wise to talk to Kyle and get him to check these things out? Or would you want to continue working for a site that is obviously heavely biased?

 

[Joe DeFuria]

Don't be too quite to assume that. 8% difference in just one test is worth investigating.

Agreed. I'm happy FutureMark is looking into it.

 

[kid_crisis]

What Is the Performance Difference Due to These Cheats?
A test system with GeForceFX 5900 Ultra and the 44.03 drivers gets 5806 3DMarks with 3DMark03 build 320. The new build 330 of 3DMark03 in which 44.03 drivers cannot identify 3DMark03 or the tests in that build gets 4679 3DMarks – a 24.1% drop.

Well, darn good thing you had a lawyer look over that release. That way there wasn't any embarassing errors, such as not being able to calculate a simple percentage correctly.....

4679 / 5806 = 80.6% or a 19.4% drop