GF4 has inflated 3dmarks scores so says the INQ.....

[Joe DeFuria]

Matt,

We know that the default benchmarks are those with the screens on. That's part of the problem. "By default" the higher scores are reported. (Of course, that's only a problem if the "lower" scores are the correct scores.)

I basically agree with what JB said, to expand one thing he said:
* Both showed a difference in FPS scores when run in a different manner that (3dmark disable splash & Q3 rename exe) and one wouldn't expect this behavior to happen.

It WOULD be interesting, and it's worth "investigating" this unexpected behavior. It's just not worth getting all wrapped up about unless there is some proof of purposeful wrong-doing. But that's what the ATI camp has been saying since the Quack thing...

 

[borzwazie]

hmm

I don't even own an ATI card but I agree with those who say the way HardOCP hammered the ATI driver "cheat" issue was far from professional. I will also admit, however, that the increased attention to drivers on public websites, as well as the realization by ATI of that attention has done more to improve their drivers than anything else.

The reason I don't own an ATI card is the driver issue, which I see to be less and less of a point at this time. I'm looking for a newer card (as soon as I get the money together for it), and it's a pretty tough call about which card to pick now. The 8500 is a damned good card, and with good drivers and it's current price point, it's a pretty tempting buy. The Ti4200 looks sweet too, oh the decisions

 

[lurtz]

who knows whether it is a cheat or an "optimization" but...

www.nvnews.net/articles/david_kirk_interview.shtml

"We at Nvidia don't make it a practice to optimize our pipeline for specific benchmarks - we want to provide high quality and high performance on a wide variety of useful and entertaining applications, rather than just getting a good score." David Kirk, Nvidia

 

[muted]

I don't know about you , but i just love how The Inquirer never gives out their sources .... since it came from a forum .. and the INQ had "DOCUMENTS"

Alain Tiquet, marketing director of Nvidia Europe, told the INQUIRER that both his company and Mad Onion were aware of the problem and they are working together to fix the problem, which he said only made a few percentage points difference.

A new driver to fix the problem will be made available later in the day, he added

now i'm troubled by this ... How is this a problem , and i thought higher was better . If Anyone knows since when this "problem" is active, it would be interesting to see

 

[Vince]

"We at Nvidia don't make it a practice to optimize our pipeline for specific benchmarks - we want to provide high quality and high performance on a wide variety of useful and entertaining applications, rather than just getting a good score." David Kirk, Nvidia

What BS. Q3A Timedemo anyone?

 

[Matt Burris]

Vince:

Could you show us the code that has the optimizations, or provide a source of where you got that info? You sound so sure of it, and I've never heard that one yet. I'm interested in seeing it.

 

[Neeyik]

In response to an earlier question, the option to disable the title screens is there to save a bit of time if you are doing LOTS of bench tests (e.g. reviewing cards). The various tests take different amounts of time to load the scene but the screen time is the same, just for the sake of...well..."pretties"! Don't forget that the first 3 frames of each test are run blank before the timing kicks in anyway.

 

[Doomtrooper]

http://discuss.madonion.com/forum/s...p;view=collapsed&sb=5&o=0&fpart=1


Don't know if its true or not

 

[jandar]

"We at Nvidia don't make it a practice to optimize our pipeline for specific benchmarks - we want to provide high quality and high performance on a wide variety of useful and entertaining applications, rather than just getting a good score." David Kirk, Nvidia

What BS. Q3A Timedemo anyone?

Considering that in Quake3, nVidia chips perform significantly higher, yet in other games that use the Q3 engine (even if modified) the nVidia lead is next to none.

http://www6.tomshardware.com/graphic/02q2/020522/ti4400_4600-12.html#benchmarks
Max Payne, using the same subsytem as 3DMark 2001
nVidia pulls ahead with every GF3 and above.

http://www6.tomshardware.com/graphic/02q2/020522/ti4400_4600-13.html
Quake 3 Engine but modified: nVidia's lead is non-existant

http://www.sharkyextreme.com/hardware/videocards/article.php/3211_1146541__4
Again, Quake 3 nVidia leads, RTCW (Q3 modified engine) it's lead is non-existant.

Fine tuning for one game or app? Yup.......
David Kirk is nothing more than a spinmeister, including making claims that the GF3 and GF4 cards WERE DX 8.1 cards, at least until Microsoft said otherwise.

It's also getting to the point where reviewers only test a few apps, and its sad to see Q3 still used as a benchmark.
It's equally sad to see someone use thigns like CodeCreatures as a benchmark, when it states that it was developed with assistance from nVidia and states itself as a DX 8.1 Benchmark, when it doesnt use PS 1.4

 

[BoardBonobo]

I've installed the new patch, and the problem is still there. It makes no difference at all. And strangely enough it is only the nature scene where the splash screen seems to make any difference. It occurs if you use Pure Hardware, T&L, or software (not that I think they have any great bearing on the matter but I'd rather safe than sorry).

I still don't see it as an optimisation but it is, surely, a driver bug that at the worst may affect some games. It's quite puzzling as to why it happens, maybe the GF isn't setting the scene up correctly? The framerate doesn't recover at all through the nature scene so it isn't something that's happening just at the beginning.

I think somebody at nVidia is about to get an almighty b*ll*cking

 

[king_iron_fist]

On a related note - this is my guess at what next years splash screen looks like :

 

[Gerry]

http://discuss.madonion.com/forum/showflat.pl?Cat=&Board=mo3dmark2001&Number=864548&page=0&view=collapsed&sb=5&o=0&fpart=1


Don't know if its true or not

Read further on and it seems pretty obvious that MadOnion believe it's simply an nVidia driver bug. Once fixed they expect the scores without the screens to increase, rather than vice versa.

This has slight similarities to the Quack issue. In that case it was more obvious, since it affected image quality. In the end it was simply a driver bug as well.

 

[Doomtrooper]

There is a difference with Quack issue, and I do think they are both 'driver' bugs but one is giving higher results while the quack issue was fixed two weeks after the discovery and the frames remained flat.

We'll see how this shakes out...so far there hasn't been alot about it and I see Worm closed the thread

 

[jandar]

makes you wonder if MadOnion ISNT smarting from someone picking apart thier code that easily.

Then again, could be a driver fix and that they are tired of being called nVmark 2001SE.

 

[DemoCoder]

Well if this is a "cheat", it's a fairly innocuous one. Vast majority of the people in the forum reported score differences of less then 50 3dmarks.

The reason Jedi Knight 2 and RTCW show a lesser benchmark spread is because those games push the Q3 engine to the limit. Massive geometry, overdraw, more textures, etc. Just go play Medal of Honor: Allied Assault, another Q3 engine game. This game really taxes T&L/CPU, and has lots of overdraw in the open-outdoor cities/towns.

Q3 just isn't an "outdoor" engine, and those games have alot of outdoor-ish levels.

 

[Doomtrooper]

Saw this posted at another forum...

system specs
amd athlon tbird 1.2
768meg mushkin rev2 pc133@2-2-2
asus a7v-133 (volt. mod)
promise raid0
creative sblive 5.1
wintv stereo pci
nvidia gf4 ti 4600

3dmark 2k1 build 300 without intro screens using leaked 29.40 drivers
3DMARK2001 PROJECT FILE

PROJECT
Name NVIDIA GeForce4 Ti 4600 1024x768x32 NoAA Z24 DXTC DB D3D Pure HW
Description
Registration Name xxxxxxxxxx
Registration Key xxxxxxxxxxxxxxxxxxxxxxx
3DMark Version 300

DISPLAY
Platform NVIDIA GeForce4 Ti 4600
CPU Optimization D3D Pure Hardware T&L
Width 1024
Height 768
Depth 32 bit
Z-Buffering 24 bit
Texture Format Compressed
Buffering Double
Refresh Rate 60 Hz
FSAA Mode None

OPTIONS
Show Title Screens No
Continuous Benchmark No
Benchmark Run Count 1
Demo Sounds Enabled Yes
Continuous Demo No
Game Sound Effects Enabled Yes
Game Music Enabled Yes
Game Detail Level Low

RESULTS
3DMark Score 7581
Game 1 - Car Chase - Low Detail 106.4 fps
Game 1 - Car Chase - High Detail 35.1 fps
Game 2 - Dragothic - Low Detail 148.1 fps
Game 2 - Dragothic - High Detail 81.6 fps
Game 3 - Lobby - Low Detail 99.0 fps
Game 3 - Lobby - High Detail 44.2 fps
Game 4 - Nature 41.4 fps
Fill Rate (Single-Texturing) 1032.2 MTexels/s
Fill Rate (Multi-Texturing) 2291.9 MTexels/s
High Polygon Count (1 Light) 37.9 MTriangles/s
High Polygon Count (8 Lights) 12.0 MTriangles/s
Environment Bump Mapping 162.6 fps
DOT3 Bump Mapping 142.3 fps
Vertex Shader 96.3 fps
Pixel Shader 117.4 fps
Advanced Pixel Shader 91.3 fps
Point Sprites 25.4 MSprites/s

3dmark 2k1 build 300 with intro screens using leaked 29.40 drivers
3DMARK2001 PROJECT FILE

PROJECT
Name NVIDIA GeForce4 Ti 4600 1024x768x32 NoAA Z24 DXTC DB D3D Pure HW
Description
Registration Name xxxxxxxxxxx
Registration Key xxxxxxxxxxxxxxxxxxxxxxx
3DMark Version 300

DISPLAY
Platform NVIDIA GeForce4 Ti 4600
CPU Optimization D3D Pure Hardware T&L
Width 1024
Height 768
Depth 32 bit
Z-Buffering 24 bit
Texture Format Compressed
Buffering Double
Refresh Rate 60 Hz
FSAA Mode None

OPTIONS
Show Title Screens Yes
Continuous Benchmark No
Benchmark Run Count 1
Demo Sounds Enabled Yes
Continuous Demo No
Game Sound Effects Enabled Yes
Game Music Enabled Yes
Game Detail Level Low

RESULTS
3DMark Score 8115
Game 1 - Car Chase - Low Detail 112.1 fps
Game 1 - Car Chase - High Detail 35.1 fps
Game 2 - Dragothic - Low Detail 160.8 fps
Game 2 - Dragothic - High Detail 92.8 fps
Game 3 - Lobby - Low Detail 102.4 fps
Game 3 - Lobby - High Detail 45.5 fps
Game 4 - Nature 44.8 fps
Fill Rate (Single-Texturing) 1032.3 MTexels/s
Fill Rate (Multi-Texturing) 2291.7 MTexels/s
High Polygon Count (1 Light) 38.1 MTriangles/s
High Polygon Count (8 Lights) 12.2 MTriangles/s
Environment Bump Mapping 161.0 fps
DOT3 Bump Mapping 143.4 fps
Vertex Shader 97.2 fps
Pixel Shader 117.4 fps
Advanced Pixel Shader 91.4 fps
Point Sprites 29.3 MSprites/s

3dmark 2k1 build 300 without intro screens using official/whql 29.42 drivers
3DMARK2001 PROJECT FILE

PROJECT
Name NVIDIA GeForce4 Ti 4600 1024x768x32 NoAA Z24 DXTC DB D3D Pure HW
Description
Registration Name xxxxxxxxxx
Registration Key xxxxxxxxxxxxxxxxxxxxxxx
3DMark Version 300

DISPLAY
Platform NVIDIA GeForce4 Ti 4600
CPU Optimization D3D Pure Hardware T&L
Width 1024
Height 768
Depth 32 bit
Z-Buffering 24 bit
Texture Format Compressed
Buffering Double
Refresh Rate 60 Hz
FSAA Mode None

OPTIONS
Show Title Screens No
Continuous Benchmark No
Benchmark Run Count 1
Demo Sounds Enabled Yes
Continuous Demo No
Game Sound Effects Enabled Yes
Game Music Enabled Yes
Game Detail Level Low

RESULTS
3DMark Score 7387
Game 1 - Car Chase - Low Detail 99.7 fps
Game 1 - Car Chase - High Detail 33.2 fps
Game 2 - Dragothic - Low Detail 145.7 fps
Game 2 - Dragothic - High Detail 81.0 fps
Game 3 - Lobby - Low Detail 95.7 fps
Game 3 - Lobby - High Detail 43.3 fps
Game 4 - Nature 41.4 fps
Fill Rate (Single-Texturing) 1033.3 MTexels/s
Fill Rate (Multi-Texturing) 2294.6 MTexels/s
High Polygon Count (1 Light) 38.6 MTriangles/s
High Polygon Count (8 Lights) 12.0 MTriangles/s
Environment Bump Mapping 163.7 fps
DOT3 Bump Mapping 143.0 fps
Vertex Shader 96.7 fps
Pixel Shader 118.2 fps
Advanced Pixel Shader 90.9 fps
Point Sprites 25.4 MSprites/s

3dmark 2k1 build 300 with intro screens using official/whql 29.42 drivers
3DMARK2001 PROJECT FILE

PROJECT
Name NVIDIA GeForce4 Ti 4600 1024x768x32 NoAA Z24 DXTC DB D3D Pure HW
Description
Registration Name xxxxxxxxxx
Registration Key xxxxxxxxxxxxxxxxxxxxxxx
3DMark Version 300

DISPLAY
Platform NVIDIA GeForce4 Ti 4600
CPU Optimization D3D Pure Hardware T&L
Width 1024
Height 768
Depth 32 bit
Z-Buffering 24 bit
Texture Format Compressed
Buffering Double
Refresh Rate 60 Hz
FSAA Mode None

OPTIONS
Show Title Screens Yes
Continuous Benchmark No
Benchmark Run Count 1
Demo Sounds Enabled Yes
Continuous Demo No
Game Sound Effects Enabled Yes
Game Music Enabled Yes
Game Detail Level Low

RESULTS
3DMark Score 8108
Game 1 - Car Chase - Low Detail 112.0 fps
Game 1 - Car Chase - High Detail 34.9 fps
Game 2 - Dragothic - Low Detail 161.1 fps
Game 2 - Dragothic - High Detail 92.9 fps
Game 3 - Lobby - Low Detail 102.6 fps
Game 3 - Lobby - High Detail 45.4 fps
Game 4 - Nature 44.3 fps
Fill Rate (Single-Texturing) 1033.3 MTexels/s
Fill Rate (Multi-Texturing) 2289.2 MTexels/s
High Polygon Count (1 Light) 39.1 MTriangles/s
High Polygon Count (8 Lights) 12.2 MTriangles/s
Environment Bump Mapping 163.4 fps
DOT3 Bump Mapping 143.4 fps
Vertex Shader 96.7 fps
Pixel Shader 119.8 fps
Advanced Pixel Shader 90.9 fps
Point Sprites 29.2 MSprites/s

3dmark 2k1 build 330 without intro screens using official/whql 29.42 drivers
3DMARK2001 PROJECT FILE

PROJECT
Name NVIDIA GeForce4 Ti 4600 1024x768x32 NoAA Z24 DXTC DB D3D Pure HW
Description
Registration Name xxxxxxxxxx
Registration Key xxxxxxxxxxxxxxxxxxxxxxx
3DMark Version 330

DISPLAY
Platform NVIDIA GeForce4 Ti 4600
CPU Optimization D3D Pure Hardware T&L
Width 1024
Height 768
Depth 32 bit
Z-Buffering 24 bit
Texture Format Compressed
Buffering Double
Refresh Rate 60 Hz
FSAA Mode None

OPTIONS
Show Title Screens No
Continuous Benchmark No
Benchmark Run Count 1
Demo Sounds Enabled Yes
Continuous Demo No
Game Sound Effects Enabled Yes
Game Music Enabled Yes
Game Detail Level Low

RESULTS
3DMark Score 7265
Game 1 - Car Chase - Low Detail 97.0 fps
Game 1 - Car Chase - High Detail 32.5 fps
Game 2 - Dragothic - Low Detail 144.9 fps
Game 2 - Dragothic - High Detail 79.8 fps
Game 3 - Lobby - Low Detail 93.5 fps
Game 3 - Lobby - High Detail 41.9 fps
Game 4 - Nature 41.4 fps
Fill Rate (Single-Texturing) 1032.3 MTexels/s
Fill Rate (Multi-Texturing) 2281.9 MTexels/s
High Polygon Count (1 Light) 38.5 MTriangles/s
High Polygon Count (8 Lights) 12.0 MTriangles/s
Environment Bump Mapping 161.1 fps
DOT3 Bump Mapping 142.5 fps
Vertex Shader 96.3 fps
Pixel Shader 116.9 fps
Advanced Pixel Shader 90.9 fps
Point Sprites 25.3 MSprites/s

3dmark 2k1 build 330 with intro screens using official/whql 29.42 drivers
3DMARK2001 PROJECT FILE

PROJECT
Name NVIDIA GeForce4 Ti 4600 1024x768x32 NoAA Z24 DXTC DB D3D Pure HW
Description
Registration Name xxxxxxxxxxx
Registration Key xxxxxxxxxxxxxxxxxxxxxx
3DMark Version 330

DISPLAY
Platform NVIDIA GeForce4 Ti 4600
CPU Optimization D3D Pure Hardware T&L
Width 1024
Height 768
Depth 32 bit
Z-Buffering 24 bit
Texture Format Compressed
Buffering Double
Refresh Rate 60 Hz
FSAA Mode None

OPTIONS
Show Title Screens Yes
Continuous Benchmark No
Benchmark Run Count 1
Demo Sounds Enabled Yes
Continuous Demo No
Game Sound Effects Enabled Yes
Game Music Enabled Yes
Game Detail Level Low

RESULTS
3DMark Score 7998
Game 1 - Car Chase - Low Detail 110.2 fps
Game 1 - Car Chase - High Detail 34.4 fps
Game 2 - Dragothic - Low Detail 160.5 fps
Game 2 - Dragothic - High Detail 91.7 fps
Game 3 - Lobby - Low Detail 99.7 fps
Game 3 - Lobby - High Detail 44.2 fps
Game 4 - Nature 44.5 fps
Fill Rate (Single-Texturing) 1033.3 MTexels/s
Fill Rate (Multi-Texturing) 2294.4 MTexels/s
High Polygon Count (1 Light) 39.0 MTriangles/s
High Polygon Count (8 Lights) 12.2 MTriangles/s
Environment Bump Mapping 163.3 fps
DOT3 Bump Mapping 144.2 fps
Vertex Shader 97.8 fps
Pixel Shader 118.7 fps
Advanced Pixel Shader 91.0 fps
Point Sprites 29.2 MSprites/s

 

[noko]

I tried something different on this test and used Fraps version 17a. This is what I came up with on the two different configuration with and without splash screens on Game 4 Nature test. I use the Frapslog feature to do a benchmark between 3 and 50 seconds of Game 4 Nature test. Two runs each:

With/Out the splash screens: GAME 4 NATURE
FRAPS
3DMark2001se

2002-06-14 18:22:23 - 3DMark2001SE
Frames: 1447 - Time: 46897ms - Avg: 30.854 - Min: 25 - Max: 55 (3DMark2001se shows 30.5FPS)

2002-06-14 18:27:47 - 3DMark2001SE
Frames: 1447 - Time: 46917ms - Avg: 30.841 - Min: 25 - Max: 55 (3DMark2001se shows 30.5FPS)

With the splash screens: GAME 4 NATURE

2002-06-14 19:14:18 - 3DMark2001SE

Frames: 1897 - Time: 46977ms - Avg: 40.381 - Min: 33 - Max: 56 (3DMark2001se shows 40FPS)

2002-06-14 19:17:38 - 3DMark2001SE
Frames: 1892 - Time: 46857ms - Avg: 40.378 - Min: 33 - Max: 57 (3DMark2001se shows 40FPS)

Fraps and 3DMark2001 agree with each other on both setups. So maybe both is right in this case and 3DMark2001 is fubar . . .

 

[Doomtrooper]

So its not a Geforce 4 specific issue .....

 

[Oompa Loompa]

Where's the hard-nosed investigation? An e-mail that was ignored...and no "red alert" to the community about potential shenanigans? Why not re-name 3DMark, or hack in a different splash screen, look at individual tests, compare FPS numbers with the reported 3D Mark score....

Obviously this "cheat", if it is a cheat, is much more difficult to prove than ATI's Q3A cheat. In that case the situation was visually apparent and easy to understand, and no alternative explanations were possible.

Here, we have a situation where no one has been able to identify a visual defect in 3dMark that might explain where extra FPS were coming from and are left with the possibility that somehow the rendering speed is actually influenced by the 2D overlay. Hacking in a different logo, which is the proper way to investigate this issue, is probably a non-trivial exercise.

I don't really care for HardOCP, but I am not surprised that they cried foul when faced with an easily-proven cheat and am equally unsurprised that they've been less confident when faced with a trickier issue.

By the way, Build 330 of 3dMarkSE has been out for a couple of days now. Hasn't anyone tried it yet? The results from my Radeon VE aren't going to be helpful, but someone with a GF3/4 should pipe up.

 

[Doomtrooper]

Where's the hard-nosed investigation? An e-mail that was ignored...and no "red alert" to the community about potential shenanigans? Why not re-name 3DMark, or hack in a different splash screen, look at individual tests, compare FPS numbers with the reported 3D Mark score....

Obviously this "cheat", if it is a cheat, is much more difficult to prove than ATI's Q3A cheat. In that case the situation was visually apparent and easy to understand, and no alternative explanations were possible.

Here, we have a situation where no one has been able to identify a visual defect in 3dMark that might explain where extra FPS were coming from and are left with the possibility that somehow the rendering speed is actually influenced by the 2D overlay. Hacking in a different logo, which is the proper way to investigate this issue, is probably a non-trivial exercise.

I don't really care for HardOCP, but I am not surprised that they cried foul when faced with an easily-proven cheat and am equally unsurprised that they've been less confident when faced with a trickier issue.

By the way, Build 330 of 3dMarkSE has been out for a couple of days now. Hasn't anyone tried it yet? The results from my Radeon VE aren't going to be helpful, but someone with a GF3/4 should pipe up.

What is the easily proven cheat chocolate dwarf ??