3DMark 2001SE behaving strange!

R

rAvEN^Rd

Newcomer
I wanted to calculate the standard deviation of some runs with 3DMark 2001 (build 330) to gain some knowledge about the insecurity of my benchmarks. After a few runs I noticed a disturbing pattern. It seems like performance degrades with every consecutive run. I have run 10 consecutive benchmarks in two different sets with a reboot in between.

I got these results
Set 1
9484 -> 9265 -> 9209 -> 9151 -> 9123 -> 9128 -> 9119 -> 9110 -> 9099 -> 9089

-REBOOT-

Set2
9459 -> 9230 -> 9205 -> 9165 -> 9139 -> 9117 -> 9109 -> 9111 -> 9101 -> 9084

I compiled the data into a spreadsheet which I have converted to PDF. The data should be easier to overview.

http://www.efd.lth.se/~d00ba/3dMark.pdf

Am I the only one that experiences this phenomenon?

I ran the benchmarks on this platform:
Epox KT333 - AMD Palomino 2000+ - 2*128 MB Kingston PC2100 - MSI G4Ti4400-VTD - SB Live! - Detonator 29.42 - VSync off - NO FSAA - 1024x768
 
Sounds like either a memory leak, or a memory fragmentation issue.
I.e. textures etc can no longer be loaded into the most optimum place within memory.
Generally benchmarks are done on a fresh reboot to avoid these issues.
Looking at the figures they seem to degrade on both sets at approx the same rate.
CC
 
It used to happen with the 8500 and was one of the many issues that everyone put under the 'high poly bug'. The 3dmark decreases were fixed in a driver update.
 
More importantly, does it imply a problem with:
  • Direct X getting slower OR
  • The particular graphics card/driver (I think you wrote GF4 (it's not showing up in the editting window) OR
  • Something else?
If it's the first one, then this implies those fanatical about a game's performance need to reboot before relaunching.

If it's the second one, then there may be problems with that card's driver.
 
Yes, it was MSI GeForce4 Ti4400 with Detonator 29.42

I believe I have just seen similar results with POV Ti4200SE (3,3 ns) on the same platform with the same drivers. I don't have time to run another 20 benchmarks so I can't tell for sure.
 
This is strange. Not just because the 10 benches (before rebooting) get slower and slower as you go. But especially that even after rebooting and doing another 10 tests each test is slower then the corresponding test before you rebooted.. that is very od indeed.
 
Teasy said:
This is strange. Not just because the 10 benches (before rebooting) get slower and slower as you go. But especially that even after rebooting and doing another 10 tests each test is slower then the corresponding test before you rebooted.. that is very od indeed.
Click to expand...
4 out of the 10 tests in the second set are faster than the corresponding test before he rebooted, the overall slightly lower scores are all within the margin of error IMHO. I don't think we're gonna encounter an "oh my god my card gets slower with every reboot" scenario here... :)
 
The computer wasn't rebooted just before the first test. We must also remember that there is a matter of insecurity with these tests. It may be coincidense that the second set is somewhat slower than the first.
 
Anyone knows exactly how the final "3DMark" is calculated? Obviously it is based on each test's framerate (and other synthetic stuff like fillrate and poly throughput) but how?
 
3DMark score = (game 1 low detail + game 2 low detail + game 3 low detail) * 10 + (game 1 high detail + game 2 high detail + game 3 high detail + game 4) * 20

Synthetic tests do not contribute to the overall score.
 
4 out of the 10 tests in the second set are faster than the corresponding test before he rebooted, the overall slightly lower scores are all within the margin of error IMHO
Click to expand...

Ah yeah I didn't notice that. I just looked at the first few after reboot, saw that they were lower and thought it was a trend, I didn't look closely at the rest of them :oops:
 
yeah but the runs are so similar in result and and the detoriating trend is so similar the actual few points up or down dont matter between the same comparitive run points.

I always understood to set a 3dmark you shoudl run it twice anyway :)

I never get identical results between runs. As long as they are near I'm happy enough.
 
rAvEN^Rd said:
I wanted to calculate the standard deviation of some runs with 3DMark 2001 (build 330) to gain some knowledge about the insecurity of my benchmarks. After a few runs I noticed a disturbing pattern. It seems like performance degrades with every consecutive run. I have run 10 consecutive benchmarks in two different sets with a reboot in between.

I got these results
Set 1
9484 -> 9265 -> 9209 -> 9151 -> 9123 -> 9128 -> 9119 -> 9110 -> 9099 -> 9089

-REBOOT-

Set2
9459 -> 9230 -> 9205 -> 9165 -> 9139 -> 9117 -> 9109 -> 9111 -> 9101 -> 9084
Click to expand...
Try closing the application between runs.
Am I the only one that experiences this phenomenon?
Click to expand...
No. :)
 
So I don't get a million bucks and the Nobel price for my amazing discovery? I have already started making T-shirts and posters with my face on it...
 
rAvEN^Rd said:
So I don't get a million bucks and the Nobel price for my amazing discovery? I have already started making T-shirts and posters with my face on it...
Click to expand...
/em hands rAvEN^rd a cookie. :)
 
Not trying to instigate anything, but do the same thing you did, except turn off splash screens. It might be interesting to see how the results differ. It might help to locate which optimizations are being done and on which test.
 
LittlePenny said:
Not trying to instigate anything, but do the same thing you did, except turn off splash screens. It might be interesting to see how the results differ. It might help to locate which optimizations are being done and on which test.
Click to expand...
???
The slowdown is caused by 3D Mark 2001 itself: It happens on all cards. The only way to get consistent results is to close the application, or reboot, before each run.
 
You must log in or register to reply here.
Back
Top