Will custom timedemos stop benchmark "cheating" ?

[Blastman]

Looking at this article … Disabling Application Detection by Reverend, using the RivaTuner's Anti-Detector it is apparent that the custom timedemo for UT2003 that Reverend used was detected and … “optimized” … for. What is going on here?

A few thoughts. When I looked at this ukgamer review they provided benchmarks for several quality settings on the cards, High-performance, Performance and Quality. Needless to say there quite a large difference in frame rates between the settings … UT2003 - dm-suntemple … 5900U … fps…

5900U … … 150.3 … High-performance
5900U … …126.8 … Performance
5900U … …107.3 … Quality

9800pro … … 136.2 … Performance
9800pro … … 116.7 … Quality

Given that Nvidia’s drivers are overriding the user control panel settings for AF for some games, it would certainly be an easy task to override the … Quality/performance settings also. UT2003 detected! --- lets just dial down the Quality settings for 20% better performance. So much for… “playing it as it was meant to be played” …here. Sacrifice IQ on the 10 or 15 common games used for benchmarking for better frame rates. Why dial in Quality at 107.3 fps and loose in the benchmarks? Bump it up to 125.0 and look good in the benchmarks. Who notices the IQ difference anyway. Of course any custom timedemo will suffer the same fate as soon as the application is detected. I wonder if renaming the executables would make a difference here.

Overriding and dialing down the game control panel settings would also be a good trick to boost performance. There are a lot of settings here that can be overridden like: Object Detail, Terrain Detail and Water Quality, but the main one is probably the Texture Quality. Both these methods of dialing down IQ would certainly be cheats as the ATI card would be rendering higher quality textures.

 

[Dave Baumann]

http://www.beyond3d.com/forum/viewtopic.php?t=6719

 

[Blastman]

Yes, I saw that thread. This goes into 2 other points.

1) If custom timedemos are enough.
2) variations on overriding user settings.

EDIT: Hmmm, after reading through that thread this looks like a duplication. You can delete it if you wish.

 

[Reverend]

No, custom timedemos will not be enough to totally wipe out "cheating", although it does help. The custom UT2003 timedemo I made didn't prevent that because NVIDIA makes specific changes when it recognizes UT2003 -- the game, not just timedemos -- is run.

It is my believe that NVIDIA will most probably not change at all (i.e. they'll continue to do what they're currently doing) because the current crop of "journalist-like" websites (like us, or others) are like flies to them. Until big media outlets start saying "This should not be happening", this is unlikely to change. Sadly. Folks (consumers, OEMs, board partners) are still buying NVIDIA-based cards by the bucket loads despite the 3DMark03 fiasco and they will continue to do so even with this UT2003-specific non-conformance of filtering.