Futuremark Announces Patch for 3DMark03

[Som]

So 99% of consumers deserve to be ripped off simply because they are not among B3D readership?

Well....yeah. That's pretty much my take on it. Besides, what serious gaming enthusiast buys from an OEM?

 

[Xmas]

I'm interested in what exactly FM changed in the new build. If they changed some of the rendering code, it's not certain that the performance drop can be wholly attributed to circumventing application-specific optimizations.

If the changes were significant, then the Radeons would show differences as well.

You mean, you only consider them significant if they affect the Radeons?

Even simple code reordering can make a driver choke on your application, but that doesn't mean all drivers will.

 

[AzBat]

So 99% of consumers deserve to be ripped off simply because they are not among B3D readership?

Well....yeah. That's pretty much my take on it. Besides, what serious gaming enthusiast buys from an OEM?

Remind me to never ask you for advice. That's messed up man.

Tommy McClain

 

[Dave Baumann]

Parhelia shows no performance differences either.

Wait up a while, I do have some images differnces between the two patches on from NVIDIA.

 

[nelg]

What about before they were available? Well, in that case you're just guessing really. 3dmark came out before the first DX9 games, fair enough, but since it is synthetic, there's no telling how it would tally with real world performance, so all you're really doing is making a slightly more educated guess than without consulting 3DM03.

Actually I feel that 3Dmark03 Gave a pretty good prediction of how well ATI’s and nV’s cards would perform in Dx9 games (in a general sense). And as Joe mentioned in the absence of any other ways to predict it should be commended.

 

[Althornin]

Or, you are missing the boat, because the 52.16 series driver DOES NOT CHEAT (has no driver optimizations that break guidelines) in 3dmark03 patch 340.

Seems to me that someone is probably missing the shore here...

From the results, it looks like the 52.16 drivers cheatin 3DMark, and do not comply with FM's rules. The cheats are then made irrelevant by patch 340. Or do you think there is a specific code path in the driver that says "If 3DMark version > 330, disable cheat" ?

please re-read this thread, then repond again, thanks.

 

[Joe DeFuria]

That's not the situation I'm discussing though - you're shifting off.

Eh? You asked why some things are important, "what difference does it make" and I told you why.

But I'm not sure that them trying to create as much performance as possible for their customers by putting in app specific code for alot of programs is cheating so much as support.

You're still not "getting it."

Putting in app specific code (even if it's GOOD app specific code) is only good FOR THOSE APPS.

nVidia has a vested interest in specifically "supporting" any apps that are used as BENCHMARKS.

I want to know what happens if my game doesn't get "support?!". 3DMark (patched) is an indication of this.

GAMES are generally NOT going to take measures to "defeat" optimizations. (Though we do hear about some developers bitching and moaning about them...see Gabe Newell.) Such optimizations can ultimately be a "benefit", or they can be a detriment. (Specifically, if such optimizations "break" whenever the game developer issues new code, new shaders, etc.)

So then, is it only "cheating" when it's done for a benchmark, or is it just goo optimization when it's done for something such as BF1942 or UT2003?

It's cheating when the developer forbids you to do it, and you do it anyway. It's also cheating when the end user doesn't get the result that the developer intends you to get.

 

[CorwinB]

So then, is it only "cheating" when it's done for a benchmark, or is it just goo optimization when it's done for something such as BF1942 or UT2003?

Well, improving performance only for a benchmark is definitely "cheating". Apart from that, there is also the very important matter of respecting developer/user settings. For example, as of now it is totally impossible, on the GFFX, to get full trilinear filtering in Direct3D. You may or may not object that "the quality is nearly the same", but still, when the user selects "application setting filtering", and the application specifies "trilinear", and you don't get trilinear, then something is seriously rotten. That's bad for benchmarking (since it's an apple to orange comparison), and of course bad for the end-user because you take choice away from him.

If Nvidia was to offer a "fast trilinear" option in the drivers settings in addition to a truly working "application setting", then it would be a gain for the consumer, and that would be providing support. But right now, it's only misleading marketing. Remember that at first this "optimization" was only used when the application UT2003 (widely used benchmark) was detected...

 

[Bouncing Zabaglione Bros.]

So then, is it only "cheating" when it's done for a benchmark, or is it just good optimization when it's done for something such as BF1942 or UT2003?

When you run a game the card should:

1. Do the work specified
2. Output the image as requested.

Optimisations that produce the same calculations and the same IQ are valid (such as reordering shader instructions). Nvidia does not do this (static clip planes, low quality shader replacement, bilinear filtering when trilinear is requested).

When you run a benchmark, the card should:

1. Do the work specified.
2. Do the work in the way specified.
3. Output the image as requested.

Again Nvidia do not do this.

Optimisations (even by Nvidia's own internal documents as well as in the opinions of developers) should not be specific to one app, should not degrade image quality, and should be generally useful to all applications that use those techniques. Nvidia has time and time again broken these rules, both within games and particularly with game benchmarks. They have even gone out of their way to target any benchmarks that have come to prominence over the last year if it shows Nvidia hardware underperforming.

 

[Althornin]

I'm not arguing with what you intended to say, but I think it's worth pointing out that Nvidia may not successfully cheat in 340, but they do *attempt* to cheat. In 330, Nvidia attempts to cheat and succeeds. After seeing these cheats, Futuremark blocked them in 340, so that the same cheats now fail. In both cases the *intention* and code from Nvidia is to artificially inflate the benchmark scores.

You may say the Nvidia do not cheat on 340 because they are not able to, but this is not due to the honesty or intention of Nvidia, but the (currently) successful attempts at policing by Futuremark.

total agreement. Doesnt change things from Futuremarks guide viewpoint though - nVidia isnt currently cheating....

 

[Doomtrooper]

Even simple code reordering can make a driver choke on your application, but that doesn't mean all drivers will.

In this case it is a range of drivers 44.67-->52.16, which also 'just happens' to be drivers that give impressive 3Dmark score boosts....now who would have thought

 

[Bouncing Zabaglione Bros.]

total agreement. Doesnt change things from Futuremarks guide viewpoint though - nVidia isnt currently cheating....

As far as Futuremark can tell. How do we know some cheats haven't sneaked past FM? Just like the battle between hackers and security, you *think* your computer is safe until it turns out that you've been hacked the whole time.

It's crazy that Nvidia are again part of the Futuremark programme, and yet are still trying to circumvent honest results. FM has to spend time and money defending against these hacks from one of their own members. How can we trust the results (or Nvidia) when we know Nvidia, with all it's multi million dollar resources is actively trying to hack the benchmark results?

 

[digitalwanderer]

Parhelia shows no performance differences either.

Wait up a while, I do have some images differnces between the two patches on from NVIDIA.

Thanks for the Parhelia info Dave and I look forward to hearing what is up with your image comparison.

"So, if Nvidia is running "non-standard" code (like that exists or something) in the background to achieve their results, WHO CARES? We know what we're getting when we buy, regardless of whether or not we truly know how we get it."

This is the second time in 10 minutes I've seen this same assignine argument used by a one-post-wonder. I was gonna call you 'paranoid', but...

(The Dig pulls out his trusty tinfoil hat and carefully unfolds it and places it firmly on his head making sure it completely encases his brain.)

Nice use of out of context quotation. The point is, we have real world performance tests. If you're getting good performance in actual game play, why bother looking at synthetic benchmarks?

I don't see how I used that out of context, I thought it was bang-on ON context. Your whole comment seemed to imply that FutureMark was somehow unfairly targetting nVidia, which just ain't the case.

And it's "assinine" I believe. Not sure m'self though, to be honest.

Yeah, I think you're right...my spelling is atrocious.

 

[BetrayerX]

What card would you buy?

Why buy a card now for DX9 games? If you're going to upgrade hardware for a specific game or games, you wait until the first is out to purchase.

I thought that was obvious.[/quote]

Hmmm....I suppose Halo and Tomb Raider don't count there.

Not every one upgrades each 6 months...some people do it yearly, some each 18 months and some whenever they care to.

My point is, maybe when the Radeon came out and 6 months after that there was nor a single DX9 game, but not everyone upgraded to obtain DX9 performance. I upgraded from a Kyro 2 and when I got the 9700 it was simply the best in the market. I wasn't expecting DX9 games, but to run all the games I wanted in a proper way. Most people upgrade in a longer cycle, so many would be upgrading from GF3, 8x00, Kyro and GF4MX cards. From there a Radeon looked like an awesome upgrade.

Having that said, if you buy a card to keep it there for a long time, it'll be wise if it might support future software so it won't die as fast. DX9 games are finally starting to arrive and it's good to know an "old card" still runs fine.

 

[Oblivious]

Som,

Also consider Microsoft's new Game Advisor initiative. They are licensing Futuremark's technology to recommend video cards for games. If Nvidia continues to release drivers that artificially inflate 3DMark scores, this could affect MS, Futuremark, OEMs, and all of the potential customers who use Game Advisor.

 

[Bambers]

As far as Futuremark can tell. How do we know some cheats haven't sneaked past FM? Just like the battle between hackers and security, you *think* your computer is safe until it turns out that you've been hacked the whole time.

Considering the PS2.0 test hasn't dropped at all between 330 and 340 on the 52.16 drivers I think they have missed one or two... :|

 

[digitalwanderer]

Som,

Also consider Microsoft's new Game Advisor initiative. They are licensing Futuremark's technology to recommend video cards for games. If Nvidia continues to release drivers that artificially inflate 3DMark scores, this could affect MS, Futuremark, OEMs, and all of the potential customers who use Game Advisor.

Well wouldn't it be just as fair to say that by lowering nVidia's score that they are negatively affecting nVidia?!?! (Careful! It's a trick question... )

 

[Oblivious]

Som,

Also consider Microsoft's new Game Advisor initiative. They are licensing Futuremark's technology to recommend video cards for games. If Nvidia continues to release drivers that artificially inflate 3DMark scores, this could affect MS, Futuremark, OEMs, and all of the potential customers who use Game Advisor.

Well wouldn't it be just as fair to say that by lowering nVidia's score that they are negatively affecting nVidia?!?! (Careful! It's a trick question... )

That's true... until their next drivers

Seriously, is it really unfair that Nvidia's hardware is now properly reflected in their scores?

Edit: Whoops, didn't see that line there. Shoot, I fell for your trick

 

[Neeyik]

As far as Futuremark can tell. How do we know some cheats haven't sneaked past FM? Just like the battle between hackers and security, you *think* your computer is safe until it turns out that you've been hacked the whole time.

Considering the PS2.0 test hasn't dropped at all between 330 and 340 on the 52.16 drivers I think they have missed one or two... :|

Either that or the compiler is actually doing what it's supposed to do...

 

[Doomtrooper]

The problem I have is still allowing the 'driver' to be approved...this may be another case of LawyerMark.

LawyerMark 2003
                           =
                           =
                           =
   =                       =
   =                       =
Futuremark              Nvidia