Futuremark is full of shit.How can they bear this?

[engall]

Recently Nvidia released new Official Drivers 45.23.
But it still contains "Specific application Optimization".
Futuremark claimed they couldnt permit this before .
Now 3dmark03 is completely useless even Nvidia have already taken part
in Beta member Because nothing has be changed.

We have never permitted 3DMark specific optimizations, and we still don't do that. Currently we are working with our development program members to form rules about what type of driver optimizations are permitted for 3DMark, and we are trying to get them published ASAP. I don't wan't to go into details until it's ready, but one thing is clear, benchmark identification and benchmark specific optimizations are not permitted.

I know the 45.23 dets were marketed as 'Futuremark approved'. The truth is that we studied the optimizations in those drivers and informed Nvidia of a number of such we were not pleased with.

That seems ridiculous.

studied the optimizations

You mean 3dmark03 can be optimized,dont you?
If so,how can you say:

We have never permitted 3DMark specific optimizations, and we still don't do that.

If not, why do you study that optimizations?
That is very obvious that what FM should do if they find optimizations.

 

[digitalwanderer]

We have never permitted 3DMark specific optimizations, and we still don't do that.

If not, why do you study that optimizations?
That is very obviously that what FM can do if they find optimizations.

Yeah, they can stomp their little foot and make mad faces at nVidia...but still allow the optimizations.

 

[Bjorn]

You mean 3dmark03 can be optimized,dont you?
If so,how can you say:

We have never permitted 3DMark specific optimizations, and we still don't do that.

If not, why do you study that optimizations?
That is very obvious that what FM should do if they find optimizations.

There is a difference between optimizations and 3D Mark specific optimizations. F.e, optimizations that doesn't apply only to 3D Mark and also doesn't need to detection 3D Mark to work. Doesn't automatically mean that the optimizations are valid though which FM seems to talk to Nvidia about with regards to the 45.23 drivers. I'm guessing that they might have validated it a bit prematurely.

 

[Joe DeFuria]

I'm guessing that they might have validated it a bit prematurely.

Yes, unfortunately, as in "the sooner we agree to validate this, the sooner we get the beta membership fees from nVidia."

I mean this statement:

I know the 45.23 dets were marketed as 'Futuremark approved'. The truth is that we studied the optimizations in those drivers and informed Nvidia of a number of such we were not pleased with.

Is very, very, disturbing.

 

[g__day]

Just when I thought the dust was settling, Rage3D really puts the boot in today about FutureMark isn't now really a benchmark if its allowing any shader substitution, because it just isn't measuring apples vs apples.

http://www.rage3d.com/articles/futuremark/

 

[engall]

You mean 3dmark03 can be optimized,dont you?
If so,how can you say:

We have never permitted 3DMark specific optimizations, and we still don't do that.

If not, why do you study that optimizations?
That is very obvious that what FM should do if they find optimizations.

There is a difference between optimizations and 3D Mark specific optimizations. F.e, optimizations that doesn't apply only to 3D Mark and also doesn't need to detection 3D Mark to work. Doesn't automatically mean that the optimizations are valid though which FM seems to talk to Nvidia about with regards to the 45.23 drivers. I'm guessing that they might have validated it a bit prematurely.

Look at THG Test.link

http://www.tomshardware.com/graphic/20030818/detonator-20.html
With the new driver, the scores in 3D Mark 2003 increase dramatically

Performance-wise, we weren't able to determine any increase with the new driver except in 3D Mark 2003 and Unreal Tournament at 4xFSAA and a 1600x1200 resolution. Instead, the driver seemed to be somewhat slower with GeForce FX cards, while this is insignificant at values of 1 to 3 fps. There were also no differences worthy of mention with the GeForce 4 Ti.

Come on,guys. What about that? 3DMark specific optimizations！
Thats it！

 

[engall]

Q: Does this mean what you called originally as "cheats" actually were acceptable "optimizations", and that you made a wrong decicion in releasing Patch 330 and the Audit Report?

A: By the definition of our benchmark and process, the optimizations are not acceptable. 3DMark scores are only comparable if drivers perform exactly the work 3DMark instructs them to do.

As earlier stated, we recommend using the latest build 330 of 3DMark03, with the 44.03 (or 43.51 WHQL) Nvidia drivers, or the Catalyst 3.4 ATI drivers. This way obtained 3DMark03 results are genuinely comparable as far as we know.

Patric Ojala

 

[Pete]

I know the 45.23 dets were marketed as 'Futuremark approved'. The truth is that we studied the optimizations in those drivers and informed Nvidia of a number of such we were not pleased with.

Patric, FM's private correspondence with nVidia is useless to potential consumers who are reading published 3DM03 benchmark results as we speak. Why not also place a notice of your misgivings on ORB's results pages for FX cards, or ask reviewers to do so when they publish 3DM03 numbers with unapproved (or approval pending) drivers?

 

[digitalwanderer]

I know the 45.23 dets were marketed as 'Futuremark approved'. The truth is that we studied the optimizations in those drivers and informed Nvidia of a number of such we were not pleased with.

Patric, FM's private correspondence with nVidia is useless to potential consumers who are reading published 3DM03 benchmark results as we speak. Why not also place a notice of your misgivings on ORB's results pages for FX cards, or ask reviewers to do so when they publish 3DM03 numbers with unapproved (or approval pending) drivers?

Probably because nVidia wouldn't like it and might not keep sending the checks.

 

[Deathlike2]

Well.. it's a confirmation of something (although 3DMark isn't really much of a reliable benchmark anymore.... to gamers especially).

 

[StealthHawk]

There is a difference between optimizations and 3D Mark specific optimizations. F.e, optimizations that doesn't apply only to 3D Mark and also doesn't need to detection 3D Mark to work. Doesn't automatically mean that the optimizations are valid though which FM seems to talk to Nvidia about with regards to the 45.23 drivers. I'm guessing that they might have validated it a bit prematurely.

I don't see what all the confusion is. I should be apparent to everyone that in current drivers NVIDIA is boosting their scores by specifically targeting 3dmark03, same as they have been doing for some time now. Whether or not 3dmark scores increases through general optimizations with a future driver like Det50 is irrelevant. Futuremark should not have allowed scores to be posted by questionable drivers which break the rules.

 

[Pete]

Probably because nVidia wouldn't like it and might not keep sending the checks.

Um, yeah--I forgot about FM's compromising position.

Well, another reason to visit informed 3D sites, I suppose.

 

[Bouncing Zabaglione Bros.]

Q: Does this mean what you called originally as "cheats" actually were acceptable "optimizations", and that you made a wrong decicion in releasing Patch 330 and the Audit Report?

A: By the definition of our benchmark and process, the optimizations are not acceptable. 3DMark scores are only comparable if drivers perform exactly the work 3DMark instructs them to do.

As earlier stated, we recommend using the latest build 330 of 3DMark03, with the 44.03 (or 43.51 WHQL) Nvidia drivers, or the Catalyst 3.4 ATI drivers. This way obtained 3DMark03 results are genuinely comparable as far as we know.

Patric Ojala

And yet you allow Nvidia to post "not acceptable" score on the ORB and say they their drivers are "Futuremark Approved" in public. When are *YOU* going to post on *YOUR* website what you have just written above?

3Dmark2003 has lost all credibility, not because of the benchmark, but because Futuremark as a company will not stand up and fight for it's credibility as a benchmarking tool. You allow Nvidia to do what it wants, and says what it wants about your software, and now you are tarred with the same brush of dishonesty.

 

[Nick[FM]]

You might be interested in a post over at Rage3D by Tero Sarkkinen (EVP Sales and Marketing, Futuremark). It's located here.

 

[Bouncing Zabaglione Bros.]

]You might be interested in a post over at Rage3D by Tero Sarkkinen (EVP Sales and Marketing, Futuremark). It's located here.

And yet Nvidia publicly state their latest cheating drivers are "Futuremark approved" and Futuremark say NOTHING in public to refute that.

 

[Patric Ojala]

And yet Nvidia publicly state their latest cheating drivers are "Futuremark approved" and Futuremark say NOTHING in public to refute that.

We have informed Nvidia that we are not pleased with how they claim their latest drivers are Futuremark approved, even though they are not. As I already wrote, we have given them a list of the optimizations in the 45.23 drivers we consider inappropriate.

I know it would be pleasant reading for enthusiasts if we would follow up our public disagreements from last spring, but to be honest, we're sick of this kind of pointless arguing. We would like to finally get a chance to concentrate on the next 3DMark version, instead of spending all our time publishing statements that in the end benefit nobody.

 

[Doomtrooper]

instead of spending all our time publishing statements that in the end benefit nobody

That statement is kind of strange considering the average consumer should be informed that their favorite print mag is posting benchmarks inflated by 'invalid optimizations' or as I call it 'cheating'. PC Gamer and other popular mags use 3Dmark alot too.

In the end it is simply blatant lying to the consumer, tricking them into buying a product that isn't performing.

 

[Bouncing Zabaglione Bros.]

And yet Nvidia publicly state their latest cheating drivers are "Futuremark approved" and Futuremark say NOTHING in public to refute that.

We have informed Nvidia that we are not pleased with how they claim their latest drivers are Futuremark approved, even though they are not. As I already wrote, we have given them a list of the optimizations in the 45.23 drivers we consider inappropriate.

I know it would be pleasant reading for enthusiasts if we would follow up our public disagreements from last spring, but to be honest, we're sick of this kind of pointless arguing. We would like to finally get a chance to concentrate on the next 3DMark version, instead of spending all our time publishing statements that in the end benefit nobody.

A public rebuttel from Futuremark to counter the (incorrect) public statement from Nvidia seems absolutely necessary. You have told Nvidia you are "not pleased" and Nvidia have done NOTHING to publicly withdraw that statement. How does that make Futuremark look? Weak and innefectual, pandering to those that send them cheques regardless of what rules are set or broken in the 3DMark programme.

Futuremark have done nothing to help your customers by informing them of the facts. You might think this would "benefit nobody" but it would benefit your customers who now incorrectly think that Nvidia's cheating drivers are "Futuremark approved" and it would benefit Futuremark by showing the public that you are prepared to stand up for the integrity of your benchmark - something you won't do.

If you won't publicly support your product in the face of Nvidia cheating and incorrect PR statement, why should I support it as a customer? Why should I or anyone else place any stock in 3DMark's results when the Futuremark won't even bother to stand up for the benchmark in public, and allows Nvidia to get away with cheating, submitting cheating scores, and then stating publicly that the Futuremark approve of those cheats?

 

[Nick[FM]]

Bouncing Zabaglione Bros. & co.,

Did you guys even read Tero's post over at Rage3D? If not, please read it carefully. As Tero wrote in his post over there:

This work has been time consuming and the issues are very complex and we have not wanted to comment on work in progress. We would like to apologize the time it has taken, but it has been time well spent and you will see the results soon.

Also:

We will publish clarifications and ground rules for our product usage on September 19th.

 

[Doomtrooper]

I read it, but found the comment "instead of spending all our time publishing statements that in the end benefit nobody" simply mind boggling.
Benchmark software lives and dies on 'credability', I would hope Futuremark 'knows' that :?