Futuremark is full of shit.How can they bear this?

[AzBat]

And yet Nvidia publicly state their latest cheating drivers are "Futuremark approved" and Futuremark say NOTHING in public to refute that.

We have informed Nvidia that we are not pleased with how they claim their latest drivers are Futuremark approved, even though they are not. As I already wrote, we have given them a list of the optimizations in the 45.23 drivers we consider inappropriate.

This is exactly the kind of thing you need to publicly address. Posting stuff like this in public forums does not cut it. It needs to be an official statement made by the company on its web site and not buried either.

I know it would be pleasant reading for enthusiasts if we would follow up our public disagreements from last spring, but to be honest, we're sick of this kind of pointless arguing.

It amazes me that you Finnish people still don't get it. IT'S NOT POINTLESS TO THE END USERS.

We would like to finally get a chance to concentrate on the next 3DMark version, instead of spending all our time publishing statements that in the end benefit nobody

Boy, that was the most moronic statement I have ever read.

Markus, I know you're reading this. It's statements like these that show me that Futuremark have lost its integrity.

There's no way in hell people are going to just forget about the problems with the current version of 3DMark if you work on and release a newer version. Do you not get it? Your integrity and trustworthiness is at stake here. If you fail to fix and respond to the problems that NVIDIA has created with the current 3DMark your company will DIE. Nobody will ever trust you no matter if you release a new or not. Making public statements refuting NVIDIA is not pointless as it benefits the people that should matter, us end-users. If we can't trust you or that you're doing the right thing then we will not use your products.

All I've got to say is that come September 19, Futuremark had better be acknowledging that NVIDIA and its drivers not living up to your rules and guidelines and that their scores will not be supported. If you end-up releasing a crap set of rules and then don't have the back-bone to enforce them, then be ready to pack-up your office belongings and find another job because I don't think your company will be around much longer.

Tommy McClain

 

[Mark0]

Maybe they are simply hoping that, with all the current "fireworks" (!) for the Half Life 2 bechmark's results, the days from here to 19th could pass without much attention to the "old 3DMark Vs. nVidia" affair.

Sad but possible... maybe?

Bye!

 

[Wolf]

They have already lost all credibility many months ago for those that can read between the lines. I sent Futuremark an email stating I would never run their program again. I deleted all my entries from 3dmark2001 and 2003 which consisted of years of configs and runs off the site and deleted the programs. I will not use a biased benchmark period. I have also let anyone who mentions their benchmark know how useless it really is.

 

[Bouncing Zabaglione Bros.]

With the new HL2 benchmark, 3Dmark2003 will be out in the cold. Not because of the benchmark itself (which will be vindicated by the HL2 benchmark showing the same deficiencies in cards like NV30 that 3DMark2003 shows), but because of the way Futuremark caved into Nvidia and refuse to support or defend 3DMark2003.

 

[digitalwanderer]

With the new HL2 benchmark, 3Dmark2003 will be out in the cold. Not because of the benchmark itself (which will be vindicated by the HL2 benchmark showing the same deficiencies in cards like NV30 that 3DMark2003 shows), but because of the way Futuremark caved into Nvidia and refuse to support or defend 3DMark2003.

Yup, I fully agree. I'm liking the way Valve is sticking to it's guns and not wavering, it's a refreshing change from "well, these things take time to look into and just because nVidia is saying we said something we didn't say doesn't mean we should call them out on it..." crap.

 

[Joe DeFuria]

All I've got to say is that come September 19, Futuremark had better be acknowledging that NVIDIA and its drivers not living up to your rules and guidelines and that their scores will not be supported. If you end-up releasing a crap set of rules and then don't have the back-bone to enforce them, then be ready to pack-up your office belongings and find another job because I don't think your company will be around much longer.

Come on Tommy, tell us how you really feel.

Again, it all comes back to the issues you highlighted with their business model.

Valve is sticking to their guns. They are calling it as they see it. They don't really give a toss if nVidia's hardware just isn't as robust beyond the point that this impacts their customers.

Valve doesn't have to tolerate "cheats", and doesn't have to pretend that coding for NV hardware isn't a hassle. How can they do this? Because nVidia doesn't pay their bills. Consumers do.

Does Valve take money from IHVs? Yup. But that money is not criticial to their business model. That money helps them create a better product, so that increases in the REAL revenue is made from the target audience: the game buying public.

Futuremark needs to decide something: WHO ARE YOUR CUSTOMERS?

I'd say that by definition, your customers are the ones who "pay" for your existence. Right now, that's the IHVs. And yet, your product is suppossed to be a fair and impartial way to judge IHV performance?

Unfortunately, the ones who USE your product, (OEMs and consumers) are not your customers. That's where the fundamental flaw in the business model lies. Because as soon as there is a conflict of interest between what the USERS require, and what your CUSTOMERS want....the "customer is always right." We were all hoping that truism would not actually hold true in the nVidia debacle, and I agree with Tommy...it seems that Sept 19th will be do or die day for Futuremark.

Do you side with your customers (IHVs) ? Or your users? (Consumers, OEMS?)

The end result of siding with customers when there is a conflict? Users lose confidence, and that makes for a lower demand from your users (OEMs / consumers) , which in turn makes your CUSTOMERS (IHVs) see less and less value in paying up.

On the other hand, what if you side with the users?

You get threats and drop outs and refusals to participate from your customers. (See nVidia.)

Damned if you do, and damned if you don't it seems.

It's a vicious circle, and is doomed to failure IMO.

And that's sad. Because I really LOVED the technical aspects, design, goals, and genuine usefulness of the "original 3DMark03." I defended it, and the SCORE, vigorously...before we even knew how the cards would stack up against one another.

It's just frustrating as all hell...because the TombRaider and HL benchmarks should be the VINDICATION of 3DMark, and would have been such vindication had you "stuck to your guns". Not the thorn in your side they appear to be now.

But then, to be fair, I don't know the contents of next week's "announcement." Though whatever it is, I fear for FutureMark that it might be too late.

 

[Bouncing Zabaglione Bros.]

But then, to be fair, I don't know the contents of next week's "announcement." Though whatever it is, I fear for FutureMark that it might be too late.

As you said earlier, the only thing that shifts an incimbent out of top spot is when something else comes along to replace it. Since the announcement of Half-Life 2 DX9 benchmarking with a suite of testing tools, it looks to me that 3DMark2003 has just been replaced by HL2mark.

Futuremark has been replaced by a company who is not afraid to stand up and say that the reason why NV3x benches badly on DX9 code is because it is not a good DX9 card, and that's why Nvidia cheat like a demon on everything they can.

 

[digitalwanderer]

But then, to be fair, I don't know the contents of next week's "announcement." Though whatever it is, I fear for FutureMark that it might be too late.

As you said earlier, the only thing that shifts an incimbent out of top spot is when something else comes along to replace it. Since the announcement of Half-Life 2 DX9 benchmarking with a suite of testing tools, it looks to me that 3DMark2003 has just been replaced by HL2mark.

Futuremark has been replaced by a company who is not afraid to stand up and say that the reason why NV3x benches badly on DX9 code is because it is not a good DX9 card, and that's why Nvidia cheat like a demon on everything they can.

Thirded. Too little, too late Futuremark.

 

[Mark0]

Futuremark has been replaced by a company who is not afraid to stand up and say that the reason why NV3x benches badly on DX9 code is because it is not a good DX9 card, and that's why Nvidia cheat like a demon on everything they can.

Amen!

Bye!

 

[Joe DeFuria]

I just want to reiterate one point.

Half-Life 2 and Tomb Raider should be VINDICATING 3DMark, and Future-mark's benchmarks.

Now , it's very possible that even if 3DMark was consitently showing the "poor" shader performance of NV3x as we believed it should have, that 3DMark03 would STILL be replaced with Half-Life2 as the "PS2.0" benchmark.

Because "real-life" DX9 games in some ways defeat the purpose of a synthetic DX9 test.

But you know what?

When 3DMark04 comes out (presumably with PS/VS 3.0 support), and there are no games on the market with such support, we would all be saying "Futuremark was spot on with their 3DMark03. Their synthetic tests, and the decisions they made when coding it, proved to be borne out when PS 2.0 games actually shipped....they've got a good handle on how these things are actually coded. So we have no reason to doubt the results of 3DMark04 as a guideline for real-world performance with PS/VS 3.0 shaders."

So as long as Futuremark stays ahead of the game development curve, they would have a perfectly viable benchmark to sell.

 

[nelg]

I just want to reiterate one point.

Half-Life 2 and Tomb Raider should be VINDICATING 3DMark, and Future-mark's benchmarks.

Now , it's very possible that even if 3DMark was consitently showing the "poor" shader performance of NV3x as we believed it should have, that 3DMark03 would STILL be replaced with Half-Life2 as the "PS2.0" benchmark.

Because "real-life" DX9 games in some ways defeat the purpose of a synthetic DX9 test.

But you know what?

When 3DMark04 comes out (presumably with PS/VS 3.0 support), and there are no games on the market with such support, we would all be saying "Futuremark was spot on with their 3DMark03. Their synthetic tests, and the decisions they made when coding it, proved to be borne out when PS 2.0 games actually shipped....they've got a good handle on how these things are actually coded. So we have no reason to doubt the results of 3DMark04 as a guideline for real-world performance with PS/VS 3.0 shaders."

So as long as Futuremark stays ahead of the game development curve, they would have a perfectly viable benchmark to sell.

Here's a suggestion for a new solgan "Futuremark- the first benchmark to show how bad the Nv3.x's would be."

 

[Bouncing Zabaglione Bros.]

So as long as Futuremark stays ahead of the game development curve, they would have a perfectly viable benchmark to sell.

As I've said before, it's no longer an issue with the benchmark itself, it's now the issue of Futuremark not being willing to defend and police it's benchmark in public, and therefore give the public perception that it (and by extension its software) is honest, unbiased, and willing to tell the truth.

What happens come Sept 19th when Futuremark come out and say "Well actually, we have decided that sub-DX9 precision and shader replacements are valid after all because that's what Nvidia convinced us all developers will do" ? Will that make 3DMark2004 valid and give Futuremark their intergity back?

If come Sept 19th Futuremark say "no low precision, no shader replacemet" will anyone care if Nvidia is still cheating like hell and no one cares enough at Futuremark to cancel those results from ORB and come out and say it's "because Nvidia cheats" just like Valve said?

FM had their chance and they blew it. They failed to police their benchmark and caved in under pressure from a moneyed IHV who refused to play by the honest rules. Now their product been superceded by the new HL2 benchmark from Valve - but note that it's not the 3DMark2003 benchmark that had been superceded, it's that Futuremark has been superceded as an honest policer of their benchmarking. Bringing out a new 3DMark2004 benchmark (no matter how good and accurate) won't change the fact that a lot of people believe FM unwilling or unable to keep the IHV's honest by accurately policing the 3Dmark results and telling the truth about what the results mean.

This makes any new benchmark from Futuremark useless because if Nvidia were cheating, people no longer believe FM would do or say anything about it. This brings all results from any FM benchmark under suspicion, and thus unreliable and useless.

 

[K.I.L.E.R]

I think some of you guys should get off Worm's back. He may be a FM employee but that doesn't mean you can harass him.

 

[Bouncing Zabaglione Bros.]

I think some of you guys should get off Worm's back. He may be a FM employee but that doesn't mean you can harass him.

No one's harrasing him. We're discussing the issues. Half these posts arn't even directed at him.

When someone (anyone) let alone an FM employee comes to the board and says it's not worth Futuremark's time to tell their customers when one of their beta members is issuing public statements about Futureamark/3DMark that are complete lies, then *of course* they are going to get taken to task. . As yet the only justification we've had is "wait till the 19th sept".

 

[Quitch]

I just want to reiterate one point.

Half-Life 2 and Tomb Raider should be VINDICATING 3DMark, and Future-mark's benchmarks.

Now , it's very possible that even if 3DMark was consitently showing the "poor" shader performance of NV3x as we believed it should have, that 3DMark03 would STILL be replaced with Half-Life2 as the "PS2.0" benchmark.

Because "real-life" DX9 games in some ways defeat the purpose of a synthetic DX9 test.

But you know what?

When 3DMark04 comes out (presumably with PS/VS 3.0 support), and there are no games on the market with such support, we would all be saying "Futuremark was spot on with their 3DMark03. Their synthetic tests, and the decisions they made when coding it, proved to be borne out when PS 2.0 games actually shipped....they've got a good handle on how these things are actually coded. So we have no reason to doubt the results of 3DMark04 as a guideline for real-world performance with PS/VS 3.0 shaders."

So as long as Futuremark stays ahead of the game development curve, they would have a perfectly viable benchmark to sell.

Actually, what I'll be thinking is "What a waste of money 3DMark 03 was... if FutureMark couldn't stick by their product then and give me a fair benchmark, why would they with this new 04? Urgh, I don't want to waste my money."

Not only that, but I would be highly dubious about any results coming out of the benchmark. It doesn't matter now even if the 19 September says nVidia are cheating, because it'll simply look like they've got the balls to say it now because Valve did. This would leave us in a situation where with 04, people can cheat, right up until the point where a developer steps in.

Sorry, but Futuremark have screwed themselves in my eyes.

 

[StealthHawk]

I think some of you guys should get off Worm's back. He may be a FM employee but that doesn't mean you can harass him.

No one's harrasing him. We're discussing the issues. Half these posts arn't even directed at him.

When someone (anyone) let alone an FM employee comes to the board and says it's not worth Futuremark's time to tell their customers when one of their beta members is issuing public statements about Futureamark/3DMark that are complete lies, then *of course* they are going to get taken to task. . As yet the only justification we've had is "wait till the 19th sept".

Yeah, and what is going to happen on September 19th? Priliminary benchmarks of Det50 show even higher 3dmark03 scores than 45.23 drivers provided! And a huge increase in the PS2.0 test!

Doesn't sound like NVIDIA removed any optimizations to me.

 

[stevem]

No, Nvidia certainly haven't removed optimizations from the 5x.x series... It could actually be seen that they've redoubled their efforts in this regard. It's just that wrt 3DMark03, they no longer do app detection per-se. More like a convoluted shader detection & (re)-optimization. It's the (re) bit the devs are not impressed about as they appear to be very liberal in their interpretation of ISV code...

 

[YeuEmMaiMai]

Do youself a favor and get some balls OK? Or better yet,, start looing for another job as Futuremark is as useles as a fart in a windstorm.


You guys let nVidia bend you over and you got exactly what you deserved and most of us do not feel sorry for your company at all.

The community told you guys but you choose not to listen so now that you have made you bed, you gotta lie in it.


Good luck and hopefully someone in your company will grow some balls and do what is right.

Recently Nvidia released new Official Drivers 45.23.
But it still contains "Specific application Optimization".
Futuremark claimed they couldnt permit this before .
Now 3dmark03 is completely useless even Nvidia have already taken part
in Beta member Because nothing has be changed.

We have never permitted 3DMark specific optimizations, and we still don't do that. Currently we are working with our development program members to form rules about what type of driver optimizations are permitted for 3DMark, and we are trying to get them published ASAP. I don't wan't to go into details until it's ready, but one thing is clear, benchmark identification and benchmark specific optimizations are not permitted.

I know the 45.23 dets were marketed as 'Futuremark approved'. The truth is that we studied the optimizations in those drivers and informed Nvidia of a number of such we were not pleased with.

 

[RussSchultz]

Yeah, and what is going to happen on September 19th?

Ooh ooh! I know what's happening on the 18th....

 

[digitalwanderer]

Yeah, and what is going to happen on September 19th?

Ooh ooh! I know what's happening on the 18th....

What?