Whatever happened with Futuremark and nVidia with 53.03?

Has anyone seen a review with the 53.03 drivers benched in 3dm2k3 yet? (I can't recall)

Well that the thing isnt it, with all the cheating in 3d mark 03 Nvidia have been doing even with drivers that are possibly legitimate using the newest version of the program reviewers dont want to use it anymore. Looks like Nvidia won :oops:
 
dan2097 said:
Has anyone seen a review with the 53.03 drivers benched in 3dm2k3 yet? (I can't recall)

Well that the thing isnt it, with all the cheating in 3d mark 03 Nvidia have been doing even with drivers that are possibly legitimate using the newest version of the program reviewers dont want to use it anymore. Looks like Nvidia won :oops:

It's a very phyric victory. Look at how many cards Nvidia sold on the back of 3DMark2001 - how are they going to do that when no one trusts FM or Nvidia anymore?

In the long run FM may be dead in the water, but in killing 3DMark off, Nvidia have gained the reputation of being untrustworthy cheaters and liars in all forms of benchmarking, as well as destroying one of their (previously) favourite marketing tools.
 
It's a very phyric victory. Look at how many cards Nvidia sold on the back of 3DMark2001 - how are they going to do that when no one trusts FM or Nvidia anymore?

In the long run FM may be dead in the water, but in killing 3DMark off, Nvidia have gained the reputation of being untrustworthy cheaters and liars in all forms of benchmarking, as well as destroying one of their (previously) favourite marketing tools.

Futuremark isnt dead yet, they still plan to make 3d mark 04/5/whatever. Im sure nvidia will support it if their current top of the line card is shown in a good light in it ;)

EDIT: the general public may believe Nvidia otpimizes specifically/cheats on 3d mark 03 but most do not know of any other cheats. i.e. that they app detect code creatures, that they perform best in popular time demos rather than actual gameplay etc.

Just as a by the by do most sites use the halo built in benchmark for benchmarking, as most web sites show the fx 5950 and 9800XT equal on halo while hardocp which explicity say they use a custom demo shows a large performance difference? :?
 
dan2097 said:
Just as a by the by do most sites use the halo built in benchmark for benchmarking, as most web sites show the fx 5950 and 9800XT equal on halo while hardocp which explicity say they use a custom demo shows a large performance difference? :?

We do a manual run through in a certain portion of the Assault on the Control room mission using FRAPS. It represents exactly how you play the game.
 
We do a manual run through in a certain portion of the Assault on the Control room mission using FRAPS. It represents exactly how you play the game.

Ok its not a demo per say, but you say its in game and specific to your site. (well you changed it between reviews but you know what I mean)

Do you still have the fx 5950 and 9800XT? If so what is the difference in performance on the halo built in benchmarks, on your in game test there seemed to be a large difference
 
Hanners said:
digitalwanderer said:
Hanners said:
Well, they've added Catalyst 3.10 to the approved driver list, but not the ForceWare 53.03 - I think that should tell you everything you need to know. ;)
No, no it doesn't. I wanna hear about how they're going to pull nVidia into a dark alley and work 'em over with a bloody f-ing rubber hose! :devilish:

They probably should, but again, the fact that they haven't tells you everything you need to know about how FutureMark are going to play this.
"That makes Mr.Panda sad!" :cry:
 
I'd say FM doing a press release restating only the drivers on their approval page are approved (possibly after ATI releases some new ones to justify it?) would do the trick. You don't have to insist the 53.03 are illicit; you just gotta say anything BUT the 52.16 is. I doubt they'd get in as much trouble for that, too.


Uttar
 
dan2097 said:
We do a manual run through in a certain portion of the Assault on the Control room mission using FRAPS. It represents exactly how you play the game.

Ok its not a demo per say, but you say its in game and specific to your site. (well you changed it between reviews but you know what I mean)

Do you still have the fx 5950 and 9800XT? If so what is the difference in performance on the halo built in benchmarks, on your in game test there seemed to be a large difference

It's worth remembering that the nature of the built-in Halo timedemo never made it a good tool for evaluation of actual in-game performance.
 
Uttar said:
I'd say FM doing a press release restating only the drivers on their approval page are approved (possibly after ATI releases some new ones to justify it?) would do the trick. You don't have to insist the 53.03 are illicit; you just gotta say anything BUT the 52.16 is. I doubt they'd get in as much trouble for that, too.
Who CARES how much "trouble" FM gets in, they're in the right! I wanna see 'em nail nVidia to a bloody cross finally!!! :devilish: :devilish: :devilish:

(Don't worry 'bout me, I ain't really all psychotic over it...it's just that winter break ended and the kids went back to school today and I'm all giddy. :D )
 
Quitch said:
52.16 had optimisations for the PS 2.0 test which FutureMark knew of and didn't do anything about, they then approved these drivers. Regardless of the reasons for this, it is a violation of their own guidelines. It is also irrelevant how big a violation this is, because once you've shown that you will violate the guidelines...

I was talking with Patric about it, since tests were coming out showing PS2.0 results most people didn't get, but after version release it was still "under investigation." So basically, they didn't "know" but found out afterwards, and took the only step they could at that point. (Recinding 52.16 would end up causing the biggest foofrah yet, while adding a cautionary message concerning the 2.0 test at least puts it in perspective.) It's discouraging to have seen them miss optimizations (I don't think any 2.0 ones had been present before, so it likely wasn't a priority) but on the whole it reinforces how hard it is to spot everything if the weight of a huge company is brought to bear.

And in the meanwhile, as the PS2.0 test has no effect on the main score or talling on the ORB, it was not mission critical to recind 52.16--for which there would be no turning back, and no ability to keep nVidia REMOTELY on board or the suite on track.
Do you mean that their idea of "enforcement" is to not put the official drivers on the officially approved list?
I "mean" very little. All I know is that the internet community is waiting to jump on any further developments and shout it from the rooftops, and at this point I think that would stop any further relations with nVidia and pit them against each other directly--and who is more capable of surviving THAT encounter? For whatever reason, people eventually just shrug off every optimization nVidia makes with an "oh well, they're just up to their old tricks" and eventually don't care, but with 3DMark it's "well I can't trust THEIR benchmark because nVidia always cheats on it!" and it both lingers and becomes the overriding point. o_O If they go all-out right now, what happens? Who's in a better position to survive the carnage? How quickly will everyone shrug THAT off?

As I don't know what's going on behind the scenes, I have no idea what is the right maneuver at this point it time. I'm SURE, however, that they are attempting to resolve this in a way that doesn't fragment the industry nor their suites, as an overt stance would. It would reinforce the opinion of those already pissed with nVidia's actions, and it might sway a few more people, but eventually FM (3DMark at least) would be dead or useless--and sadly I can't the built-up momentum and "success" of their actions leading to anything but more of it. And then other IHV's having to do it to compete.

Who's in line to take over? Aquamark 3? Woo! Let the games begin! :devilish: Do we really think the industry's positions on "single number bragging points" is going to change? The mindset would have to change first. Illicit optimizations would have to be seen by the development community AS bad and have a stance against it--and notice just how much nothing everyone has said every time previously that this exact matter has come to light? Mmm-hmm.

In the long run FM may be dead in the water, but in killing 3DMark off, Nvidia have gained the reputation of being untrustworthy cheaters and liars in all forms of benchmarking, as well as destroying one of their (previously) favourite marketing tools.

Likely not. In the long run, something else would take over as "favorite marketing tool" and not only would the optimizations begin again, but there would be even LESS resistance to them--and none from the developer itself.
Futuremark isnt dead yet, they still plan to make 3d mark 04/5/whatever.
I'm sure they do, but just what support will it end up getting from industry or the enthusiasts at this point? The instigator has already proven eminantly capable of rising above their actions, have shown they won't support a major benchmark unless it shows them in good light (or can be made to), and the much of the community blames FutureMark for the utterly precarious position the industry and nVidia in particular has put them in.

But hey, I suppose everyone could eventually take the foolish cop-out of saying "all synthetic benchmarks are useless" or think all game benches show an inexorable link to game performance, or that they too can't be optimized for. We're not yet at that state ANYWHERE, and until we are (if we can be), the same actions will freely move where is deemed "most profitable" and snowball in effect.
 
digitalwanderer said:
Did the FM time limit expire? Are they supposed to be doing SOMETHING now about it, or am I jumping the gun?
All approved drivers are listed here http://www.futuremark.com/community/drivers/?approved. If a driver does not get our approval to be used with 3DMark03, it simply won't be listed there. We test all new official WHQL drivers as they are released, and we will keep that list of approved drivers updated.

AFAIK the official & available XGI drivers are not WHQL, and thus aren't approved. We only approve WHQL drivers and when we get WHQL drivers from XGI, we sure will test them.
 
worm[Futuremark said:
]
digitalwanderer said:
Did the FM time limit expire? Are they supposed to be doing SOMETHING now about it, or am I jumping the gun?
All approved drivers are listed here http://www.futuremark.com/community/drivers/?approved. If a driver does not get our approval to be used with 3DMark03, it simply won't be listed there. We test all new official WHQL drivers as they are released, and we will keep that list of approved drivers updated.

Are there no plans to announce drivers that fail the tests publically, and give reasons for their failure? I think most people (quite rightly) assumed that this would be the way FutureMark would deal with things.
 
worm[Futuremark said:
]
digitalwanderer said:
Did the FM time limit expire? Are they supposed to be doing SOMETHING now about it, or am I jumping the gun?
All approved drivers are listed here http://www.futuremark.com/community/drivers/?approved. If a driver does not get our approval to be used with 3DMark03, it simply won't be listed there. We test all new official WHQL drivers as they are released, and we will keep that list of approved drivers updated.
That's IT?!? You won't put 'em on the approved list? What about when websites start benching 'em for reviews? Will you say anything? Or will you happily assume that since the drivers are not on your approved drivers list that people reading the reviews will check your list and realize that the results shouldn't be taken into consideration?

Do you really bloody mean that's it?!?!?!
 
digitalwanderer said:
worm[Futuremark said:
]
digitalwanderer said:
Did the FM time limit expire? Are they supposed to be doing SOMETHING now about it, or am I jumping the gun?
All approved drivers are listed here http://www.futuremark.com/community/drivers/?approved. If a driver does not get our approval to be used with 3DMark03, it simply won't be listed there. We test all new official WHQL drivers as they are released, and we will keep that list of approved drivers updated.
That's IT?!? You won't put 'em on the approved list? What about when websites start benching 'em for reviews? Will you say anything? Or will you happily assume that since the drivers are not on your approved drivers list that people reading the reviews will check your list and realize that the results shouldn't be taken into consideration?

Do you really bloody mean that's it?!?!?!

Uh-oh..... digi mate - remember that heart of yours....... remember calm cool chilled digi now......

/me runs off and and sneaks a look at the old threads and what was said and 'hoped' would happen...
 
Hanners said:
Are there no plans to announce drivers that fail the tests publically, and give reasons for their failure? I think most people (quite rightly) assumed that this would be the way FutureMark would deal with things.

Friendly Reminder Time

LawyerMark 2004
Code:
                           =
                           =
                           =
   =                       =
   =                       =
Futuremark              Nvidia
 
jimbob0i0 said:
Uh-oh..... digi mate - remember that heart of yours....... remember calm cool chilled digi now......
Yeah, yeah. :rolleyes: Thanks Jim, I keep forgetting I ain't supposed to get all worked up anymore over silly shit...it's just a hard concept to get used to for me.

/me runs off and and sneaks a look at the old threads and what was said and 'hoped' would happen...
Lemme know what ya find please, I'm gonna go exercise or eat something disgustingly healthy and meditate for a bit. :)
 
worm[Futuremark said:
]
digitalwanderer said:
Did the FM time limit expire? Are they supposed to be doing SOMETHING now about it, or am I jumping the gun?
All approved drivers are listed here http://www.futuremark.com/community/drivers/?approved. If a driver does not get our approval to be used with 3DMark03, it simply won't be listed there. We test all new official WHQL drivers as they are released, and we will keep that list of approved drivers updated.

AFAIK the official & available XGI drivers are not WHQL, and thus aren't approved. We only approve WHQL drivers and when we get WHQL drivers from XGI, we sure will test them.

Hi worm,

I want to thank all of you at futuremark for being willing to go through and test each driver revision from all these companies to make sure they comply with your guidelines. I'm sure it must be a lot of work, and you guys probably have to walk a bit of a tightrope when it comes to politics in the field.

I have a question about the policy you quoted above though. Given that I'm sure it takes a fair amount of time to actually test the drivers to make sure there arn't any performance altering bugs in them, how should we as readers distinguish between drivers that you have tested and failed (which do not conform to futuremark's guidelines) versus drivers that you have not yet tested or have not finished testing (which may conform to futuremark's guidelines)? This distinction is something that I'm guessing a lot of your users would be interested in knowing.

Thanks,
Nite_Hawk
 
Back
Top