Ops,"I" did it again. Why does Futuremark still ke

[engall]

Ops,Nvidia did it again. New drivers Dets 44.67 optimized for 3dmark03 patch 3.30.FX made much better scores again.
Why does Futuremark still keep silence?
3dmark03 now is meaningless, isnt it?

[DeathKnight]

All I care is that my 9500 Pro with a 1.46GHz T-Bird gets 3353 3DMarks in it.

[rwolf]

I don't think futuremark is meaningless. I think Nvidia benchmarks are meaningless. If you think that the same think isn't happening in quake, ut2003, or other benchmarks you have your head in the sand.

[Chalnoth]

Improvement of benchmark scores does not mean there wasn't a legitimate and beneficial optimization.

[StealthHawk]

Improvement of benchmark scores does not mean there wasn't a legitimate and beneficial optimization.

And it just happened to bring scores to the same level as when NVIDIA was cheating? Talk about coincidences. Considering that in 44.65 we know NVIDIA is altering the shader code I don't see how they are getting similar scores without altering it in 44.67

[FUDie]

Improvement of benchmark scores does not mean there wasn't a legitimate and beneficial optimization.
Except that you can see obvious image quality problems (texture compression, fake trilinear, etc.)

Please stop being an NVIDIA apologist, Chalnoth.

-FUDie

[Entropy]

Accepting this continued behaviour is quite damaging to FutureMark.
They lost a lot of public goodwill and interest already in nVidias initial attack against their product. They then proceed to alienate the substantial group who still saw value in the benchmark with their continued acceptance of nVidia and their shenanigans, making the benchmark useless for comparisons between IHVs, and generally reducing the value of their data base.

They are progressively loosing public interest in their products, and as they loose public interest, some of their revenue streams will eventually dry out.

IMHO, being passive and hoping that things will blow over just isn't an option at this point. The downward spiral is firmly in place, now it's simply a question of pace.

While it might be damaging to their short term financials, taking steps to ensure that their benchmark products can produce useful data might give the company some kind of long term viability. That way, they produce something that is actually interesting or even useful to some parties. It also provides a handle to turn the continued loss of public interest around.
Going down the other path, where they only produce benchmarketing material, is a dead end route when the public no longer cares about the numbers that are produced. Then they are useless even to the companies that have an interest in paying for benchmarketing to promote their wares.

Entropy

[Dave Baumann]

As of Monday NVIDIA rejoined the Beta program. As of Monday one of the most stand up guys I know left Futuremark.

[Bouncing Zabaglione Bros.]

As of Monday NVIDIA rejoined the Beta program. As of Monday one of the most stand up guys I know left Futuremark.

So it's official - Futuremark sold out to Nvidia. :evil:
Nvidia couldn't be top of the test fairly, so they cheated, then threatened the company with court, and now have given a load of cash to Futuremark to keep quiet about the cheating. All for this "useless" benchmark.

What have the other 3DMark members got to say about this? Will B3D withdraw from membership of Futuremark? Will ATI introduce blatent cheats now that Futuremark have sanctioned such behaviour?

What are Nvidia apologist sites like [H] going to say now that Nvidia endorses 3Dmark, and even quotes their (cheating) results during their financial conferences? Will they start using 3Dmark results under pressure from Nvidia, now their masters tell them the test isn't "useless", even when the rest of us know it's more broken than ever?

[Fred]

Dave, if you have an accusation to make, just say it. Instead of making veiled inferences about Nvidia and FutureMarks integrity.

Why don't you interview this guy, and ask him the reasons why he left. At least thats proffessional.

[Proteus]

As of Monday NVIDIA rejoined the Beta program. As of Monday one of the most stand up guys I know left Futuremark.

If that implies what I think it means.. I don't forsee myself buying Nvidia again, regardless of card quality and technology.

[Dave Baumann]

Fred, there was no accusations there, that was just two statements that I found out to be the case yesterday. However, it doesn't take a mind reader to to fathom that they are inevitably linked - take a look back at some of the posts by Rev over the history of this issue.

I'm expecting a call from Futuremark on Monday to find out some more details. As for an interview it wouldn't do much good because he'll inevitably still be bound by his NDA's - we already know far more about the situation than we can ever talk about publically.

[Grall]

Dave, if you have an accusation to make, just say it. Instead of making veiled inferences about Nvidia and FutureMarks integrity.

Agreed.

If you got something to say, just say it. If there's a connection, say so outright! If the two are unrelated, why even mention them together in the same post?

*G*

[Entropy]

Why don't you interview this guy, and ask him the reasons why he left. At least thats proffessional.
Oh, come on.
How on earth can you suggest that "this guy" is at liberty to discuss the internals of Futuremark, never mind that it is hardly in his best interests to do so no matter what?

It has been bleeding obvious that there has been internal conflicts regarding how the situation with nVidia should be handled. Not surprising. Nor is it surprising that there might be individuals who feel that Futuremark may now be a less attractive place to spend their professional futures than, say, a year ago.

Dave is a human being, not only a representative of B3D. It is obvious that it was Dave-the-human who spoke above, about a guy he would seem to respect a lot.

Entropy

Edit: Dave beat me to it. There is nothing that says that everything he posts here must be checked and accepted by the legal and PR departments of various companies. Live with it.

[StealthHawk]

As of Monday NVIDIA rejoined the Beta program. As of Monday one of the most stand up guys I know left Futuremark.

So it's official - Futuremark sold out to Nvidia. :evil:
Nvidia couldn't be top of the test fairly, so they cheated, then threatened the company with court, and now have given a load of cash to Futuremark to keep quiet about the cheating. All for this "useless" benchmark.

What have the other 3DMark members got to say about this? Will B3D withdraw from membership of Futuremark? Will ATI introduce blatent cheats now that Futuremark have sanctioned such behaviour?

What are Nvidia apologist sites like [H] going to say now that Nvidia endorses 3Dmark, and even quotes their (cheating) results during their financial conferences? Will they start using 3Dmark results under pressure from Nvidia, now their masters tell them the test isn't "useless", even when the rest of us know it's more broken than ever?

Since when did Futuremark sanction cheating? I'd say they are turning a blind eye to it. I haven't yet seen them say "yes, NVIDIA's optimizations are acceptable." In fact, we've heard the opposite. That cheating or optimization, they are still invalid.

The question is, once NVIDIA launches some new official drivers which bring the scores back up, will Futuremark do anything or stay silent? I'm betting at this point that they will just keep quiet.

[SvP]

If you got something to say, just say it. If there's a connection, say so outright! If the two are unrelated, why even mention them together in the same post? Why are you so upset? :wink:

[engall]

I wanna know where is auditor now!

[Bjorn]

The question is, once NVIDIA launches some new official drivers which bring the scores back up, will Futuremark do anything or stay silent? I'm betting at this point that they will just keep quiet.


Unless of course Nvidia manages to get the scores back up without cheating. Something that doesn't seem very likely at this moment though. But i reserve my judgement until the next official drivers comes out. Maybe the're the drivers that B3D are testing now ?

[Harlequin]

AJ is the one who has left futuremark.

[incurable]

Improvement of benchmark scores does not mean there wasn't a legitimate and beneficial optimization.
While this is true in general, can you look at the facts of this whole nVidia/3DMark03-thing and still believe valid optimizations to be the cause for the recent re-alignment of 3DMark-scores?

As of Monday NVIDIA rejoined the Beta program. As of Monday one of the most stand up guys I know left Futuremark.
Very interesting info, thanks for sharing, Dave!

What have the other 3DMark members got to say about this? Will B3D withdraw from membership of Futuremark? Will ATI introduce blatent cheats now that Futuremark have sanctioned such behaviour?
Good questions, very good questions.

Since when did Futuremark sanction cheating? I'd say they are turning a blind eye to it. I haven't yet seen them say "yes, NVIDIA's optimizations are acceptable." In fact, we've heard the opposite. That cheating or optimization, they are still invalid.

The question is, once NVIDIA launches some new official drivers which bring the scores back up, will Futuremark do anything or stay silent? I'm betting at this point that they will just keep quiet.
I don't see much of difference between sanctioning the suspect increases and just keeping quite about it while allowing scores to be entered into the ORB using suspect drivers.

cu

incurable

[Bouncing Zabaglione Bros.]

Since when did Futuremark sanction cheating? I'd say they are turning a blind eye to it. I haven't yet seen them say "yes, NVIDIA's optimizations are acceptable." In fact, we've heard the opposite. That cheating or optimization, they are still invalid.

If you know it's happening and you keep quiet about it, then you are sanctioning it. Don't forget, Futuremark are in the position of policing the validity of 3Dmark scores. They have an obligation to keep the scores "clean"


The question is, once NVIDIA launches some new official drivers which bring the scores back up, will Futuremark do anything or stay silent? I'm betting at this point that they will just keep quiet.

Just after the 330 patch came out, Nvidia launched new drivers that seemed to put the cheats back in. Performance went up to pre-330 cheat levels. Neither FM or Nvidia made any comment about this. Given it would have been in FM and Nvidia's best interest to put out a statement saying "all cheats have been removed and the GFFX really does bench this well" (especially given all the furore over Nvidia's initial cheating), I can only surmise that the cheating is still happening, but that FM have been induced (either by lawyers or money) to keep quiet about it. Thus in my opinion, FM are sanctioning these cheats, and are complicit in the deceit of both Futuremark's and Nvidia's customers.

[Entropy]

Just after the 330 patch came out, Nvidia launched new drivers that seemed to put the cheats back in. Performance went up to pre-330 cheat levels. Neither FM or Nvidia made any comment about this. Given it would have been in FM and Nvidia's best interest to put out a statement saying "all cheats have been removed and the GFFX really does bench this well" (especially given all the furore over Nvidia's initial cheating), I can only surmise that the cheating is still happening, but that FM have been induced (either by lawyers or money) to keep quiet about it. Thus in my opinion, FM are sanctioning these cheats, and are complicit in the deceit of both Futuremark's and Nvidia's customers.

True words.
I quoted them because they deserved to be read again.

While Futuremark could have weathered the nVidia storm and come out in as strong as before, they have now managed to alienate just about everyone. Even their own people, apparently. It is one thing to see yourself as part of the foremost benchmarking outfit in the business, a driving force for PC and graphics development as well as equipment sales. It's quite another to look in the mirror every morning and see someone who helps egg the kids on to spend their money buying kit from corporations out to decieve them, a deception you're an accomplice to. I'd predict that Futuremark will loose other good people in the relatively close future.

But IMHO too much attention in this affair has been focussed on Futuremark because nVidia tactics became obvious in their high profile benchmark. Those tactics aren't limited to 3DMark, and it is interesting to follow how the industry (manufacturers/vendors/media/consumers) react and deal with that.

And I wonder just how much nVidia has lost by sticking to their deceptive tactics, and will loose in the future. While there are blind fans, most people take a dim view of being consistently misled and aren't necessarily eager to support it financially. Or are they? If this had been about cars, and a manufacturer had been found to consistently lie about the information on engine power, safety equipment et cetera, they would be deservedly dead in the market. nVidia obviously sells huge volumes to the preconfigured part of the market, but what about the part where consumers make active choices? If anyone has solid retail sales data, it would be very interesting to see it.

Entropy

[Reverend]

Dave, if you have an accusation to make, just say it. Instead of making veiled inferences about Nvidia and FutureMarks integrity.
Dave is giving you a "scoop". Scoops invariably never tells the whole story. If you do not prefer to have such scoops, don't comment or if you do, try to respect the publicly recognized host of your public forum participation. I do not like the inference about dave's charactyer you're making here.

Why don't you interview this guy, and ask him the reasons why he left. At least thats proffessional.
I gave you guys the reason he left a long time ago. Look for my post about this, I don't have the time to search for you. I didn't name him for very obvious reasons but someone here already mentioned his initials. He didn't agree with the route taken by Futuremark about the whole NV/FM fiasco.... and he doesn't agree with it now according to the latest agreements (not public yet, or may never be) between NV and FM. Interviewing him would be like interviewing Scott Sellers about why he never wanted to joined NVIDIA -- Scott could be sued for revealing his true feelings and thoughts... so what's the point?

We, as a beta member, means we cannot disclose non-public information. Asking us to do so would mean possibly paying legal fees and fines. Would you be willing to foot such possible bills?

Beyond3D will be watching how things turn out at FM. We will have no hesitation in leaving the beta program if we feel that the way things are going at FM doesn't agree with our own principles. We don't know all the details at this point. When we do, we can decide what to do but we not be able to reveal it due to NDAs. We may still be tied to NDAs for a certain period if we decide to leave the beta program, so anything we may have to say about why we may be leaving the beta program will have to come upon NDA expiry. This should not infer that we have already decided what we're going to do based on what we know because, as mentioned, we don't have all the details (FM has not emailed us)... I'm only saying all of this because I'm annoyed by the apparent fact that you don't seem to know what it means to be a FM beta member and all the legal tie-ins this means.

[Ailuros]

While this is true in general, can you look at the facts of this whole nVidia/3DMark03-thing and still believe valid optimizations to be the cause for the recent re-alignment of 3DMark-scores?

Clipping planes are like Partisans. They randomly appear in your driver code, when you expect them the least :shock:

:roll:

[WaltC]

Ops,Nvidia did it again. New drivers Dets 44.67 optimized for 3dmark03 patch 3.30.FX made much better scores again.
Why does Futuremark still keep silence?
3dmark03 now is meaningless, isnt it?

44.03 is the latest official nVidia Detonator package officially released by the company from www.nvidia.com . Everything else is either leaked or an OEM driver (which may use different, and misleading, numbering schemes.)

I think all that FM can reasonably be asked to do is to issue a recompile patch along the lines of 330 every time nVidia officially releases a driver package. I think FM is entirely right to ignore all other "releases."

I saw a blurb somewhere yesterday which stated nVidia would officially release a new Detonator set on Monday, Aug 11. Have no idea whether it's true, of course. But if it is then that's when FM should do another patch. If and when nVidia releases another official Det package, if FM at that time (within a few days) refuses to issue another recompile patch then I'll agree with you.

[Fred]

Reverend, theres ways of putting things that are more apt to be dissapproved of. Like it or not, Dave is an integral part of the 3d graphics community and his word (and yours) carry more weight and therefore more responsibility than mine or any others. Responsibility of the press, and all that silly stuff.

His comments could be taken like 20 different ways, some of which are accusatory in nature.

For instance, I could infer (as I initially did) that he was making aspersions about the integrity of 3dmarks future programs, because as he put it, a 'stand up' guy left the very same day that Nvidia rejoined. Inference: all 'stand up' guys should feel the very same way. Thats one way of interpreting it, wrong as it may be.

It would be unfair to 3dmark to do just that, despite what others on this forum posted. No information has been given as to the details and the 'whys' to this mans departure, and for all we know they still have the consumers best interests at heart. Unless you wish to show evidence to the contrary?

I don't like it when Kyle mudslings publically, and veiled mudslinging is just as bad.

(My excuses go to Dave of course, as I doubt that was the intent. Just a quibble with the wording is all)

[Mark]

So... now that nVidia apparently "endorses" 3DMark03 once again, any takers on how long it'll be before [H] reintroduces it back into their reviews? How about the "Oops, sorry we were wrong. 3DMark03 is a great benchmark" editorial by Kyle?

[nyt]

So... now that nVidia apparently "endorses" 3DMark03 once again, any takers on how long it'll be before [H] reintroduces it back into their reviews? How about the "Oops, sorry we were wrong. 3DMark03 is a great benchmark" editorial by Kyle?

Why don't you wait instead of speculating about some more potential bashing? Ain't you tired of it? AFAIK they made some efforts with their latest review, you could acknowledge that. IMHO the discussed move from 3DMark03 will further prove [H]'s point regarding this very benchmark. Now if they do consider using it again, it's another story.

[thatdude90210]

So... now that nVidia apparently "endorses" 3DMark03 once again, any takers on how long it'll be before [H] reintroduces it back into their reviews? How about the "Oops, sorry we were wrong. 3DMark03 is a great benchmark" editorial by Kyle?

I predict FM will introduce a new patch 340, it will do absolutely nothing but will allow Nv and [K] to save face, "patch 340 has address our previous concerns so 3Dmark03 is now useful."

[just me]

Not surprised at any of these events. FM has changed since their 'original' Marks with the most significant changes after 2001. Look who came 'onboard' since then & look how MO changed & what it has become. $$$ is the only reason FM is in business anymore & I wouldn't doubt if SW & a few others follow AJ's lead.

Seems the NV NDA muzzle is on here too. How does it feel to have to watch every word for fear of legal repercussions?

"Keep your friends close & your enemies closer" & under an NDA. :wink:

.02,

[WaltC]

As of Monday NVIDIA rejoined the Beta program. As of Monday one of the most stand up guys I know left Futuremark.

Well, just like me to reply to the first post without reading the thread...and miss the most important post in it....Tells me pretty much all I need to know. I am really bothered by the fact that there's been no press release from nVidia reversing the press release it made when it quit last year.

[WaltC]

Beyond3D will be watching how things turn out at FM. We will have no hesitation in leaving the beta program if we feel that the way things are going at FM doesn't agree with our own principles. We don't know all the details at this point. When we do, we can decide what to do but we not be able to reveal it due to NDAs. We may still be tied to NDAs for a certain period if we decide to leave the beta program, so anything we may have to say about why we may be leaving the beta program will have to come upon NDA expiry. This should not infer that we have already decided what we're going to do based on what we know because, as mentioned, we don't have all the details (FM has not emailed us)... I'm only saying all of this because I'm annoyed by the apparent fact that you don't seem to know what it means to be a FM beta member and all the legal tie-ins this means.

Rev, nVidia quit the program last year and made plenty of public statements--negative ones--about FM and their benchmarks. I can't see at all how nVidia would be less NDA-bound than B3d, and apparently nVidia wasn't bound at all. So I'd say you have at least as much freedom to say what you like as nVidia has done for the past 8-9 months. I'd say you have nothing to worry about. I can't see how FM could hold you to any higher standard than it held nVidia. (I would think an NDA going beyond the code you've seen and used would be very excessive.) If you are concerned for some reason about this then I would suggest you do what nVidia has done--find a convenient proxy to release the info for you, if you are concerned about releasing it directly.

It also strikes me as very odd that as a partner in the program you have not been informed of these developments, if they have actually occurred, and there is nothing stopping you as a partner from emailing FM to inquire (no need to "wait" on them necessarily.) Perfectly legitimate, straightforward, question which all the partners have a right to know about. I mean, it's not as if FM keeps its partner list a secret...;)

[WaltC]

...
For instance, I could infer (as I initially did) that he was making aspersions about the integrity of 3dmarks future programs, because as he put it, a 'stand up' guy left the very same day that Nvidia rejoined. Inference: all 'stand up' guys should feel the very same way. Thats one way of interpreting it, wrong as it may be.

...

Fred, I don't think it's possible for anyone, anywhere, to cast more aspersion on FM than FM has thus far cast upon itself. I have been trying to give them the benefit of the doubt, however, and was hoping they'd be quick to release another recompile patch shortly after the next official, bona-fide Detonator release. I wanted to give them that much room, at least.

But if Dave has received the straight skinny on this, and FM and nVidia are conspiring to keep nVidia's renewed participation a secret, that will tear it for me. Without a doubt.

Frankly, if this situation is the correct one as described, I am amazed--amazed--that ATi or any other reputable partner would continue to pay to be a member of this program. I hope we'll get some quick confirmation of these events, one way or the other.

[StealthHawk]

Ops,Nvidia did it again. New drivers Dets 44.67 optimized for 3dmark03 patch 3.30.FX made much better scores again.
Why does Futuremark still keep silence?
3dmark03 now is meaningless, isnt it?

44.03 is the latest official nVidia Detonator package officially released by the company from www.nvidia.com . Everything else is either leaked or an OEM driver (which may use different, and misleading, numbering schemes.)

I think all that FM can reasonably be asked to do is to issue a recompile patch along the lines of 330 every time nVidia officially releases a driver package. I think FM is entirely right to ignore all other "releases."

I saw a blurb somewhere yesterday which stated nVidia would officially release a new Detonator set on Monday, Aug 11. Have no idea whether it's true, of course. But if it is then that's when FM should do another patch. If and when nVidia releases another official Det package, if FM at that time (within a few days) refuses to issue another recompile patch then I'll agree with you.

These actions actually bring up an interesting dilemma though. How exactly did NVIDIA claim the number one spot in 3dmark03. What driver were the cards using? Certainly it wasn't 44.03...

So Futuremark is willing to accept possibly tainted drivers but is not willing to release a new patch. That is very irresponsible. Of course, there may be something bigger here going on.

If NVIDIA releases a new driver on Monday with the same ridiculously high scores, and Futuremark does nothing, then it will be obvious to everyone that they have sold out their integrity.

[Dave Baumann]

But if Dave has received the straight skinny on this, and FM and nVidia are conspiring to keep nVidia's renewed participation a secret, that will tear it for me. Without a doubt.

I don't think there is any secret here, its just that the deal is rather fresh. I've probably just done a rather poor job of preannouncing things!

[Rookie]

Dave wrote:
As of Monday NVIDIA rejoined the Beta program. As of Monday one of the most stand up guys I know left Futuremark.

Rev wrote:


Beyond3D will be watching how things turn out at FM. We will have no hesitation in leaving the beta program if we feel that the way things are going at FM doesn't agree with our own principles. We don't know all the details at this point. When we do, we can decide what to do but we not be able to reveal it due to NDAs. We may still be tied to NDAs for a certain period if we decide to leave the beta program, so anything we may have to say about why we may be leaving the beta program will have to come upon NDA expiry. This should not infer that we have already decided what we're going to do based on what we know because, as mentioned, we don't have all the details (FM has not emailed us)... I'm only saying all of this because I'm annoyed by the apparent fact that you don't seem to know what it means to be a FM beta member and all the legal tie-ins this means.

Does your b3d guys imply when next monday NVIDIA back to beta program,B3D will quit ? :?:

[Fred da Roza]

So Futuremark is willing to accept possibly tainted drivers but is not willing to release a new patch. That is very irresponsible. Of course, there may be something bigger here going on.

If NVIDIA releases a new driver on Monday with the same ridiculously high scores, and Futuremark does nothing, then it will be obvious to everyone that they have sold out their integrity.

How do you fight a company that has no ethics and another that is willing to sell theirs? This is a serious dilemma for ATI. If Futuremark won’t enforce the validity of their benchmark, does ATI have any other choice but to cheat as well? If it destroys the usefulness of Futuremark's benchmark so be it. They made their bed.

[Sazar]

So... now that nVidia apparently "endorses" 3DMark03 once again, any takers on how long it'll be before [H] reintroduces it back into their reviews? How about the "Oops, sorry we were wrong. 3DMark03 is a great benchmark" editorial by Kyle?

as has been stated already... this move.. if [H] keeps their stated stance with regards to 3dmark03 == they were right... hard as it may be to believe :)

ergo... [H] MAY yet be vindicated for SOME of their statements made over the past several months...

has yet to be seen but I for one am not going to jump the gun and start another KB bashing thread... :) at least not this time...

until such a time as an editorial or other is released by [H]... any comment on their exected position in light of the events that have apparently taken place is pure speculation... which is all well and good... but bashing == non-constructive...

btw... cheers for the news and info Dave/Rev... appreciated...

[digitalwanderer]

How do you fight a company that has no ethics and another that is willing to sell theirs?
Well generally with sarcasm, although I tend to rant and stomp about a bit in outrage and disgust too. ;)

[Arun]

How do you fight a company that has no ethics and another that is willing to sell theirs? This is a serious dilemma for ATI. If Futuremark won’t enforce the validity of their benchmark, does ATI have any other choice but to cheat as well? If it destroys the usefulness of Futuremark's benchmark so be it. They made their bed.

Well, nVidia tried to quit 3DMark with 0 community support.
This time, ATI would have nearly full community support ( certainly including mine ) - so that's an option.
And then there's what Digitalwanderer said, hehe :)


Uttar

[demalion]

So... now that nVidia apparently "endorses" 3DMark03 once again, any takers on how long it'll be before [H] reintroduces it back into their reviews? How about the "Oops, sorry we were wrong. 3DMark03 is a great benchmark" editorial by Kyle?

as has been stated already... this move.. if [H] keeps their stated stance with regards to 3dmark03 == they were right... hard as it may be to believe :)

ergo... [H] MAY yet be vindicated for SOME of their statements made over the past several months...

How can they be vindicated on their stance against Futuremark and saying it is a useless benchmark, when they followed nVidia's party line of attacking them for being the opposite? Everything [H] parroted about the "flaws" of 3dmark 03 were because it was useful (sans cheating) as a DirectX benchmark...if you're part of an attack on what makes something useful, you're not vindicated when it becomes useless: you were simply part of a successful campaign in making it that way.

"Congratulations" to [H], since it seems that's what they wanted...too bad for the existence of "informed consumers", though, as it seems they're becoming an endangered species. :-?

Anyways, I've said my piece to, for, and against Futuremark before...I'm waiting to see what occurs next.

[Fred da Roza]

Well, nVidia tried to quit 3DMark with 0 community support.
This time, ATI would have nearly full community support ( certainly including mine ) - so that's an option.
And then there's what Digitalwanderer said, hehe :)


Uttar

Unless sites like THG and Anandtech stop using it, 1 small communities support won't mean much. You have got people like Unwinder painting ATI with the same brush they use on nVidia, while ignoring the reality of the situation ATI has to fight. And then there are sites like [H] who protect nVidia and push their PR while taking every opportunity to smear ATI. And now the company who makes the most recognized 3D benchmark turns a blind eye to nVidia antics. Unless your comments means ATI should use the same tactics nVidia uses, that doesn't help ATI's bottom line.

[WaltC]

I don't think there is any secret here, its just that the deal is rather fresh. I've probably just done a rather poor job of preannouncing things!

OK, fair enough...I was reacting more to my interpretation of Rev's remarks, which seemed to place the whole thing in an opaque light. I guess this is the unvarnished reality, then....

How do you fight a company that has no ethics and another that is willing to sell theirs? This is a serious dilemma for ATI. If Futuremark won’t enforce the validity of their benchmark, does ATI have any other choice but to cheat as well? If it destroys the usefulness of Futuremark's benchmark so be it. They made their bed.

What I find remarkable about this is what nVidia imagines it's going to get out of rejoining the FM program...? The company has spent the last 8-9 months publicly undermining FM and its software. Heh...;) When it gets out that nVidia's rejoined the program after all of that, most people willing to give FM the benefit of a doubt will no longer be inclined to do so. The only thing that might help a little bit is to see a "humble-pie" press release coming out of nVidia acknowledging past indiscretions with a pledge to fly right in the future, along with an FM statement of intent to make sure nVidia does that. Although such statements would initially be viewed with skepticism, they would certainly help matters, and would be easy enough to check out going forward. It's still baffling, however.

Hopefully, FM will not come out with a statement to the effect of: "nVidia has now paid us enough to say that we were completely wrong the first time." That will indeed be The End....;)

ergo... [H] MAY yet be vindicated for SOME of their statements made over the past several months.


Heh...;) I simply cannot imagine how frgmstr will feel when he discovers nVidia has rejoined the program---talking about going out on a limb and then having it sawed out from under you...! *chuckle* Wow!....I hope he learns a valuable lesson from this, and will see how nVidia is using him and then discarding him like so much toilet paper when it's finished....

For months now frgmstr has been forwarding the position that synthetic benchmarking with programs like 3dMk03 is "wrong"....and defending his interpretation of what nVidia is doing...and parroting the nVidia party line as he was lead to believe it. Hopefully, he will at last get the message that nVidia only pretends to "listen to him" to get him to listen to them and to get him to say whatever they don't want to say publicly themselves--so that they can then later reverse themselves with impunity and leave folks like him holding the bag.

Yes, while it may be true that ultimately it will turn out that 3dMk03 and FM are thoroughly discredited, it won't be for *any* of the reasons [H] has promulgated over the last few months, ironically. It will be because FM will have destroyed the credibility of its own software through incessant capitulation to monied interests. To me the irony is that FM may pass from the public scene with nVidia as a full partner, instead of going out with nVidia in opposition. All of this just boggles the mind.

Edit: Typos

[Sazar]

ergo... [H] MAY yet be vindicated for SOME of their statements made over the past several months.


Heh...;) I simply cannot imagine how frgmstr will feel when he discovers nVidia has rejoined the program---talking about going out on a limb and then having it sawed out from under you...! *chuckle* Wow!....I hope he learns a valuable lesson from this, and will see how nVidia is using him and then discarding him like so much toilet paper when it's finished....

For months now frgmstr has been forwarding the position that synthetic benchmarking with programs like 3dMk03 is "wrong"....and defending his interpretation of what nVidia is doing...and parroting the nVidia party line as he was lead to believe it was. Hopefully, he will at last get the message that nVidia only pretends to "listen to him" to get him to listen to them and to get him to say whatever they don't want to say publicly themselves--so that they can then later reverse themselves with impunity and leave folks like him holding the bag.

Yes, while it may be true that ultimately it will turn out that 3dMk03 and FM are thoroughly discredited, it won't be for *any* of the reasons [H] has promulgated over the last few months, ironically. It will be because FM will have destroyed the credibility of its own software through incessant capitulation to monied interests. To me the irony is that FM may pass from the public scene with nVidia as a full partner, instead of going out with nVidia in opposition. All of this just boggles the mind.

Edit: Typos

lol... yup... thats pretty much my thoughts...

though I have grown very wary of many of the reviews [H] puts out... Sean does review there.. and I do read his stuff... and brent also has done some good stuff...

on the whole... I am interested in seeing what the position will be once nvidia rejoins the beta group and what the so-called *high level nvidia brass* people will have told Kyle and if that is put through in an editorial or other on his website...

but as of right now... can't really take a potshot based solely on rumour and innuendo :)

bad form really :)

so I'll bide my time...

[digitalwanderer]

How do you fight a company that has no ethics and another that is willing to sell theirs? This is a serious dilemma for ATI. If Futuremark won’t enforce the validity of their benchmark, does ATI have any other choice but to cheat as well? If it destroys the usefulness of Futuremark's benchmark so be it. They made their bed.

Well, nVidia tried to quit 3DMark with 0 community support.
This time, ATI would have nearly full community support ( certainly including mine ) - so that's an option.
I never thought of that. If ATi left it would be an entirely different situation, wouldn't it? :D

Can I have everyone's permission to tell ATi we'll all stand behind 'em if they leave? :?: :lol:

And then there's what Digitalwanderer said, hehe :)

Heh-heh-heh-heh-heh! THAT'S what I'm talking about! ;)

[digitalwanderer]

Well, nVidia tried to quit 3DMark with 0 community support.
This time, ATI would have nearly full community support ( certainly including mine ) - so that's an option.
And then there's what Digitalwanderer said, hehe :)


Uttar

Unless sites like THG and Anandtech stop using it, 1 small communities support won't mean much. You have got people like Unwinder painting ATI with the same brush they use on nVidia, while ignoring the reality of the situation ATI has to fight. And then there are sites like [H] who protect nVidia and push their PR while taking every opportunity to smear ATI. And now the company who makes the most recognized 3D benchmark turns a blind eye to nVidia antics. Unless your comments means ATI should use the same tactics nVidia uses, that doesn't help ATI's bottom line.
I disagree with you on the "one small communities support" bit, I think there are a whole bunch of little websites out there that make up the true heart of the graphics community that really would back ATi to the hilt on this one as it's just SO bloody wrong.

It might be a "little" community, but it's voice is pretty unstoppable thanks to the internet and even the big guys are finding out they can't ignore it. (I wouldn't feel half as confident saying that if I hadn't been in a very pleasent e-mail correspondence with Derek Perez over the last few days. ;) )

It might be a little voice, but it's right in what it's saying and you can't deny the truth forever. :)

[nyt]

3DMark03 == coverage. ATi won't leave FM because of this. NVidia were being arrogant when they did, and they suddenly notice they can't really afford not to be there in those difficult times. What will be done to regain credibility after all this mess remains to be seen, and I wouldn't be surprised if FM managed to get both ATI and NV to come to an agreement regarding what's acceptable and what's not. Let's hope it will be made public!

[WaltC]

...
but as of right now... can't really take a potshot based solely on rumour and innuendo :)

bad form really :)

so I'll bide my time...

I certainly agree with this...but as I interpret Dave's response to me above this is not a matter or rumor or innuendo...nVidia has rejoined the FM program.

[digitalwanderer]

...
but as of right now... can't really take a potshot based solely on rumour and innuendo :)

bad form really :)

so I'll bide my time...

I certainly agree with this...but as I interpret Dave's response to me above this is not a matter or rumor or innuendo...nVidia has rejoined the FM program.
Pagh! Methinks you're all being a bit silly-headed "professional" about this, I've been taking potshots at 'em for months based on less! ;)

Seriously, there's enough ammo already out there to spend a relaxing afternoon taking potshots at 'em...this one though is gonna be a real humdinger come Monday! :lol:

[WaltC]

3DMark03 == coverage. ATi won't leave FM because of this. NVidia were being arrogant when they did, and they suddenly notice they can't really afford not to be there in those difficult times. What will be done to regain credibility after all this mess remains to be seen, and I wouldn't be surprised if FM managed to get both ATI and NV to come to an agreement regarding what's acceptable and what's not. Let's hope it will be made public!

I can't agree with you here...and frankly it's a mystery to me why IHVs think this way about the benchmark. I've personally bought, eh, ~a dozen or so 3d cards since the V1, and the number of those purchases influenced in any capacity at all by a FutureMark-Mad Onion-FutureMark benchmark is absolutely zero. I can tell you that those benchmarks have never influenced my purchases whatsoever. Other factors were far more influential in my decisions (FM benchmarks had no influence.)

What's of interest to me here in this case is the story of a software company publishing a benchmark and the attempts of an IHV to subvert it simply because they weren't happy with how it portrayed their current 3D hardware, which just so happens to be empirically inferior to the 3D hardware offered by their competitors. In the beginning FM made the wise choice of defending itself against these crass attempts at subversion and subornation. Now, FutureMark's position is anything but clear.

The thing is, 3dMk has no value as a promotional tool to IHVs if a sizable majority of the people who buy 3D cards no longer believe it tells them anything relevant. The future of FutureMark will be determined not by the IHVs which are its partners, but by the degree of utility and credibility the 3d-card buying public believes their software has. Therefore, if FM software is failing to sell 3D cards then it has no utility to IHVs. And the only way 3dMk will influence sales of 3d cards is if the people who buy 3d cards believe the benchmark has something meaningful to tell them about 3d hardware. If FM continues to lose its credibility with *the buying public* then, inevitably, it will lose IHV participation as well. This is exactly why the company should not shirk from aggressively defending its software against IHV attacks.

[digitalwanderer]

Do y'all remember when nVidia said this about 3dm2k3 (http://www.extremetech.com/article2/0,3973,884239,00.asp)? Granted it was a whole six months ago and an arwful lot has changed since then, but here is some fun quotage from it to remember for tomorrow's "surprise" announcement from nVidia & FM:
"The reason that we're not all gung ho about it is that (3DMark'03) is not representative of (actual) games, nor is it a good benchmark," said Tony Tamasi, senior director of desktop product management at Nvidia. "That means Nvidia has to expend effort to make sure it runs well on our hardware. All that energy that we spend doesn't benefit the user. None. Zero. All that effort doesn't go to benefit any game, either. That's kind of depressing."
So what is Nvidia's choice for benchmarking? "Games," Tamasi said. "Use games."
Specifically, Tamasi said he objected that Futuremark apparently chose to emphasize single-textured pixels in the benchmark's four tests, while previous versions had pushed multitexturing. Tamasi also criticized FutureMark's use of older version pixel and vertex shaders, and the benchmark's heavy emphasis on running and rerunning vertex shader operations -- 36 times, by his count.
(Man, I am enjoying this WAY too much! :lol: )

[Althornin]

Do y'all remember when nVidia said this about 3dm2k3 (http://www.extremetech.com/article2/0,3973,884239,00.asp)? Granted it was a whole six months ago and an arwful lot has changed since then, but here is some fun quotage from it to remember for tomorrow's "surprise" announcement from nVidia & FM:
"The reason that we're not all gung ho about it is that (3DMark'03) is not representative of (actual) games, nor is it a good benchmark," said Tony Tamasi, senior director of desktop product management at Nvidia. "That means Nvidia has to expend effort to make sure it runs well on our hardware. All that energy that we spend doesn't benefit the user. None. Zero. All that effort doesn't go to benefit any game, either. That's kind of depressing."
So what is Nvidia's choice for benchmarking? "Games," Tamasi said. "Use games."


Actually, this makes 100% sense for the nV3x architecture.
It performs very well on TODAYS GAMES. Of course they dont want it checked out on a test that is future looking. Cause the Shading engine blows.....

[digitalwanderer]

Do y'all remember when nVidia said this about 3dm2k3 (http://www.extremetech.com/article2/0,3973,884239,00.asp)? Granted it was a whole six months ago and an arwful lot has changed since then, but here is some fun quotage from it to remember for tomorrow's "surprise" announcement from nVidia & FM:
"The reason that we're not all gung ho about it is that (3DMark'03) is not representative of (actual) games, nor is it a good benchmark," said Tony Tamasi, senior director of desktop product management at Nvidia. "That means Nvidia has to expend effort to make sure it runs well on our hardware. All that energy that we spend doesn't benefit the user. None. Zero. All that effort doesn't go to benefit any game, either. That's kind of depressing."
So what is Nvidia's choice for benchmarking? "Games," Tamasi said. "Use games."


Actually, this makes 100% sense for the nV3x architecture.
It performs very well on TODAYS GAMES. Of course they dont want it checked out on a test that is future looking. Cause the Shading engine blows.....
Ah, but that then begs the question of why they are now endorsing it! :lol:

OMFG, I just had a thought that I personally found hilarious for some reason!

Kyle is about to be caught up in his very own M|22! :lol:

(Sorry, you have to know FrgMstr sig for that one. ;) )

[just me]

Ah, but that then begs the question of why they are now endorsing it!

Who says they are 'endorsing iit? *IF* NV did indeed rejoin the BETA program it is for 3DM04, not 3DM03. :wink:

Over at Futuremark 2 Admins have already said AJ is still there & did NOT quit. :shock:

What if this is an attempt to discredit B3D & Tero purposely inferred AJ quit just to see what 'rumours' would be spread? Why? Well B3D is a thorn is NV's side & if B3D is forced out of the BETA program due to 'violations' of the BETA agreement ....

I understand B3D's concerns as a member of the BETA program, but 3DM03 is a useless benchmark to the majority of ppl I know/conversate w/on the web: It is a game for the highest points & anything goes. It is not representational of todays or tomorrows games, is not a "Gamers Benchmark" at all, is not an accurate representation of system performance in a gaming environment & is being cheated to achieve 'high scores'.

*IF* AJ did indeed leave for 'ethical' reason > BRAVO AJ! 8) I doubt it will make any difference tho' as the proggie is already useless in the eyes of the majority. FM did it to themselves too. :evil:

.02,

[Dave Baumann]

Over at Futuremark 2 Admins have already said AJ is still there & did NOT quit.

Actually, Nick said he was currently still working in the office, not that he didn't quit. Tell me, if you resign do you not work out any notice period?

Anyway, just had another great and, as ever, wide ranging conversation with AJ. You'll hear more sooner or later.

[digitalwanderer]

Over at Futuremark 2 Admins have already said AJ is still there & did NOT quit.

Actually, Nick said he was currently still working in the office, not that he didn't quit. Tell me, if you resign do you not work out any notice period?

Anyway, just had another great and, as ever, wide ranging conversation with AJ. You'll hear more sooner or later.
Mr. Baummann I have the highest respect for ya as a source of information and knowledge, but I do declare there are times I wish I could just reach out and grab you by the shoulders and shriek "TELL ME!!! TELL ME!!!!!" into your face when you do stuff like that! ;)

Can you tell us if we'll be hearing more tomorrow? I got this feeling in the pit of me belly that tomorrow the whole power paradigm in the graphics world is going to get turned on it's ear again a bit. (I'd prefer Tuesday actually, I'll have more time to follow it.... )

(I don't mean that shaking bit as a threat, it's meant as a funny. I've been having some troubles with people misunderstanding me so I thought I'd best clarify that I'm joking. ;) )

[CorwinB]

I don't know why people are surprised that Nvidia is rejoining the Beta program after spending 8-9 months making negative comments about the validity of the software... Nvidia's PR (like most PRs) never was afraid to say something and then go the opposite way a few months later. Remember, back in TNT2 days, 32bits and image quality was everything. Then, when they faced the Voodoo5, all that mattered was Q3 scores in 640x480x16bits...

[just me]

Over at Futuremark 2 Admins have already said AJ is still there & did NOT quit.

Actually, Nick said he was currently still working in the office, not that he didn't quit. Tell me, if you resign do you not work out any notice period?

Anyway, just had another great and, as ever, wide ranging conversation with AJ. You'll hear more sooner or later.

Ahh, my bad. You are correct > they said he is still there, not that he didn't quit. :wink:

I, myself, walk when I've quit a job over issues (I imagine) as similar to this. If I have another job waiting & didn't leave over moral/ethical 'issues', then I 'give notice' & work thru that period. 8)

If AJ does leave: I certainly hope some others follow & they continue the 'tradition' AJ & Markus established for fair & reliable benchmarks. :D

.02,

[Sazar]

...
but as of right now... can't really take a potshot based solely on rumour and innuendo :)

bad form really :)

so I'll bide my time...

I certainly agree with this...but as I interpret Dave's response to me above this is not a matter or rumor or innuendo...nVidia has rejoined the FM program.

rumour and innuendo pertaining to the perceived position [H] will take on the matter m8... not about nvidia rejoining the beta program :)

that I will take as a given considering the source(s)

[Mark]

I notice some of you guys wondering what nVidia will do to regain their credability regarding the FM fiasco. I'll tell you right now what they'll do: Nothing. In their eyes, they've done no wrong. They even have an official statement from FM to back it up.

What about community credability? Why worry about it? In a few months, thanks to their massive PR machine and the hoards of n[V]idiots swarming around all over the place, most people will either have forgotten about the whole long list of nV BS, or will be simply labeled a "fanboi" everytime they bring it up and their thoughts simply discarded. Credability regained. Sales up. Problem solved.

[digitalwanderer]

I notice some of you guys wondering what nVidia will do to regain their credability regarding the FM fiasco. I'll tell you right now what they'll do: Nothing. In their eyes, they've done no wrong. They even have an official statement from FM to back it up.

What about community credability? Why worry about it? In a few months, thanks to their massive PR machine and the hoards of n[V]idiots swarming around all over the place, most people will either have forgotten about the whole long list of nV BS, or will be simply labeled a "fanboi" everytime they bring it up and their thoughts simply discarded. Credability regained. Sales up. Problem solved.
No way, it won't happen. They tried that and it just didn't work.

If they do try that route, they'll die. I'll call that one right now. :)

[dream caster]

:twisted:
I think that reading this forum every day would be enough punishment and cure for Nvidia officers!

[Bouncing Zabaglione Bros.]

Ah, but that then begs the question of why they are now endorsing it!

Who says they are 'endorsing iit? *IF* NV did indeed rejoin the BETA program it is for 3DM04, not 3DM03. :wink:



In an investors/analyst conference call a few days ago, Nvidia pointed to a 5900 being top of 3Dmark2003 ORB as a "good thing". That's endorsement.
Nvidia didn't mention anything about cheating drivers in relation to the high scoring.

[digitalwanderer]

:twisted:
I think that reading this forum every day would be enough punishment and cure for Nvidia officers!
I think that nVidia officers are only allowed to read [H]ard, Anand, & THG...they don't like negative thoughts. ;)

[just me]

Ah, but that then begs the question of why they are now endorsing it!

Who says they are 'endorsing iit? *IF* NV did indeed rejoin the BETA program it is for 3DM04, not 3DM03. :wink:



In an investors/analyst conference call a few days ago, Nvidia pointed to a 5900 being top of 3Dmark2003 ORB as a "good thing". That's endorsement.
Nvidia didn't mention anything about cheating drivers in relation to the high scoring.

I didn't know that. Thanks for the info. Wouldn't remember when this was, would you? (Wondering if it was before or after FM changed the ORB recently & if after > how soon. :wink: )

I did read the excerpt of the IRQ chat where NV said they do read web boards & I'd bet this one is scrutinized. :shock:

[just me]

Wondering if it was before or after FM changed the ORB recently & if after > how soon.


:idea: :shock: :idea:

Man I'm slow. :?

Tero strikes a deal w/NV over the cheats. NV agrees not to optimize & FM says they'll not allow cheats. Neither side keeps its' word. AJ is 1 of 2 founders of FM (Markus Maki [Remedy] is the other, FWIW). Tero & AJ set benchmark policy. FM places the 5900U on top so NV can make that statement. AJ is pissed that his creation is being manipulated & that cheats were allowed & the 5900U is on top for mere marketing means. :!: {I wondered why FM was soo defensive when I asked them for proof on the 5900U being on top!}

AJ leaves & Tero is left to set policy & allow the 'optimizations'. Tero is VP, Marketing & Sales. His job is 'the bottom line' $$$.

Business is business & nice guys finnish last. (pun intended, sorry AJ)

Close enough Dave? He's going to stay on the Board of Directors tho', right? Where does Patrics' statements concerning NV & that 'demo' fit in?

[digitalwanderer]

Wouldn't remember when this was, would you? (Wondering if it was before or after FM changed the ORB recently & if after > how soon. :wink: )

I did read the excerpt of the IRQ chat where NV said they do read web boards & I'd bet this one is scrutinized. :shock:
Thursday was the conference call and Friday their stock went down 19%+ and ATi passed 'em for biggest market capitilization or some such.

There's a thread on the conference call here (http://www.beyond3d.com/forum/viewtopic.php?t=7307) and there's a thread in that link that you can listen to the conference call yourself (http://biz.yahoo.com/cc/9/32929.html).

It's the same call where they bragged about the 5900 Ultra having a 67% lead over the comptetion and that the 5900 Ultra was currently the fastest card in 3dm2k3, the "industries most prestigious graphics benchmark" or some such. :roll:

Go read, it's a fun one. :)

[just me]

Thanks Digi, been awhile. Hope this finds you & yours well. Both kids in school soon? :D

The 'saving grace' is gonna be when IBM takes a bite outta NV. :twisted:

Off to read. 8)

[digitalwanderer]

Thanks Digi, been awhile. Hope this finds you & yours well.
The family is good, but I think I'm a bit stressed. (At least I've been getting an awful lot of PMs the last few days of people telling me so... :bleh: )

Both kids in school soon? :D
Two weeks and a handful of hours, first time in about 4-5 years I'm going to actually get some type of regular time off. I'm excited as hell at the idea of going back to work...if I make it thru the next two weeks. :roll:

The 'saving grace' is gonna be when IBM takes a bite outta NV. :twisted:
Hmm? How so? I missed this angle. :|

Did you see the thread about nVidia moving down to the #2 spot behind ATi on Friday? (http://www.beyond3d.com/forum/viewtopic.php?t=7317)

FUN time to be a color commentator, tomorrow should be even better! 8)

[WaltC]

I notice some of you guys wondering what nVidia will do to regain their credability regarding the FM fiasco. I'll tell you right now what they'll do: Nothing. In their eyes, they've done no wrong. They even have an official statement from FM to back it up.

What about community credability? Why worry about it? In a few months, thanks to their massive PR machine and the hoards of n[V]idiots swarming around all over the place, most people will either have forgotten about the whole long list of nV BS, or will be simply labeled a "fanboi" everytime they bring it up and their thoughts simply discarded. Credability regained. Sales up. Problem solved.

No way, it won't happen. They tried that and it just didn't work.

If they do try that route, they'll die. I'll call that one right now. :)

Yes, nVidia's been on the PR bandwagon for months and the issue hasn't gone away, and isn't likely to, unless...nVidia begins selling elegant, competitive 3d hardware. In which case they can adopt the much easier position of letting their hardware essentially sell itself. I doubt the issues will recede until then. What nVidia could do until then to diffuse these issues is to own up to some things publicly, pledge to fix them, and move on from there. But as long as the company refuses to own up to its responsibility in these matters I have little confidence better hardware is on the way...

[CapsLock]

I called this one back when FM recanted its cheating statement. Right there you knew it was only going to get worse. In a thread at R3D I suggested to Catalyst Maker that it was time for Ati to withdraw from FM's program. They had the right to do it then and didn't of course. Now look how its developing. FM can't survive without the support of both companies. Ati could make a very prominent splash and get some very good coverage here. They would dramatically reveal the "problem" and nvidia's cheating. If they stay, they will only get taken for a ride. I also used the phrase, "They made thier bed", as Fred did. But instead of cheating in 3DM as Fred suggested, I think they should just abandon it and stand with fair benchmarks (if they can find them). Bad on FM, though of course they might have always been scr@#ed anyway, 3DMark is a battlefield and it was bound to get ugly at some point.

Caps

[hjs]

if this is true, ATI won't leave the beta program
Are ATI, Nvidia colluding to cool down graphics war

HERE'S AN INTERESTING rumour that's come our way.
Rather than battle each other hammer and tongs, sources claim that ATI and Nvidia have recently had meetings in an effort to put the brakes on the speed at which new graphics products are released.

The story goes that both companies are weary of the huge money they're putting into research and development, and the subsequent hammering they get from the gentlefolk of the press if launch dates are missed


http://theinquirer.net/?article=10952

Or is nVidia losing it and try other ways to survive?

[Fred da Roza]

Yes, nVidia's been on the PR bandwagon for months and the issue hasn't gone away, and isn't likely to, unless...nVidia begins selling elegant, competitive 3d hardware. In which case they can adopt the much easier position of letting their hardware essentially sell itself. I doubt the issues will recede until then. What nVidia could do until then to diffuse these issues is to own up to some things publicly, pledge to fix them, and move on from there. But as long as the company refuses to own up to its responsibility in these matters I have little confidence better hardware is on the way...

I doubt you will see nVidia stop their antics until they have a competitive product. Until then it will be cheat and deny to buy themselves time. Some may say that NV35 is competitive. Clearly in nVidia's eyes it isn't competative enough. Otherwise we wouldn't be seeing NV40 so soon. Three top end chips released in one year have to eat up a lot of R&D cash. Even when NV40 comes out I doubt it will stop. ATI still has a big trump card to play with their impending die shrink.

[Fred da Roza]

But instead of cheating in 3DM as Fred suggested, I think they should just abandon it and stand with fair benchmarks (if they can find them). Bad on FM, though of course they might have always been scr@#ed anyway, 3DMark is a battlefield and it was bound to get ugly at some point.

Caps

Unfortunately ATI can't stop anyone from using 3DMark as they see fit. So even if ATI disassociates themselves from it, that won't stop review sites from using it.

[Quitch]

More importantly, it won't stop OEMs from using it.

[Dave Baumann]

Close enough Dave?

Ithink the words "rampant speculation" and "running away with" might be suitable here!

[Hanners]

What about community credability? Why worry about it?

The suprising thing is that they are genuinely worried about it, or appear to be from what I've seen and heard in the last few weeks. Whether they can change and move away from the pattern they've followed for years remains to be seen, but I think they are beginning to realise where they're going wrong.

I think that nVidia officers are only allowed to read [H]ard, Anand, & THG...they don't like negative thoughts. ;)

No no, they are allowed to read other sites, but only while wearing the nVidia standard issue tinfoil hat. ;)

[digitalwanderer]

What about community credability? Why worry about it?

The suprising thing is that they are genuinely worried about it, or appear to be from what I've seen and heard in the last few weeks. Whether they can change and move away from the pattern they've followed for years remains to be seen, but I think they are beginning to realise where they're going wrong.
I totally agree with ya. (Again, go figure. :roll: :lol: )

Derek Perez has been being very nice to me in e-mails and is talking like he is aware that there is a very real problem with nVidia and the enthusiast community and that it is actually starting to affect/hurt nVidia and they want to try and do something to fix it.

Finding something to do to fix it and still staying within nVidia's corporate policies is a REAL biatch so far...I don't think they're going to be able to do it without some major corporate policy re-shaping.

But hey, that's what over-paid private consultants are for...right? ;)

[Bouncing Zabaglione Bros.]

Derek Perez has been being very nice to me in e-mails and is talking like he is aware that there is a very real problem with nVidia and the enthusiast community and that it is actually starting to affect/hurt nVidia and they want to try and do something to fix it.

Finding something to do to fix it and still staying within nVidia's corporate policies is a REAL biatch so far...I don't think they're going to be able to do it without some major corporate policy re-shaping.

But hey, that's what over-paid private consultants are for...right? ;)

If you're trying to fix the base causes of their problems, but not change any corporate policies, then it is just lip service. It means they don't want to change, they just want to "handle" the situation. Be careful you don't end up like Kyle, convinced that you are having an effect on Nvidia because they "listen to you", when in fact they are just running a PR game to try and convince everyone they are not lying.

[digitalwanderer]

If you're trying to fix the base causes of their problems, but not change any corporate policies, then it is just lip service. It means they don't want to change, they just want to "handle" the situation. Be careful you don't end up like Kyle, convinced that you are having an effect on Nvidia because they "listen to you", when in fact they are just running a PR game to try and convince everyone they are not lying.
I have the weirdest feeling that's not gonna be a problem for me, one of the greatest luxuries I give myself is I always try and be as open and honest as humanly possible when "being Dig" and it's just too valuable too me to give up. :)

That's why I'm sort of shocked at how cool he's being in the correspondence....I'm not pulling any punches about what I think is wrong and what they have to do to fix it. (Although I'm not cursing or ranting as much...if they'll try, I'll try. ;) )

Then again, he hasn't gotten very deeply into any answers yet either. :roll:

We'll see. It's interesting if nothing else. :)

[swanlee]

"Derek Perez has been being very nice to me in e-mails and is talking like he is aware that there is a very real problem with nVidia and the enthusiast community and that it is actually starting to affect/hurt nVidia and they want to try and do something to fix it. "


Here is a real simple thing they can do. STOP LYING AND CHEATING !!!.
Seems simple enough but Nvidia does not see that do they? If they want to patch things up with the community they need to let the hardware speak for itself, and stop the cheating stop the lying stop the little dirver tricks left and right. Even if the hardware is not as good as ATI they have done themselves a bigger diservice by all this crap. I think if the FX line was put out without all these cheats and didn't perform as well as ATI it would be a lot better than the corner they have painted themsevles in now.

Now even if Nvidia does produce some great hardware they have tainted the benchmark portion of this industry so much no one will ever trust them again, and now people have to scramble to try and find a set of tools that will show us what is really going on when benchmarking hardware.

Nvidia has shot thesemvles in the foot and prevented themselves from ever making an honest comeback by killing off 3dmark and cheating in every possible way.

I honestly hope that Nvidia does not get away with it and I hope they do go under and are held up as an example of what happens when companies stray to far the simple idea of putting out the best product possible. I hope they go down in flames and another 3rd party steps up to the plate to guive ATI some good legit competition.

[WaltC]

...

Derek Perez has been being very nice to me in e-mails and is talking like he is aware that there is a very real problem with nVidia and the enthusiast community and that it is actually starting to affect/hurt nVidia and they want to try and do something to fix it.

Finding something to do to fix it and still staying within nVidia's corporate policies is a REAL biatch so far...I don't think they're going to be able to do it without some major corporate policy re-shaping.

But hey, that's what over-paid private consultants are for...right? ;)

The only thing I can get out of this, dig, is that Perez is telling you he'd like to come clean but that he's being overruled by higher-ups in the company (with references to such inane cop-outs as "nVidia's corporate policies," etc.)

The thing is Perez could do much of what he's telling you he'd "like" to do by way of a press release he could write himself in which nVidia relates that it understands what it has done to create its currently poor reputation among enthusiasts, and what it is going to do to correct those internal deficiencies within the company, and to map out some specific things the company is doing for the benefit of its enthusiast customer base, which it presumably is interested in serving.

So the fact is that if nVidia was really concerned, it's so simple, and so easy, for them to do something about it. This has been true for months. That of course necessarily involves taking responsibility for what you've done and telling the truth and publicly announcing a different course for the future. Very simple, uncomplicated concepts.

Look at what ATi did at the onset of the 3dMk scandal. The company immediately admitted to a shader reordering optimization, explained it thoroughly (to my satisfaction, at least), and pledged to discontinue the practice in its drivers even though the company did not agree that it had cheated since the instruction reordering it did had no impact on the benchmark workload (which they demonstrated mathematically.) Despite Perez's intimate revelation to you of his innermost feelings on this subject, he has failed to address the issue in this easy, simple, effective manner in public, and that's the problem. It's no longer a problem for ATi because of the comparatively wise choice ATi made in talking candidly about this issue at the start. ATi did not blame nebulous "bugs", or chant that "FutureMark is out to get us," etc., or hire bottom-feeding lawyers to "speak" to FM.

Basically, you have to be very careful in entertaining what is being told to you in private when it conflicts with what a company is doing and saying *in public* about issues of this sort. Example:

Let's take frgmstr's PowerPoint slides which were leaked to him a couple of weeks ago from unidentified sources within nVidia, and which frgmstr, believing himself the recipient of some sort of "insider" information he alone was privy to, dutifully published along with very evasive editorial implication that those slides constituted that nVidia was taking internal corrective steps to stop cheating benchmarks. All of this was done without nVidia or frgmstr ever *directly* addressing nVidia's culpability for anything.

Now, let's suppose that what Perez has evidently told *you* about his hands being tied by "corporate policy," which has supposedly prevented him from doing the right thing, and his pining to you about his innermost yearning to do the right thing contrary to that policy--let's suppose for a moment that those statements are absolutely true. 100% true.

OK, then when frgmstr published the "unauthorized, against-corporate-policy-for-public-release" information contained in the Power Point slides, which presumably came from Perez's office--why was Perez not immediately fired for having violated corporate policy by the unauthorized leak of this information?....;) *chuckle*

Well, if Perez, or somebody else in nVidia PR, simply manufactured those slides using PowerPoint and made frgmstr think he was receiving "smuggled" corporate info...*chuckle*...it's not beyond the realm of possibility that those people not only retained their jobs, but got raises and promotions as well...;)

My response to individuals within companies who want to tell me something in "private" which completely contradicts what those same people are saying in public, to the public, is that I don't believe their "secret" remarks. Rather, I am somewhat insulted and incensed because it seems they want to use me to spread rumors they themselves have no intention of spreading.

[digitalwanderer]

Thanks you two, I appreciate both your inputs on it and forwarded both your posts to DP in a new e-mail. I look forward to seeing what he replies, it should tell me which way he's playing it (or me ;) ) at least. :)

[CapsLock]

But instead of cheating in 3DM as Fred suggested, I think they should just abandon it and stand with fair benchmarks (if they can find them). Bad on FM, though of course they might have always been scr@#ed anyway, 3DMark is a battlefield and it was bound to get ugly at some point.

Caps

Unfortunately ATI can't stop anyone from using 3DMark as they see fit. So even if ATI disassociates themselves from it, that won't stop review sites from using it.

I'm not an expert by any means, but I would think that if application detection can insert special code. than why not an application blocker. I would think it would be supremely easy to do (in my ignorant opinion). If you can't test with the latest new drivers, people will be unable to truly test at all, thus stopping 3dmark from being on the map whatsoever.

Quitch said:

"More importantly, it won't stop OEM's from using it."

There are other effective means of determining performance. (Popular games especially, rightmark, etc.) If Ati doesn't officially support 3DMark, they can suggest other ways. Reviewers can still do reviews and it will very soon not be missed. What the bleep is the point of allowing your card to be measured by a benchmark which is biased to the competition?
That's taking it a little too politely if you ask me, not to mention the harm it does to your bottom line and reputation.

Caps

[Fred da Roza]

My response to individuals within companies who want to tell me something in "private" which completely contradicts what those same people are saying in public, to the public, is that I don't believe their "secret" remarks. Rather, I am somewhat insulted and incensed because it seems they want to use me to spread rumors they themselves have no intention of spreading.

I completely agree. As if this insider information was not intentionally leaked. Give me a break.

If they really want to regain the trust of their customers just release drivers without cheats that aren't encrypted. It's really that simple.

[digitalwanderer]

My response to individuals within companies who want to tell me something in "private" which completely contradicts what those same people are saying in public, to the public, is that I don't believe their "secret" remarks. Rather, I am somewhat insulted and incensed because it seems they want to use me to spread rumors they themselves have no intention of spreading.

I completely agree. As if this insider information was not intentionally leaked. Give me a break.

If they really want to regain the trust of their customers just release drivers without cheats that aren't encrypted. It's really that simple.
I like that attitude, I think I'll adopt it if'n y'all don't mind. :)

[Wolf]

Perez is an idiot. I can't believe you would trust any word out of that mans mouth. His hands are tied is bs. I believe he is almost soley responsible for spewing the lies and deceit from Nvidia for years and he knows it. I wouldn't trust him for one second.

[Bouncing Zabaglione Bros.]

Anyone who wants to change a company but not change a company's internal policies either (a) has no power to change anything, or (b) has no intention of changing anything.

The behaviour of a company is governed by it's policies, and those policies are set by the management. Nvidia cheats on benchmarks because the the company ethos sets the policy that this is an "okay" thing to do.

Digi, if Perez says he wants to fix things, but he can't/won't change the policies (and the attitude/ethos behind those policies), then he's just spinning you along. There is no intention or posibility of changing things if they won't change their policies and attitudes, because that is where the very root of the problem lies, and that is where it needs to be fixed. Anything else is just the window dressing of patching the symptoms here and there. Perez doesn't want to stop Nvidia from cheating and lying, he just want a way to make the community stop hating Nvidia for it.

Like a person who says "I'm sorry you feel that way", it's not an apology. It contains the word "sorry" but it still makes things your fault, and it refers to the other party wishing you wouldn't feel upset at something, rather than any kind of remorse for that same behaviour.

Don't forget that Nvidia is now a marketing-led company. That means anything goes if it gets sales. If that means cheating on 3DMark, then that's okay. If it means lying to DigitalWanderer in the hope of creating another Kyle-type spokesman, then that's okay too.

Until Nvidia actually do and say something, instead of all these rumours, driver leaks and misleading comments by proxy of lapdog websites, it's all just empty words in my opinion.

[digitalwanderer]

Hmmm....so y'all trying to tell me to make sure I don't exactly take everything DP says at face value? :shock:

;) :lol:

EDITED BITS: Ooops, I forgot to say "Thank you" also. I do appreciate all the advice and everybody keeping me grounded in reality and all instead of listening to empty words. Words won't work for me either, at least not ones in private. That's one of my points in my discussions with him that is sort of a sticking point of mine too. Sorry I'm being so silly & sarcastic, it's just been a looooong weekend that ain't quite over yet and me kids got me a BIT stressed out. Only 13 days to go. :roll:

[swanlee]

I say DW leach off him get every bit of info you can on this situation then turn around and expose them. Do kind of like an Anti Kyle and reverse the roles or something. would be nice if one of their lame exploits back fired on them.

[Dave Baumann]

Personally, I wuld say that its difficult to get a true large scale change in policy without a substancial management overhaul.

[digitalwanderer]

I say DW leach off him get every bit of info you can on this situation then turm around and expose them. Do kind of like an Anti Kyle and reverse the roles or something. would be nice if one of their lame exploits back fired on them.
Nah, I can't break a private cofidence. It wouldn't be right. :(

I will talk to him though. :)

[Dave Baumann]

NVIDIA Joins Futuremark Beta Program (http://www.beyond3d.com/forum/viewtopic.php?t=7374)

[just me]

Posted @ FutureMark:

Beyond3D usually have very interesting hardware reviews, but as far as rumours go, you should never believe them 100%. Forums are forums, people may say what they want, and that means seldom all 100% facts. Dave is something like the reponsible editor of B3D, but in the forums, he may write about his opinion on things, or quote rumours he has heard.

If you want the facts, read press releases or official statements of the company it concerns. Forums are for less formal discussion. Anything about Futuremark written in the Beyond3D forums should not be taken as facts. We'll make an announcement when there is something significant to tell about our company.

A good example of rumour and an official announcement, is the talk about Nvidia re-joining the 3DMark development program. There have been widely spread rumours about it for some time already, but the truth is, that during that time only negotiations were ongoing, they had not joined yet. Now the negotiations are completed and we are happy to announce it in a press release:
http://www.futuremark.com/pressroom/pressreleases/?081203

Anything else you might have read in some other forum about Futuremark, is most likely not quite true, not until we have officially confirmed it.

Patric - 3DMark®03 Producer

[Bouncing Zabaglione Bros.]

Oh - so Patric says "don't believe everything you read, and by the way IT'S ALL TRUE!!"

:roll:

[incurable]

If you want the facts, read press releases or official statements of the company it concerns. [...]

Patric - 3DMark®03 Producer
Is that a joke or something? :roll:

cu

:roll: incurable :roll:

[Reverend]

I think Patric is just concerned that the posted opinions of folks (including Dave, and especially FM's beta members) in public forums may be read as representative of the opinions of personnels at FM. I'm sure many of you trust most of what Dave has to say/post but even Dave will tell you he doesn't know all the details about what he says/posts.

[digitalwanderer]

Oh - so Patric says "don't believe everything you read, and by the way IT'S ALL TRUE!!"

:roll:
LOL~~~

Yeah, that's what I read too. :)

[kemosabe]

So does the staff at B3D expect to be consulted about these new optimization "guidelines" FutureMark is drafting for our collective benefit?

http://www.extremetech.com/article2/0,3973,1217342,00.asp

"Our next big thing", indeed. :roll:

[micron]

Sarkkinen said FutureMark's reputation has not been tarnished by the spat. "I don't feel like that at all," he said, noting that the feedback from both OEMs and hardware review sites has been positive, and that users have contributed a record number of benchmark submissions to the FutureMark database.
:?

[banksie]

What interests me more from the extreme tech article is this :-

"Nvidia, meanwhile, has issued statements claiming that the original FutureMark optimizations were the unsupervised work of a programmer, and the company has set down clear public guidelines over the use of such optimizations."

Where have they claimed that? This is the first I have heard of it.

Philip

[just me]

:lol:

So Tero is setting 'optimization' policy & working w/NV. Just as I thought! Of course he would since everyone else @ FM is in Europe & Tero is a few miles away from NV in California. :wink:

Odd that ET said 'BETA Program' when NV isn't listed as a member of the "BETA Program" by FM & FM says they joined the "Benchmark Development Program". Semantics? No. Legalese! :wink:

Odd that FM hasn't posted here on this. Patric has his own forum even. I did see someone associated w/FM here yeaterday (not worm either) but he didn't post anything. :D

.02,

[digitalwanderer]

What interests me more from the extreme tech article is this :-

"Nvidia, meanwhile, has issued statements claiming that the original FutureMark optimizations were the unsupervised work of a programmer, and the company has set down clear public guidelines over the use of such optimizations."

Where have they claimed that? This is the first I have heard of it.

Philip
I believe ET is referencing those three power-point slides that Kyle put up on hard and how Kyle said nVidia said/implied those things. (I don't remember exactly)

[Dave Baumann]

So does the staff at B3D expect to be consulted about these new optimization "guidelines" FutureMark is drafting for our collective benefit?

I've already talked to Tero about the general gist of it, although I'm not sure of how much I should talk about now. Given the situation with NVIDIA ts probably about all they can do (its further muddied by the fact one of the major OEM's allow something as well).

[Slides]

(its further muddied by the fact one of the major OEM's allow something as well).

Dave, could you clarify this statement please? What do you mean by "allow something"?

[Nick[FM]]

Just to make sure that you (actually all of you) don't get the wrong picture, I'll comment on these "misunderstandings":


So Tero is setting 'optimization' policy & working w/NV. Just as I thought! Of course he would since everyone else @ FM is in Europe & Tero is a few miles away from NV in California. :wink:
All/any policies are set in cooperation with all our Beta Members.

Odd that ET said 'BETA Program' when NV isn't listed as a member of the "BETA Program" by FM & FM says they joined the "Benchmark Development Program". Semantics? No. Legalese!
NVIDIA re-joined our Development Program, which is the same as our Beta Program. The NVIDIA logo will appear on the Beta Program page asap. Refresh the page in an hour or two! :wink:

Odd that FM hasn't posted here on this. Patric has his own forum even. I did see someone associated w/FM here yeaterday (not worm either) but he didn't post anything.
I was here yesterday, and now I have posted about this. :)

[Mark]

]Just to make sure that you (actually all of you) don't get the wrong picture, I'll comment on these "misunderstandings":


So Tero is setting 'optimization' policy & working w/NV. Just as I thought! Of course he would since everyone else @ FM is in Europe & Tero is a few miles away from NV in California. :wink:
All/any policies are set in cooperation with all our Beta Members.

Odd that ET said 'BETA Program' when NV isn't listed as a member of the "BETA Program" by FM & FM says they joined the "Benchmark Development Program". Semantics? No. Legalese!
NVIDIA re-joined our Development Program, which is the same as our Beta Program. The NVIDIA logo will appear on the Beta Program page asap. Refresh the page in an hour or two! :wink:

Odd that FM hasn't posted here on this. Patric has his own forum even. I did see someone associated w/FM here yeaterday (not worm either) but he didn't post anything.
I was here yesterday, and now I have posted about this. :)
I'm sorry, but none of that makes me feel any better about me spending my money on a copy of 3DMark03. Frankly, I feel like I've been cheated out of my money... no not optimized out of it, cheated. Where can I apply for a refund?

[Nick[FM]]

I'm sorry, but none of that makes me feel any better about me spending my money on a copy of 3DMark03. Frankly, I feel like I've been cheated out of my money... no not optimized out of it, cheated. Where can I apply for a refund?
I fail to understand why you feel like you have been cheated?

[Quitch]

I fail to understand why you feel like you have been cheated?

If you mean that, then I feel very scared.

Oh - so Patric says "don't believe everything you read, and by the way IT'S ALL TRUE!!"

LOL! That's what I was thinking :)

I think Patric is just concerned that the posted opinions of folks (including Dave, and especially FM's beta members) in public forums may be read as representative of the opinions of personnels at FM. I'm sure many of you trust most of what Dave has to say/post but even Dave will tell you he doesn't know all the details about what he says/posts.

I suspect thay're worried that what Dave says IS representative of how many FM employee's feel.

[K.I.L.E.R]

]I'm sorry, but none of that makes me feel any better about me spending my money on a copy of 3DMark03. Frankly, I feel like I've been cheated out of my money... no not optimized out of it, cheated. Where can I apply for a refund?
I fail to understand why you feel like you have been cheated?

I was going to make an optimisation joke but I believe that's been done to death.
I don't feel cheated spending the money. :P

Can I have a cookie? :)

[digitalwanderer]

]I'm sorry, but none of that makes me feel any better about me spending my money on a copy of 3DMark03. Frankly, I feel like I've been cheated out of my money... no not optimized out of it, cheated. Where can I apply for a refund?
I fail to understand why you feel like you have been cheated?
Because you changed the rules of how the benchmark was to be run AFTER they purchased it, and some of us feel that by changing the rules you've invalidated 3dm2k3's usefullness as a tool...as well as all the rampant speculation about upcoming favoritism on FM's part towards nVidia.

Has ATi contacted you people yet today? I'm kind of waiting for that shoe to drop.... :twisted:

[just me]

Thanks worm. 8)

So what is this "Benchmark Development Program" Tero spoke of & is quoted in FM's press release for the world to see? Is there such a thing or is Tero making up names as he goes along? :?

Lemme see if I got this part right: 'optimizations' are going to be determined 'en masse' by all the members of the 'BETA Program'? Does that mean 100% must agree or just a 50.0001% majority? Once the BETA Members agree, then Tero will announce the 'new & improved' way it is meant to be benched? :wink:

Thanks for stopping by to answer ?'s. Nice to see someone from FM is still posting here. 8)

[Doomtrooper]

] I fail to understand why you feel like you have been cheated?

http://burns.thefinaldimension.org/contrib/edoom/theyareontome.gif

[WaltC]

]Just to make sure that you (actually all of you) don't get the wrong picture, I'll comment on these "misunderstandings":

All/any policies are set in cooperation with all our Beta Members.

Does this mean they all have to agree on something before it's implemented by unanimous assent, or that you deal with your IHV partners individually?

Also, are there any policies/standards which FM sets itself independently of your partners which they must adhere to in order to maintain their participation in your program (aside from paying their dues)? IE, do you set the standards or do they?

NVIDIA re-joined our Development Program, which is the same as our Beta Program. The NVIDIA logo will appear on the Beta Program page asap. Refresh the page in an hour or two! :wink:

What brought this about--has nVidia changed its demands, or have you simply agreed to theirs? Something has changed nVidia's mind about your company--explaining what that was in your opinion would go a long way towards diffusing misunderstandings.

[digitalwanderer]

]NVIDIA re-joined our Development Program, which is the same as our Beta Program. The NVIDIA logo will appear on the Beta Program page asap. Refresh the page in an hour or two! :wink:

What brought this about--has nVidia changed its demands, or have you simply agreed to theirs? Something has changed nVidia's mind about your company--explaining what that was in your opinion would go a long way towards diffusing misunderstandings.
Or go a long way towards providing new ones. :roll: (I am just NOT feeling very optimistic about this...sorry. )

[WaltC]

Or go a long way towards providing new ones. :roll: (I am just NOT feeling very optimistic about this...sorry. )

Well, right now I'm fairly ambivalent about the situation. I would at least (speaking for myself) like to provide FM some time to explain this turn of events fully and completely. Hopefully, they won't be coy about it and pretend there's nothing to explain to people and will recognize the value of full disclosure in this case. I'm genuinely puzzled though by the fact that it's not clear the company understands that its whole program rests on the bying public believing their benchmarks have something credible to say about 3d technology. I mean, if the general public loses confidence in their software, surely they know that their partners will follow suit...? I guess we'll see...

I'm hoping they are not so naive as to believe they can turn back the clock to how things were before...not a chance, IMO...The genie's out and there's no getting him back in the bottle.

[Patric Ojala]

Wörm told me there is an interesting discussion going on here, since I'm a bit lazy to follow all forums personally :D

As usual, I'll comment in the nice personal forum Dave and the guys gave me (thanks again). Check it out in a few minutes, I'll start typing right away.

[K.I.L.E.R]

I'm waiting Mr Ojala. :D
BTW: Where's my cookie? :P

[madshi]

As usual, I'll comment in the nice personal forum Dave and the guys gave me (thanks again). Check it out in a few minutes, I'll start typing right away.
I've no problem with NVidia rejoining FutureMark's beta program. I agree, that this is the best solution to get an impartial 3DMark04/05/06.

The main problem I currently have (and I think most of the other guys here think similarly) is about the driver optimizations. First FutureMark said, 3DMark03 specific driver optimizations like shader replacements would not be allowed at all. ATI even took their (mathematical correct) replacement shader(s) out of their drivers. But now after that joint NVidia/FutureMark statement NVidia is putting out one driver after the other which all seem to have most optimizations including replacement shaders back in. And FutureMark sits around and seems to accept that. Then have a look here: http://www.theinquirer.net/?article=11037 . Now where does that leave ATI? NVidia currently has the top performing card in 3DMark03 and everyone knows that this is bullshit, since ATI's cards have a much better shader performance. Could you please add some comments about these problems? Because these are the *real* problems IMHO. Thank you!

[Patric Ojala]

The main problem I currently have (and I think most of the other guys here think similarly) is about the driver optimizations. First FutureMark said, 3DMark03 specific driver optimizations like shader replacements would not be allowed at all. ATI even took their (mathematical correct) replacement shader(s) out of their drivers. But now after that joint NVidia/FutureMark statement NVidia is putting out one driver after the other which all seem to have most optimizations including replacement shaders back in. And FutureMark sits around and seems to accept that. Then have a look here: http://www.theinquirer.net/?article=11037 . Now where does that leave ATI? NVidia currently has the top performing card in 3DMark03 and everyone knows that this is bullshit, since ATI's cards have a much better shader performance. Could you please add some comments about these problems? Because these are the *real* problems IMHO. Thank you!

Fair enough, I think I understand you concern.

First of all, we need to be very careful about what public statements we make. This industry is full of eager writers, who seem to spend their days trying to read between the lines and dig out some issues from anything and everything that is published. If we don't immediately comment some new turn in the industry, it might be due to the fact that:
a) We don't know yet the whole extent of the issue and we're still trying to get to the bottom of it. This is usually what's happening when we don't react to something the very next second.
b) We see that the issue in question as not ours to comment. Our goal is to provide the industry with tools for hardware and driver testing. Analyzing the results and judging what's good or bad is what the professional hardware press is for.
c) The whole issue was raised due to some wild rumour, and we don't see any need to comment the issue at all.

The whole shader optimization issue could have gone much smoother if (well, if there wouldn't have been any tension between us and Nvidia, but that's beside the point) the gfx IHVs would have first discussed the issue with us and then implemented the shader replacements. We were asked to do some scrambling of the shader instructions, in order to find out who does direct shader identification and replacement + this was at the time considered an all bad thing. Further discussions have shown, that under some circumstances (that we're trying to define at the moment) shader replacements can be accepted. If these discussions would have been held first, and shader replacement would have been implemented in drivers only afterwards, I doubt there would have been problems like the ones we ran into. Also, the whole shader replacement thing was a brand new issue. Nobody knew at the time how to react to it. This is a fine example of why it is important that we don't impulsively comment all issues immediately, rather first consider them carefully. And this is why our response might take some time, but it does not always mean that we don't give a d---. It does mean that sometime though :lol:

[CorwinB]

Wörm told me there is an interesting discussion going on here, since I'm a bit lazy to follow all forums personally :D

As usual, I'll comment in the nice personal forum Dave and the guys gave me (thanks again). Check it out in a few minutes, I'll start typing right away.

I've just read your interesting post in your personal forums, and I have a few questions left :
a) what are (if you can answer) your "gut feelings" regarding the "optimizations" in drivers
b) where will 3DMark 2003 go from here ? I agree that NV rejoining the Beta program can be a good thing, but only if they play nice. If they continue with the various "optimizations" while your other partners are denied the same thing, then you have to agree that the benchmark loses all meaning.
c) What does FM intends to do to convince people that their benchmark still is relevant ? Will you change your stance of what "optimizations" are acceptable ? How will you check that IHVs don't pull dirty driver tricks ? You mention a large number of times how happy you are that NV is rejoining. But you seem to forget (or you dodge the issue) that there is another thing you need besides major IHVs participating in your beta program : you need the public perception that your benchmark is an useful tool for evaluating video adapter performance. And, as of now, this is not the case : some people (mostly due to Nvidia bashing 3DMark and synthetic benchmarks, which they still have to apologize for) now have a very [h]arsh stance against 3DMark, and other, while still being convinced that synthetic tools are useful, also don't believe in 3DMark anymore because of the 180° on the "C" word and the lack of communication regarding driver updates since... For many people, FutureMark seems to have absolutely zero pride regarding its products and their butchering through various "optimizations". How do you plan to adress this ? What about FM developing a demo for NV as was rumored ?

Edit : just seen your post

Also, the whole shader replacement thing was a brand new issue.

While *shaders* on personal computers are technically a new thing, "cheatoptimizing" in benchmarks (any benchmark, not just GFX) is not. You would think a company whose entire business model is supposed to rely on accurate benchmarking tools would have been prepared...

[digitalwanderer]

we need to be very careful about what public statements we make. This industry is full of eager writers, who seem to spend their days trying to read between the lines and dig out some issues from anything and everything that is published.
And here I didn't think anyone noticed me, I'm touched! :)

[madshi]

...
Thank you for your comment. I understand the problem and I understand your situation. I just hope that you get it all sorted out soon. Because right now the order of the graphics cards in your hall of fame is somehow "wrong" (if you know what I mean :wink:). And that's probably the reason why lots of people currently consider 3DMark03 as being invalid - which I find a pity...

[swanlee]

I think you guys are opening a huge can of worms here by agreeing to Nvidia's terms about optimizations and that your original stance of no cheating allowed ever was the proper response.

You guys put out a GPU Benchmark based on DX8/9 API and that is what you should stick to. By allowing any optimizations you open your program up to the fact that games might not be optimzed the same way or considering the past that these might open up the fact that a crappy card can do well with specific optimizations in the dirvers.

Your program is a benchmark and any optimizations are for the sole purpose of increasing the benchmark score which may or may not be reflective in any games.

You guys need to stick to the DX API' standards and gear your benchmark toward these standards. And if another companies hardware performs badly using DX standards then that is just plain tough most games will be geared using these standards and not developed specifically for one vendors cards.

You are ruining the integrity of your product by allowing this.

[Tridam]

First of all, we need to be very careful about what public statements we make. This industry is full of eager writers, who seem to spend their days trying to read between the lines and dig out some issues from anything and everything that is published.

This industry is also full of 'false marketing truths'. It's not always easy to deal with them. We need to read between the lines to separate true facts from marketing facts. We need to do this with every official statement.

Of course, quoting a board post from someone who works for company xxx and trying to dig out issues from it is really unfair. But what must we do when some board posts are very different from official statements? We can't ignore them…

If we don't immediately comment some new turn in the industry, it might be due to the fact that:
a) We don't know yet the whole extent of the issue and we're still trying to get to the bottom of it. This is usually what's happening when we don't react to something the very next second.
b) We see that the issue in question as not ours to comment. Our goal is to provide the industry with tools for hardware and driver testing. Analyzing the results and judging what's good or bad is what the professional hardware press is for.
c) The whole issue was raised due to some wild rumour, and we don't see any need to comment the issue at all.
d) It's not good for our business to say bad things about one of our partners. :D

It's always a problem for a company when there are some concerns about their partners. You have commitments with your partners (NVIDIA, ATI, AMD, Intel, …) and with everyone who use or buy 3dmark. Interests of your partners and interests of your users are different. How do you deal with that ?

The whole shader optimization issue could have gone much smoother if (well, if there wouldn't have been any tension between us and Nvidia, but that's beside the point) the gfx IHVs would have first discussed the issue with us and then implemented the shader replacements. We were asked to do some scrambling of the shader instructions, in order to find out who does direct shader identification and replacement + this was at the time considered an all bad thing. Further discussions have shown, that under some circumstances (that we're trying to define at the moment) shader replacements can be accepted. If these discussions would have been held first, and shader replacement would have been implemented in drivers only afterwards, I doubt there would have been problems like the ones we ran into. Also, the whole shader replacement thing was a brand new issue. Nobody knew at the time how to react to it. This is a fine example of why it is important that we don't impulsively comment all issues immediately, rather first consider them carefully. And this is why our response might take some time, but it does not always mean that we don't give a d---. It does mean that sometime though :lol:

I fully agree.

But now, with all these issues, Futuremark and 3dmark has unfortunately lost a part of its "credibility". Do you plan to do something to get it back?

I think that it could be very easy for you

1. Let IHV use good optimisations. -> You must say clearly what you'll consider as a good optimisation and do everything to avoid the use of bad optimisations.
2. Add an optional path in 3dmark with no optimisations and ask IHV to respect this path.
3. Add the possibility to switch from standard precision to partial precision.
4. Add the fly mode in the public version. It's really easy to use optimisations which look good only in some circumstances. The fly mode is an easy way to check for optimisations problems.

It'll be very easy for you to do this with the standard or the professional version of 3dmark03. Why don't you do something like that?


I have another question :) :

What's the main objective of 3dmark03: show how a card will run current and future games or show the 3d abilities of a graphic card?

The difference between these 2 objectives is small but very important. For example : I think that everyone knows that the floating point power of Radeon are higher than floating point power of GeForce FX. DX9 shader are floating point shader. Games developers will have to use complex DX9 shader not too much because a big part of the market won't have the ability to run them at a sufficient speed. Do you think that a benchmark has to do the same thing?

[Reverend]

I am not breaking any NDAs by saying the following! :) :) ;)

Futuremark is still studying what is meant by "optimizations" (i.e. everything that this incredibly important word means). Beyond3D is also studying it. We are all also studying if such "optimizations" should be app-specific or application-wide (generally, but even "generally" can be an important word).

I think we can all gather that FM, as well as the entire industry, is still learning. That's a Good Thing... hopefully, the results will be enlightening... as well as determinant (in many regards).

[swanlee]

Application specific optimizations cannot be a good thing in a benchmark. Because if they are truely application specific and what you are running is a benchmark the only reason for these optimizations is to bump up the score. The only acceptable optimization that would effect a benchmark would be one that effects other games in the same way, but then it would not be an application specific optimization it would either be an api optimization or possibly a game engine optimization.

I think either way you cut it Nvidia is trying to force it's will on 3dmark and 3dmark are folding. If nividia's hardware was better it would follow dx9 specs and not have so much toruble with 3dmark and then Nvidia would not have to optimize a freakin benchmark to get decent scores.

This all boils down to nvidia's newest hardware is not up to snuff and Nvidia cannot accept 2nd place so they must push there weight around to change the way benchmarks are done so they look better.

[Malo]

We are all also studying if such "optimizations" should be app-specific or application-wide (generally, but even "generally" can be an important word).

rev nvidia themselves stated on those slides that these "optimisations" (which will forever be in quotations) must accelerate more than just a specific application. I really don't understand how FM can allow such an app specific hack when nvidia promise otherwise.....

[micron]

swanlee,
Thanks for enlightening us.

[jb]

Patric Ojala,

I am sadden that you let any optimizations in. You folks made it very clear in your first public statement and the 330 patch. That sets a precedent. Then you back off these statements and now are chaging your word to allow optimizations in. This is very bad. A benchmark that is sythentic should never allow any optimizations of any kind. You ask the GPU to do a certian amount of work, in a certian way in a given time. Buy sticking to a very firm stanace you create a controlled enviroment that very usefull. However now one vendor is compling with your first statement while the other is not. The fact that you let this stand adds bias to your benchmark weather you like it or not :(

[digitalwanderer]

Thanks Rev, I keep forgetting you folks are active with FM behind the scenes.

I'll reserve judgement until some more info is known....if B3D suddenly pulls out of FM I'll consider that pretty telling even if no one can say a thing due to NDAs. ;)

[Tridam]

Actually, some DX9 shader optimizations are good because of the lack of a runtime compiler in DX9. ATI can optimize for co-issue and NVIDIA can optimize for register usage. These ones are good optimizations.

[WaltC]

Patric,

I just read your post in your personal forum here at B3d, and I've read your comments in this thread as well. The following was among the most troubling of your remarks, and the most disappointing to me personally:

...Immediately as Nvidia left our development program, getting them back into the program was raised to a very high priority. Our highest priority naturally is and remains the development of high quality and impartial benchmarks.

I'm trying to get you readers to understand what a great thing it is for us that Nvidia re-joined our development program. I think this is the best thing our company has achieved since we launched 3DMark03. The Futuremark employees went through some tough times back there, but we're all happy to get Nvidia back, including AJ (even though some weird rumours tell differently) .Immediately as Nvidia left our development program, getting them back into the program was raised to a very high priority. Our highest priority naturally is and remains the development of high quality and impartial benchmarks.

I have to tell you I find this statement simply baffling. nVidia belonged to the 3dMk03 development program for at least 15 months out of the 18-month 3dMK03 development cycle (as they didn't quit the program until December of '02.) What was it, precisely, that you could obtain from nVidia by "getting them back" into the program that would allow you to make 3dMK03 a "high quality and impartial" benchmark that you failed to obtain from nVidia during the long months in which it belonged to the FM program during the 3dMK03 development cycle? I simply cannot imagine what that might be. Please elaborate.

nVidia was in the 3dMK03 development cycle long enough to learn how to cheat the benchmark with dispatch when it shipped--it boggles the mind how you might think that getting nVidia back had anything at all to do with making 3dMK03 a "high quality and impartial" benchmark, since presumably that's what it was when it shipped.

How is nVidia being in or out of the program relevant to 3dMk03 being a high-quality benchmark as you define it, since nVidia was a full partner for virtually the entire development cycle of 3dMK03?

Hopefully, you are not declaring that 3dMK03 when it shipped was anything but a high-quality and impartial benchmark...? This is baffling for me, honestly.


I'm trying to get you readers to understand what a great thing it is for us that Nvidia re-joined our development program. I think this is the best thing our company has achieved since we launched 3DMark03. The Futuremark employees went through some tough times back there, but we're all happy to get Nvidia back, including AJ (even though some weird rumours tell differently) .

It should be obvious why Nvidia is so important for our development program, but I'll repeat some of the key reasons for those unfamiliar with our company's business.

In order to produce top quality forward looking benchmarks, we need the input and cooperation of all the major players of the industry.


So, how is it you did not receive "input and cooperation" from nVidia for 3dMK03 development when nVidia was paying you for the privilege of providing such throughout the 3dMK03 development cycle? I'm sure I don't have to remind you that the events which occurred prior to nVidia leaving your program, prior to 3dMk03 shipping, prior to nVidia cheating the benchmark, and prior to your audit report detailing exactly how they cheated it--prior to these things nVidia was just as much a part of the FM program as it is today.

Your supposition seems to be that you require nVidia's participation to be able to write a good benchmark, and yet you had nVidia's participation when you wrote 3dMK03 and before *nVidia* made the elective decision to pull out and discredit your company's software.

So if nVidia was unable to assist you in writing a high-quality, impartial benchmark prior to resigning from the program, what has changed so that now nVidia's participation is *required* to produce that result?

How could we otherwise get the highly confidential information of where the industry is going next? Each 3DMark version is aimed at the new if not next generation of hardware, and it is of crushial importance to have comprehensive information of that next generation, in order to make a benchmark for it in advance. In addition to making just high quality benchmarks, they must be impartial, and it is not convincing to release an impartial benchmark developed in cooperation with just one of the major IHVs. If we would not have got Nvidia back, we would have still done our best to optimize it also with Nvidia's upcoming products in mind, but it would have been way harder.

Again, this does nothing to explain the puzzling issue you've raised, which is why you failed to obtain that "comprehensive information of that next generation, in order to make a benchmark for it in advance" from nVidia during the development of the 3dMK03 software (since nVidia belonged to the program for the bulk of the development cycle.) Of course, the truth is you did obtain it, but that nVidia wasn't happy with what you did with the "comprehensive information" and so it quit your program. That's my take on it based on what you've said.

Of course you've glossed over the very odd structure of your business model--which is that companies pay you for the privilege of providing their own next-gen information to you so that ostensibly that information will be used in the construction of a benchmark which will portray their products in a positive light and help them sell those products. Hopefully, you can see the glaring conflict of interest in doing that and in providing a "high-quality, impartial" benchmark for the 3d-card buying public. To that end I can see nVidia's point in quitting--their view was that they were paying you for specific services which you were no longer providing them. It's the conflict of interest inherent in your business model which has caused all of this to happen, IMO.

Sadly, I must regrettably conclude that nVidia's participation is a requirement for your company because of nVidia's financial contribution to your company based on the membership fees it pays you. Obviously, I guess you've agreed to start giving nVidia its money's worth and nVidia is now back in. Seriously, unless you guys start doing some serious "rethinking" about your business model and understand the inherent conflict of interest within it I don't see much of a future for you in this endeavor, and I see more of this same sort of trouble for you on the horizon.

[micron]

ouch.

[Mark]

@Futuremark guys (worm, Patric, and whoever else).

I paid for 3DMark03 because at the time it provided honest and meaningful results for my videocard reviews. Now, after the deals made with nVidia to allow optimizations for the benchmark, those results are difficult to understand and compare. The fact is 3DMark03 simply can't be trusted to provide those honest results anymore. Therefore, through Futuremarks own actions (and inactions), the 3Dmark03 program is now worthless and my money has been wasted.

[digitalwanderer]

@Futuremark guys (worm, Patric, and whoever else).

I paid for 3DMark03 because at the time it provided honest and meaningful results for my videocard reviews. Now, after the deals made with nVidia to allow optimizations for the benchmark, those results are difficult to understand and compare. The fact is 3DMark03 simply can't be trusted to provide those honest results anymore. Therefore, through Futuremarks own actions (and inactions), the 3Dmark03 program is now worthless and my money has been wasted.
What I'm interested in seeing is how the FM people react to such a post, because there are a whole bunch of similar minded folk out there right now who are waiting for an answer/some reassurance and it's another case of the silence being all deafening. :(

Well, 'cept for nVidia who keeps floating rumors... :?

Seriously, does FM plan to react to this or address it in some way soon?

[micron]

Whats interesting is that on the Futuremark forums, there were members there who owned 5900Ultra's, and were'nt aloud to submit scores. I'm talking about people who purchased the pro version, and couldnt even use it to it's intended functionality. Some were requesting to get their money back, it never happened. What a tough situation for Futuremark......

[digitalwanderer]

Whats interesting is that on the Futuremark forums, there were members there who owned 5900Ultra's, and were'nt aloud to submit scores. I'm talking about people who purchased the pro version, and couldnt even use it to it's intended functionality. Some were requesting to get their money back, it never happened. What a tough situation for Futuremark......
Not FMs fault, blame that on nVidia. FM's policy on allowed submissions was quite clear, and the drivers in question did not qualify. ;)

[micron]

Whats interesting is that on the Futuremark forums, there were members there who owned 5900Ultra's, and were'nt aloud to submit scores. I'm talking about people who purchased the pro version, and couldnt even use it to it's intended functionality. Some were requesting to get their money back, it never happened. What a tough situation for Futuremark......
Not FMs fault, blame that on nVidia. FM's policy on allowed submissions was quite clear, and the drivers in question did not qualify. ;)
FM needs to put a clause in their 'Pro' purchase contract that warns of score submision refusals if any of the IHV's become cheating bastards.
I dont think any of the guy's who couldnt submit scores saw this coming.
I woulda been pissed if I was them...
It's Futuremark's problem.
(edit)
I understand what you are saying :wink: , I'm simply trying to put myself in the position of the people who were not aloud to submit scores.

[Malo]

It's Futuremark's problem.

it's FM's problem that the drivers didn't meet the standards required for a fair and comparable benchmark rating? please explain....

[micron]

I'm saying that it sux to have a new videocard and not be allowed to submit scores, after you've paid for the pro version.
It doesnt really matter now because FM fixed the problem :wink:
The scores are being submitted, and everyones happy :D

[digitalwanderer]

FM needs to put a clause in their 'Pro' purchase contract that warns of score submision refusals if any of the IHV's become cheating bastards.
Yeah, but I was talking about the bit in their policy where they state they only accept results from officially released WHQL drivers. ;)

[micron]

FM needs to put a clause in their 'Pro' purchase contract that warns of score submision refusals if any of the IHV's become cheating bastards.
Yeah, but I was talking about the bit in their policy where they state they only accept results from officially released WHQL drivers. ;)
I know you were, and I understand FM's policy.
The pro version end-users were still complaining constantly, and it was something that FM had to deal with. And they did.

[digitalwanderer]

FM needs to put a clause in their 'Pro' purchase contract that warns of score submision refusals if any of the IHV's become cheating bastards.
Yeah, but I was talking about the bit in their policy where they state they only accept results from officially released WHQL drivers. ;)
I know you were, and I understand FM's policy.
The pro version end-users were still complaining constantly, and it was something that FM had to deal with. And they did.
I don't follow FM's boards, how did they deal with it?

[micron]

They got back in bed with Nvidia.

[digitalwanderer]

They got back in bed with Nvidia.
Ok, I knew that. (I guess I HAVE been following it a bit better than I thought. ;) )

[WaltC]

ouch.

Actually, in thinking a bit more about the conflict of interest inherent in the business model, it strkes me there are actually two major conflicts of interest there.

The first is that fact that while the assumption of the IHV partners is that they are paying for software which can be used as a marketing tool to portray their products in a positive light, the public is thinking they are using software which provides a level playing field for an ojective evaluation of the features supported by differing 3d hardware in a comparative context.

The second is that you have competing IHVs paying dues, but expecting entirely different things from the software. For instance, nVidia pays x dollars and tells FM to "go this way" with its software, and ATi pays FM x dollars and tells them to "go that way" with its software. So what does FM do? If they go "this way" they risk alienating ATi and having ATi quit the program, but if they go "that way" with it they risk alienating nVidia and having nVidia quit the program (which has already happened.) Last, if they take the middle route and split the difference between them then they risk alienating both of the IHVs and possibly having both of them quit the program.

Pre-R300 I'm guessing that conflict of interest between IHVs was not that much of a problem since nVidia had a pretty good lead at the high end. But now that ATi is emerging as dominant, I can see several bones of contention developing along the lines that prompted nVidia to quit last year. As the competition intensifies between the IHVs my thought is that the situation isn't likely to subside, but rather to grow more acrid.

The funny thing about companies is when they shell out money for programs like this they naturally expect some tangible benefit in the way of goods and services in return. If they don't perceive they are getting such benefits they lose the motivation to pay for the privilege of participating. Looks like a thorny problem for FM going forward.

I think the best solution therefore would be for them to change their business model and deal with the IHVs on a voluntary basis through NDAs and drop them from the partnership dues-paying rolls. They could maintain their partnership with oems like Dell (and other entities) for the purposes of sales and distribution, and concentrate on creating benchmark software for sale to the general public, primarily to serve the general public. If an IHV wanted to see its hardware represented in a forthcoming FM benchmark it could supply whatever help it wanted to FM, sans payment, and FM could derive its income from its non-IHV partners and sales of its software to the general public. That way FM could eliminate these potentially serious conflicts of interest entirely. If they don't do something like this I can see these conflicts eventually tearing the company apart. Just my opinion, of course.

[Mark]

only Walt could take a one word post and post and reply with an editorial (ok, maybe demalion too) ;)

[StealthHawk]

We are all also studying if such "optimizations" should be app-specific or application-wide (generally, but even "generally" can be an important word).

rev nvidia themselves stated on those slides that these "optimisations" (which will forever be in quotations) must accelerate more than just a specific application. I really don't understand how FM can allow such an app specific hack when nvidia promise otherwise.....

The slides say no such thing. One of the guidelines is that an optimization must accelerate more than a benchmark. I can only assume that this applies more to games with a benchmark mode, rather than saying that no optimizations are allowed in synthetic benchmark programs period.

45.23 was supposed to be the first driver to implement these guidelines...I for one have not heard of any changes.

[Malo]

The slides say no such thing. One of the guidelines is that an optimization must accelerate more than a benchmark. I can only assume that this applies more to games with a benchmark mode, rather than saying that no optimizations are allowed in synthetic benchmark programs period.

I see what you mean yes. But it's still a little ambiguous.

If Nv replaced a shader in 3DMark03 and used the same shader replacement in a game, does that constitute as a valid optimisation?

[WaltC]

One of the guidelines is that an optimization must accelerate more than a benchmark.

The way I interpret that is to mean that they'll no longer be making performance-increase claims for driver sets which do nothing except to "accelerate" benchmarks. Or, looking at it another way, they are saying they won't be optimizing strictly for benchmarks anymore, such as 3dMk, for instance.

Obviously this latest Detonator set has flunked the new guidelines test, so I await with relish the moment when nVidia actually does what it says it is going to do...;)

[digitalwanderer]

Obviously this latest Detonator set has flunked the new guidelines test, so I await with relish the moment when nVidia actually does what it says it is going to do...;)
I'm waiting with a healthy dose of skepticism & suspicion.... :?

[Quitch]

Further discussions have shown, that under some circumstances (that we're trying to define at the moment) shader replacements can be accepted.

I hear the funeral march playing in the distance...

[tEd]

[quote]Further discussions have shown, that under some circumstances (that we're trying to define at the moment) shader replacements can be accepted.

who said that?

[FUDie]

Further discussions have shown, that under some circumstances (that we're trying to define at the moment) shader replacements can be accepted.

who said that?
Patric Ojala in this (http://www.beyond3d.com/forum/viewtopic.php?p=153926#153926) post.

-FUDie

[WaltC]

Further discussions have shown, that under some circumstances (that we're trying to define at the moment) shader replacements can be accepted.

who said that?
Patric Ojala in this (http://www.beyond3d.com/forum/viewtopic.php?p=153926#153926) post.

-FUDie

Heh...;) Further discussion:

nVidia: We want to use our own shader code.

FM: Hmmmm....OK, uh, how to say this delicately...? Ummm....we haven't received your sign-up fee as of today....it's 3 p.m., our time....

(Two hours later)

nVidia: OK, did you get the PayPal receipt?

FM: Yes. In the bank.

nVidia: So?

FM: After lengthy discussions we've determined that yes, under some circumstances, it will be OK for you to use your own shader code--yes, certainly.

nVidia: For Pete's sake, what circumstances?

FM: Well....ummm....before we divulge that, there's the little matter of a slight...shall we say...shortfall?...in your initial deposit of trust with us...

nVidia: I don't believe this...

FM: Yes, well...Believe it, believe me...

nVidia: *sigh* Same PayPal account...?

FM: Well, you see we have a delicate matter of taxation in our country to consider and the additional....uh, contribution...cannot go through PayPal. I want you to call Pytor--his first name will be sufficient--at 555-1212, and you are to say, "The rose is red today," and he will answer, "My mother lives in Cambodia," in reply. If the proper phrase is not returned immediately *hang up the phone and get out of there.* Otherwise, Pytor will call later with instructions on what you must do. I would advise waiting by your telephone until Pytor rings you up...Once "the package" has been received, then you will receive clear instructions from me on what to do with your shader code....And, trust me, if Pytor is thwarted in any way I will only tell you where you can stuff your shader code, instead. I hope we understand each other...

nVidia: We get the picture, all right!

FM: I'm gratified. Nice doing uh, the business with you!

[micron]

Thats quite a picture you've painted there Walt....

[Quitch]

ROFLOL!

[just me]

Patric,

I just read your post in your personal forum here at B3d, and I've read your comments in this thread as well. The following was among the most troubling of your remarks, and the most disappointing to me personally:

...Immediately as Nvidia left our development program, getting them back into the program was raised to a very high priority. Our highest priority naturally is and remains the development of high quality and impartial benchmarks.

I'm trying to get you readers to understand what a great thing it is for us that Nvidia re-joined our development program. I think this is the best thing our company has achieved since we launched 3DMark03. The Futuremark employees went through some tough times back there, but we're all happy to get Nvidia back, including AJ (even though some weird rumours tell differently) .Immediately as Nvidia left our development program, getting them back into the program was raised to a very high priority. Our highest priority naturally is and remains the development of high quality and impartial benchmarks.

I have to tell you I find this statement simply baffling. nVidia belonged to the 3dMk03 development program for at least 15 months out of the 18-month 3dMK03 development cycle (as they didn't quit the program until December of '02.) What was it, precisely, that you could obtain from nVidia by "getting them back" into the program that would allow you to make 3dMK03 a "high quality and impartial" benchmark that you failed to obtain from nVidia during the long months in which it belonged to the FM program during the 3dMK03 development cycle? I simply cannot imagine what that might be. Please elaborate.

nVidia was in the 3dMK03 development cycle long enough to learn how to cheat the benchmark with dispatch when it shipped--it boggles the mind how you might think that getting nVidia back had anything at all to do with making 3dMK03 a "high quality and impartial" benchmark, since presumably that's what it was when it shipped.

How is nVidia being in or out of the program relevant to 3dMk03 being a high-quality benchmark as you define it, since nVidia was a full partner for virtually the entire development cycle of 3dMK03?

Hopefully, you are not declaring that 3dMK03 when it shipped was anything but a high-quality and impartial benchmark...? This is baffling for me, honestly.


I'm trying to get you readers to understand what a great thing it is for us that Nvidia re-joined our development program. I think this is the best thing our company has achieved since we launched 3DMark03. The Futuremark employees went through some tough times back there, but we're all happy to get Nvidia back, including AJ (even though some weird rumours tell differently) .

It should be obvious why Nvidia is so important for our development program, but I'll repeat some of the key reasons for those unfamiliar with our company's business.

In order to produce top quality forward looking benchmarks, we need the input and cooperation of all the major players of the industry.


So, how is it you did not receive "input and cooperation" from nVidia for 3dMK03 development when nVidia was paying you for the privilege of providing such throughout the 3dMK03 development cycle? I'm sure I don't have to remind you that the events which occurred prior to nVidia leaving your program, prior to 3dMk03 shipping, prior to nVidia cheating the benchmark, and prior to your audit report detailing exactly how they cheated it--prior to these things nVidia was just as much a part of the FM program as it is today.

Your supposition seems to be that you require nVidia's participation to be able to write a good benchmark, and yet you had nVidia's participation when you wrote 3dMK03 and before *nVidia* made the elective decision to pull out and discredit your company's software.

So if nVidia was unable to assist you in writing a high-quality, impartial benchmark prior to resigning from the program, what has changed so that now nVidia's participation is *required* to produce that result?

How could we otherwise get the highly confidential information of where the industry is going next? Each 3DMark version is aimed at the new if not next generation of hardware, and it is of crushial importance to have comprehensive information of that next generation, in order to make a benchmark for it in advance. In addition to making just high quality benchmarks, they must be impartial, and it is not convincing to release an impartial benchmark developed in cooperation with just one of the major IHVs. If we would not have got Nvidia back, we would have still done our best to optimize it also with Nvidia's upcoming products in mind, but it would have been way harder.

Again, this does nothing to explain the puzzling issue you've raised, which is why you failed to obtain that "comprehensive information of that next generation, in order to make a benchmark for it in advance" from nVidia during the development of the 3dMK03 software (since nVidia belonged to the program for the bulk of the development cycle.) Of course, the truth is you did obtain it, but that nVidia wasn't happy with what you did with the "comprehensive information" and so it quit your program. That's my take on it based on what you've said.

Of course you've glossed over the very odd structure of your business model--which is that companies pay you for the privilege of providing their own next-gen information to you so that ostensibly that information will be used in the construction of a benchmark which will portray their products in a positive light and help them sell those products. Hopefully, you can see the glaring conflict of interest in doing that and in providing a "high-quality, impartial" benchmark for the 3d-card buying public. To that end I can see nVidia's point in quitting--their view was that they were paying you for specific services which you were no longer providing them. It's the conflict of interest inherent in your business model which has caused all of this to happen, IMO.

Sadly, I must regrettably conclude that nVidia's participation is a requirement for your company because of nVidia's financial contribution to your company based on the membership fees it pays you. Obviously, I guess you've agreed to start giving nVidia its money's worth and nVidia is now back in. Seriously, unless you guys start doing some serious "rethinking" about your business model and understand the inherent conflict of interest within it I don't see much of a future for you in this endeavor, and I see more of this same sort of trouble for you on the horizon.

Ditto, ditto, ditto Walt! 8) Nice to see I'm not alone in 'seeing it' the same way. FM can't have it both ways > either it shipped under false pretenses or it is a valid bench & doesn't need changing. Seems FM is admitting it shipped under false marketing pretenses & is not what they represented it as originally. :wink:

I'm also aghast at the claim of 'we didn't know'. My gawd they're in the business to know & if they don't > then they shouldn't release a product for sale w/the claims they do. :!:

.02,

[Hellbinder]

You know the problem here? The problem is that Nvidia is going to get various developers to write specific code to support their short commings. Id like to know when having the cash and influence to cover up shortcommings suddenly Transformed into what we all want to accept as the Golden Standard?? Also being included in so called *industry standard benchmarks*??

This is just Ridiculous. Because the *kinds* of optimizations that ATi made, and would make are not the same endless ream of *CHEATING* BS that Nvidia is going to pull. FX12 is *NOT* part of the DX9 spec. Which is what they are going to do. Among other Questionable practices. All,, I repeat *ALL* in an effort to give them the best 3dmark03 score. So they can have the bragging rights and help line their cofferes. Does it matter that they get it Honestly?? Hell no. Becuase now everyone is Cow-Towing to them Handing them the *Undeserved* Crown on a Golden Platter. In a few months when the PCmags of the World crown Nvidia as the Performance Champion, and the "Card to own" based on 3dmark03 alone (becuase Nvidia endorses it now).. Not one comment will be uttered about *How* they suddenly came out so far on top.

Honestly, I hope that ATi optomizes the **** out of 3dmark03 now. I mean to Ridiculous levels. And overrides any user changes that can be made. Or, Lets Put in a Driver Setting called *Runny soup* that can be used for *all games* that way it passes as a legit *optimization*. I say, Futuremark wants to play the Nvidia Game Fine. Then lets Turn ATi's Rendering of 3dmark03 into a bowl of Runny Soup ;)

(I am a little Irritated right now.. so please bear that in mind)

[Mariner]

Cow-Towing

I know what you mean here (spelt kowtow, by the way), but the way you have written it brings a peculiar image to mind! :D

[Reverend]

(I am a little Irritated right now.. so please bear that in mind)
99% of your posts show that you're in a irritated state. The reason why you're irritated is obvious but I think your health will improve if you calm down a bit when participating in a public forum such this site's (which btw should also help improve this site's forum quality).

[AzBat]

ouch.

Actually, in thinking a bit more about the conflict of interest inherent in the business model, it strkes me there are actually two major conflicts of interest there.

The first is that fact that while the assumption of the IHV partners is that they are paying for software which can be used as a marketing tool to portray their products in a positive light, the public is thinking they are using software which provides a level playing field for an ojective evaluation of the features supported by differing 3d hardware in a comparative context.

The second is that you have competing IHVs paying dues, but expecting entirely different things from the software. For instance, nVidia pays x dollars and tells FM to "go this way" with its software, and ATi pays FM x dollars and tells them to "go that way" with its software. So what does FM do? If they go "this way" they risk alienating ATi and having ATi quit the program, but if they go "that way" with it they risk alienating nVidia and having nVidia quit the program (which has already happened.) Last, if they take the middle route and split the difference between them then they risk alienating both of the IHVs and possibly having both of them quit the program.

Pre-R300 I'm guessing that conflict of interest between IHVs was not that much of a problem since nVidia had a pretty good lead at the high end. But now that ATi is emerging as dominant, I can see several bones of contention developing along the lines that prompted nVidia to quit last year. As the competition intensifies between the IHVs my thought is that the situation isn't likely to subside, but rather to grow more acrid.

The funny thing about companies is when they shell out money for programs like this they naturally expect some tangible benefit in the way of goods and services in return. If they don't perceive they are getting such benefits they lose the motivation to pay for the privilege of participating. Looks like a thorny problem for FM going forward.

I think the best solution therefore would be for them to change their business model and deal with the IHVs on a voluntary basis through NDAs and drop them from the partnership dues-paying rolls. They could maintain their partnership with oems like Dell (and other entities) for the purposes of sales and distribution, and concentrate on creating benchmark software for sale to the general public, primarily to serve the general public. If an IHV wanted to see its hardware represented in a forthcoming FM benchmark it could supply whatever help it wanted to FM, sans payment, and FM could derive its income from its non-IHV partners and sales of its software to the general public. That way FM could eliminate these potentially serious conflicts of interest entirely. If they don't do something like this I can see these conflicts eventually tearing the company apart. Just my opinion, of course.

Great posts Walt. I would have to agree with everything you said. In fact, if I were still writing benchmark articles for JPA I would have written the same thing. In fact, why don't you forward your comments to my former boss which still works for Jon at Jon Peddie Research? Her email address is kathleen@jonpeddie.com. Tell her I sent you and maybe she'll include them in next week's TechWatch Report.

Anyway, FM really needs a reality check. During the time I was doing beta testing for 3DMark since before the creation of Futuremark, there was no cost to IHVs to be part of the program that I was ever aware of. I understand that they need capital to pay for development costs, but they should seriously reconsider their business model. I remember the day that Nathan Harley came to San Francisco to meet with me, my boss and Jon Peddie about their plans for Futuremark. I was excited and genuinely liked the company's new ideas. I don't know when they started making IHVs pay to be part of the program, but I would have to say that probably started their demise. It's pretty bad when you fire your whole Toronto(marketing) office which included a founder, but to also have another founder leave to work on MP2 and yet another to leave after this fiasco. I want everybody to know that I was once approached by Nathan and AJ to work on the 3DMark program. Things didn't work out. I guess it was bad timing, but I'm kinda glad it didn't work out because I don't think I could have stayed with the company knowing what's happened recently. I'm even ashamed that my name is still listed in the 3DMark credits. In my eyes, FM has only 2 things they can do to regain my trust...

1) Quit accepting money from IHVs to be part of the 3DMark BETA program. Make your money from end-users, OEM bundles, selling of benchmark data, support services and custom demo services. If that's not enough, then think of other ways to make money, but do not continue the conflict of interest by requesting thousands of dollars from IHVs.

2) Stand-by your original stance posted in the white-paper. This means actively searching for cheats and optimizations. If any are found then they need to be rejected publicly and results removed from the database. In other words, grow a spine and don't let the IHVs push you around. The point is that all hardware should run the exact same code. If you want to provide additional tests that are not included the score, but also allow shader replacements, then that's fine, but DO NOT ALLOW application specific be used solely for increasing scores or performance.

Tommy McClain

[Pete]

I don't see the conflict in your first point, Walt. The public could care less about the black box that is game code and hardware features and drivers--they just want a good-looking game at playable framerates. It's the developer's job to deliver that, multiple hardware standards or not. It's an uneven "playing field" to begin with, if you consider differing feature sets or hardware optimizations to be uneven. I consider it a reality that must be dealt with. FM can surely devise benchmarks that reflect well on both ATi's and nV's current hardware, as game developers are forced to do.

I agree with your second point. It seems antithetical to 3DMark's goal of an impartial benchmark to allow IHVs pay to be Beta members. It's almost like blackmail--FM starts charging after they find themselves the only game in town in terms of one-click 3D evaluation,* and IHVs are forced to pay a relatively small fee in relation to their overall profits (consider it part of their marketing budget) to stay competitive in the eyes of the always-important VARs (like Dell). From this perspective, I can agree with nVidia's dislike of 3DM03, but I still can't excuse the actions they took to achieve their goal of discrediting 3DM.

In the end, 3DM seems to still be a potentially useful tool if you know how to use and interpret it--much like any other benchmark, and moreso given the ever-widening field of IHV-forced "optimizations." Just as you can't extrapolate Quake 3 performance to Splinter Cell or Tiger Woods or even another FPS based on the Q3 engine, you shouldn't expect a catch-all benchmark written by a single group of developers. If reviewers and companies want to present their less-diligent readers/consumers with an easy-to-digest number, they're better off with an amalgamated benchmark like ExtremeTech's GameGauge that takes into account performance of a variety of game categories coded by a variety of developers. They're still bound by developers including a repeatable, simple benchmark mode, though, and that's one area where FM excels.


* Quick, someone patent this before Amazon does! ;)

[Solomon]

Patric,

I just read your post in your personal forum here at B3d, and I've read your comments in this thread as well. The following was among the most troubling of your remarks, and the most disappointing to me personally:

...Immediately as Nvidia left our development program, getting them back into the program was raised to a very high priority. Our highest priority naturally is and remains the development of high quality and impartial benchmarks.

I'm trying to get you readers to understand what a great thing it is for us that Nvidia re-joined our development program. I think this is the best thing our company has achieved since we launched 3DMark03. The Futuremark employees went through some tough times back there, but we're all happy to get Nvidia back, including AJ (even though some weird rumours tell differently) .Immediately as Nvidia left our development program, getting them back into the program was raised to a very high priority. Our highest priority naturally is and remains the development of high quality and impartial benchmarks.

I have to tell you I find this statement simply baffling. nVidia belonged to the 3dMk03 development program for at least 15 months out of the 18-month 3dMK03 development cycle (as they didn't quit the program until December of '02.) What was it, precisely, that you could obtain from nVidia by "getting them back" into the program that would allow you to make 3dMK03 a "high quality and impartial" benchmark that you failed to obtain from nVidia during the long months in which it belonged to the FM program during the 3dMK03 development cycle? I simply cannot imagine what that might be. Please elaborate.

nVidia was in the 3dMK03 development cycle long enough to learn how to cheat the benchmark with dispatch when it shipped--it boggles the mind how you might think that getting nVidia back had anything at all to do with making 3dMK03 a "high quality and impartial" benchmark, since presumably that's what it was when it shipped.

How is nVidia being in or out of the program relevant to 3dMk03 being a high-quality benchmark as you define it, since nVidia was a full partner for virtually the entire development cycle of 3dMK03?

Hopefully, you are not declaring that 3dMK03 when it shipped was anything but a high-quality and impartial benchmark...? This is baffling for me, honestly.


I'm trying to get you readers to understand what a great thing it is for us that Nvidia re-joined our development program. I think this is the best thing our company has achieved since we launched 3DMark03. The Futuremark employees went through some tough times back there, but we're all happy to get Nvidia back, including AJ (even though some weird rumours tell differently) .

It should be obvious why Nvidia is so important for our development program, but I'll repeat some of the key reasons for those unfamiliar with our company's business.

In order to produce top quality forward looking benchmarks, we need the input and cooperation of all the major players of the industry.


So, how is it you did not receive "input and cooperation" from nVidia for 3dMK03 development when nVidia was paying you for the privilege of providing such throughout the 3dMK03 development cycle? I'm sure I don't have to remind you that the events which occurred prior to nVidia leaving your program, prior to 3dMk03 shipping, prior to nVidia cheating the benchmark, and prior to your audit report detailing exactly how they cheated it--prior to these things nVidia was just as much a part of the FM program as it is today.

Your supposition seems to be that you require nVidia's participation to be able to write a good benchmark, and yet you had nVidia's participation when you wrote 3dMK03 and before *nVidia* made the elective decision to pull out and discredit your company's software.

So if nVidia was unable to assist you in writing a high-quality, impartial benchmark prior to resigning from the program, what has changed so that now nVidia's participation is *required* to produce that result?

How could we otherwise get the highly confidential information of where the industry is going next? Each 3DMark version is aimed at the new if not next generation of hardware, and it is of crushial importance to have comprehensive information of that next generation, in order to make a benchmark for it in advance. In addition to making just high quality benchmarks, they must be impartial, and it is not convincing to release an impartial benchmark developed in cooperation with just one of the major IHVs. If we would not have got Nvidia back, we would have still done our best to optimize it also with Nvidia's upcoming products in mind, but it would have been way harder.

Again, this does nothing to explain the puzzling issue you've raised, which is why you failed to obtain that "comprehensive information of that next generation, in order to make a benchmark for it in advance" from nVidia during the development of the 3dMK03 software (since nVidia belonged to the program for the bulk of the development cycle.) Of course, the truth is you did obtain it, but that nVidia wasn't happy with what you did with the "comprehensive information" and so it quit your program. That's my take on it based on what you've said.

Of course you've glossed over the very odd structure of your business model--which is that companies pay you for the privilege of providing their own next-gen information to you so that ostensibly that information will be used in the construction of a benchmark which will portray their products in a positive light and help them sell those products. Hopefully, you can see the glaring conflict of interest in doing that and in providing a "high-quality, impartial" benchmark for the 3d-card buying public. To that end I can see nVidia's point in quitting--their view was that they were paying you for specific services which you were no longer providing them. It's the conflict of interest inherent in your business model which has caused all of this to happen, IMO.

Sadly, I must regrettably conclude that nVidia's participation is a requirement for your company because of nVidia's financial contribution to your company based on the membership fees it pays you. Obviously, I guess you've agreed to start giving nVidia its money's worth and nVidia is now back in. Seriously, unless you guys start doing some serious "rethinking" about your business model and understand the inherent conflict of interest within it I don't see much of a future for you in this endeavor, and I see more of this same sort of trouble for you on the horizon.

Well Said...

[WaltC]

Great posts Walt. I would have to agree with everything you said. In fact, if I were still writing benchmark articles for JPA I would have written the same thing. In fact, why don't you forward your comments to my former boss which still works for Jon at Jon Peddie Research? Her email address is kathleen@jonpeddie.com. Tell her I sent you and maybe she'll include them in next week's TechWatch Report.

Anyway, FM really needs a reality check. During the time I was doing beta testing for 3DMark since before the creation of Futuremark, there was no cost to IHVs to be part of the program that I was ever aware of. I understand that they need capital to pay for development costs, but they should seriously reconsider their business model. I remember the day that Nathan Harley came to San Francisco to meet with me, my boss and Jon Peddie about their plans for Futuremark. I was excited and genuinely liked the company's new ideas. I don't know when they started making IHVs pay to be part of the program, but I would have to say that probably started their demise. It's pretty bad when you fire your whole Toronto(marketing) office which included a founder, but to also have another founder leave to work on MP2 and yet another to leave after this fiasco. I want everybody to know that I was once approached by Nathan and AJ to work on the 3DMark program. Things didn't work out. I guess it was bad timing, but I'm kinda glad it didn't work out because I don't think I could have stayed with the company knowing what's happened recently. I'm even ashamed that my name is still listed in the 3DMark credits. In my eyes, FM has only 2 things they can do to regain my trust...

1) Quit accepting money from IHVs to be part of the 3DMark BETA program. Make your money from end-users, OEM bundles, selling of benchmark data, support services and custom demo services. If that's not enough, then think of other ways to make money, but do not continue the conflict of interest by requesting thousands of dollars from IHVs.

2) Stand-by your original stance posted in the white-paper. This means actively searching for cheats and optimizations. If any are found then they need to be rejected publicly and results removed from the database. In other words, grow a spine and don't let the IHVs push you around. The point is that all hardware should run the exact same code. If you want to provide additional tests that are not included the score, but also allow shader replacements, then that's fine, but DO NOT ALLOW application specific be used solely for increasing scores or performance.

Tommy McClain

Thank you, Tommy. I found your post fascinating--very interesting, indeed. I'll tell you, since you and I share the same opinions here, but you have some in-depth experience with some of the principals involved which I lack, it might be better for you to forward these sentiments to Kathleen as your being able to flesh out the background adds a worthwhile dimension (or, if you'd rather, feel free to reference me to whatever degree you'd like.) I, of course, agree with your conclusions as well. What puzzles me is that what's so obvious to us apparently isn't to FM. I guess we can only hope they'll do some soul searching about where their present model appears to be taking them.

Well Said...

And thank you, too, D....;)

[WaltC]

I don't see the conflict in your first point, Walt. The public could care less about the black box that is game code and hardware features and drivers--they just want a good-looking game at playable framerates. ....

Pete, what I was talking about there was the public expectation that running 3dMK03 on competing 3d hardware made by different companies will provide them comparative results which tell them something meaningful about those products in relation to each other (which they assume they can infer by looking at the numerical scores the benchmark puts out--FM's "hall of fame," etc.) But since the IHVs contribute financially to FM they each expect that the resulting benchmark software will produce scores flattering to their products, since otherwise the fees they pay to belong to the program would not be justified from their point of view. Therefore the IHVs have one expectation, the public which runs the benchmark another. Thus, the conflict of interest.

[BetrayerX]

@Futuremark guys (worm, Patric, and whoever else).

I paid for 3DMark03 because at the time it provided honest and meaningful results for my videocard reviews. Now, after the deals made with nVidia to allow optimizations for the benchmark, those results are difficult to understand and compare. The fact is 3DMark03 simply can't be trusted to provide those honest results anymore. Therefore, through Futuremarks own actions (and inactions), the 3Dmark03 program is now worthless and my money has been wasted.
What I'm interested in seeing is how the FM people react to such a post, because there are a whole bunch of similar minded folk out there right now who are waiting for an answer/some reassurance and it's another case of the silence being all deafening. :(

Well, 'cept for nVidia who keeps floating rumors... :?

Seriously, does FM plan to react to this or address it in some way soon?



If FM changed their stance about the banchmark. Ain't that a breach of the agreement you supposdely signed (when clicked yes), invalidating it? Thus, can a refund of the money be asked?

[BetrayerX]

Fred da Roza wrote:
How do you fight a company that has no ethics and another that is willing to sell theirs? This is a serious dilemma for ATI. If Futuremark won’t enforce the validity of their benchmark, does ATI have any other choice but to cheat as well? If it destroys the usefulness of Futuremark's benchmark so be it. They made their bed.

Yeah, let the cheats begin.

Ladies and Gentlemen:
In the left corner we have the 800 pound gorilla...the master of all cheats, the discrediter of benchmarks, the optimizer of useles tools....Ladies and gentleman, let's give a big welcome (right) to NVidiaaaaaaaaa!

Who can beat that beast? Quackmark03?
LOL, if ATI comes with something like this (and I hope they do), I dare someone to call them cheaters or something n those lines. I'll go personally to his/her home to kick his/her ass. :lol: :lol: :lol: :lol:

[Mark]

@Futuremark guys (worm, Patric, and whoever else).

I paid for 3DMark03 because at the time it provided honest and meaningful results for my videocard reviews. Now, after the deals made with nVidia to allow optimizations for the benchmark, those results are difficult to understand and compare. The fact is 3DMark03 simply can't be trusted to provide those honest results anymore. Therefore, through Futuremarks own actions (and inactions), the 3Dmark03 program is now worthless and my money has been wasted.
What I'm interested in seeing is how the FM people react to such a post, because there are a whole bunch of similar minded folk out there right now who are waiting for an answer/some reassurance and it's another case of the silence being all deafening. :(

Well, 'cept for nVidia who keeps floating rumors... :?

Seriously, does FM plan to react to this or address it in some way soon?



If FM changed their stance about the banchmark. Ain't that a breach of the agreement you supposdely signed (when clicked yes), invalidating it? Thus, can a refund of the money be asked?
Hah, funny you should mention that. I asked for a refund from FM for exactly that reason, to which they denied. I would post the email exchange I had with them, but I'm not sure they would appreciate it... I'll think about it anyways...

[BetrayerX]

Who cares about what they (FM) appreciate?

Did they cared when they changed politics like a pair of dirty socks? (actually they took out the clean ones then got the dirty ones...you get the point). :wink:

I wonder if someone might sue FM for fraud.....thats what they did, basically....the change of politics is so 180 deg that no one can deny it KILLED 3dMark's credibility, and of course, usefulness.

[BetrayerX]

Worm[FutureMark] wrote:

NVIDIA re-joined our Development Program, which is the same as our Beta Program. The NVIDIA logo will appear on the Beta Program page asap. Refresh the page in an hour or two!

So, NVidia paid only God knows how many hundreds of thousands just to have the logo on their website?

Cut the crap, we have been swiming in it for more than 6 months now.

[BetrayerX]

]I'm sorry, but none of that makes me feel any better about me spending my money on a copy of 3DMark03. Frankly, I feel like I've been cheated out of my money... no not optimized out of it, cheated. Where can I apply for a refund?
I fail to understand why you feel like you have been cheated?
Because you changed the rules of how the benchmark was to be run AFTER they purchased it, and some of us feel that by changing the rules you've invalidated 3dm2k3's usefullness as a tool...as well as all the rampant speculation about upcoming favoritism on FM's part towards nVidia.

Has ATi contacted you people yet today? I'm kind of waiting for that shoe to drop.... :twisted:


I wonder if ATI could potentially sue FM...they could also ask for a refund too (and a biiig one I bet)

[micron]

BetrayerX = :roll:

[AzBat]

Thank you, Tommy. I found your post fascinating--very interesting, indeed. I'll tell you, since you and I share the same opinions here, but you have some in-depth experience with some of the principals involved which I lack, it might be better for you to forward these sentiments to Kathleen as your being able to flesh out the background adds a worthwhile dimension (or, if you'd rather, feel free to reference me to whatever degree you'd like.) I, of course, agree with your conclusions as well. What puzzles me is that what's so obvious to us apparently isn't to FM. I guess we can only hope they'll do some soul searching about where their present model appears to be taking them.

Walt,

You're welcome. I went ahead and forwarded your comments to Kathleen. I gave you full credit, but if I had your email address I could have sent that along too. Anyway, I'll let you know what I hear.

As for FM, I did receive a PM from one of my contacts at FM. Evidently he wasn't too pleased with my comments, but like I told him it's a good thing I'm no longer in the business of writing benchmark articles for JPA. ;) Personally I would had hoped that they would understand where I and others were coming from, but I believe the problem they're having is mainly staying alive through all this mess. If I was in charge I think I would rather die fighting and keep my honor and integrity instead of worrying about losing my job. Jobs come and go, but it's hard to get back honor and integrity after you've lost it. If anybody knows about this, it's me as I was "laid off" by JPA. I'm not proud of how I left the company, but it's a good thing I had long and great relationships with them or it could have been a lot worse. Anyway, hopefully some of these comments will sink in their head.

Tommy McClain

[WaltC]

...As for FM, I did receive a PM from one of my contacts at FM. Evidently he wasn't too pleased with my comments, but like I told him it's a good thing I'm no longer in the business of writing benchmark articles for JPA. ;) Personally I would had hoped that they would understand where I and others were coming from, but I believe the problem they're having is mainly staying alive through all this mess. If I was in charge I think I would rather die fighting and keep my honor and integrity instead of worrying about losing my job....

I would hope they'd realize that the integrity of their software is at stake and has been put at risk. If the general perception of FM benchmarks becomes one of skepticism and mistrust, it won't take the IHVs long to figure out that they are not getting their money's worth any longer and they'll drop off anyway. The sum total is that FM's position will actually be much worse at that point than it was when nVidia quit. But that's the long view, and many companies today don't believe they can afford to take a long view about much of anything. I agree--if such interest conflicts are obvious to outsiders, why can't FM see them? Perhaps they just don't want to...

[BetrayerX]

BetrayerX = :roll:

micron = :?:


What's that supposed to mean dude?

What would you do when a product doesn't do what it was promised to do?
Return it and ask for a refund. That's exactly what FM has done and I believe people and companies as clients and partners that have invested some hard earned money must have a say in the matter. This move is not only a simple Hypocritical decision from FM but also one where no one else was asked for even an opinion. If you as a costumer might feel ripped off by it, what can you expect from companies that endorse some huge amounts of money in FM.

[micron]

BetrayerX = :roll:

micron = :?:


What's that supposed to mean dude?


You figure it out dude......