Futuremark Announcement Delayed
- Thread starter Nick[FM]
- Start date
- Status
Erhm... I must say that I (personally) think that several people (not only here) only see what they want to see, and are ignoring what's really there. As we have said 1000 times before, do not believe any rumors, or some users made up conspiracy theories. Don't read "between the lines" and make up something that doesn't exist. It will get you nowhere...
Some forget that we are really doing our best to "bring balance to the force".
In other words, try to make it so that the users, media and IHV's are all satisfied with our rules. You think it is easy? Certainly not!
About the "Digitalwhat" thing; I was joking. That " " was meant to point it out. Sorry if someone took it as a personal attack or anything. Wasn't my intention.
Some forget that we are really doing our best to "bring balance to the force".
In other words, try to make it so that the users, media and IHV's are all satisfied with our rules. You think it is easy? Certainly not!About the "Digitalwhat" thing; I was joking. That "
" was meant to point it out. Sorry if someone took it as a personal attack or anything. Wasn't my intention.Futuremark has a hell of a lot of work to do to regain any trust people once may have had in the company. A new set of guidelines is all good but it's up to Futuremark to show some backbone and actually enforce the rules contained within. Forgive me, but from what I've seen of Futuremark in the past it doesn't look like you guys are up on the vertebrae-stiffness scale very far.worm[Futuremark said:]Erhm... I must say that I (personally) think that several people (not only here) only see what they want to see, and are ignoring what's really there. As we have said 1000 times before, do not believe any rumors, or some users made up conspiracy theories. Don't read "between the lines" and make up something that doesn't exist. It will get you nowhere...
Some forget that we are really doing our best to "bring balance to the force".
Time will tell I guess, but you'll hopefully understand if I don't give you the benefit of the doubt this time around.
I think your problem is you are trying to please everyone and in this case you cannot please everyone cause then you will compromise the integrity of your product and in a benchmark integrity is the largest factor when people look at the end numbers of the score.
To put it bluntly Nvidia will not like it if 3dmark shows nvidia cards the way they actually perform, NV3X cards are slow when it comes to DX9 features and that's just the plain reality of it, your benchmark should reflect that reality but Nvidia doesn't seem to want to let you.
If you cater to Nvidia's desire to have inflated scores that do not accurately refelct how the card performs in real life then you are doing you users a disservice. If you do clamp down and do not allow any cheats then Nvidia will have a fit and act like a spoiled kid.
You really need to focus on the long term, Nvidia is a sinking ship the warning signs are everywhere, do not cater to their needs cater to the needs of the customers that want accurate reliable results from this benchmark. Catering to Nvidia now may make thing happy for you both for a short time but when there ship sinks your now left with the long lasting image of an unreliable benchmark program that compromised it's policy from ONE vendor.
To put it bluntly Nvidia will not like it if 3dmark shows nvidia cards the way they actually perform, NV3X cards are slow when it comes to DX9 features and that's just the plain reality of it, your benchmark should reflect that reality but Nvidia doesn't seem to want to let you.
If you cater to Nvidia's desire to have inflated scores that do not accurately refelct how the card performs in real life then you are doing you users a disservice. If you do clamp down and do not allow any cheats then Nvidia will have a fit and act like a spoiled kid.
You really need to focus on the long term, Nvidia is a sinking ship the warning signs are everywhere, do not cater to their needs cater to the needs of the customers that want accurate reliable results from this benchmark. Catering to Nvidia now may make thing happy for you both for a short time but when there ship sinks your now left with the long lasting image of an unreliable benchmark program that compromised it's policy from ONE vendor.
worm[Futuremark said:
Well, that's very easy to say, when you are in the know.
The best way to squelch rumors is to be open and frank, something Futuremark has not been with regard to the 180° "cheatoptimizations" fiasco. All we got was the PR about "we apologize for saying bad things about NV", then "NV and FM make up and FM proceeds to cash in NV's money (beta program fees)", then the 5900 comes on top of 3DMark2003 (although it is still cheating, as demonstrated by other sources)... All we got was some comments by P. Ojala about why "getting NV back was the top priority for FM", without saying to what length FM was ready to go to achieve this goal. And you wonder why people are doing conspiracy theories ?Some forget that we are really doing our best to "bring balance to the force".
As WaltC said, if that's your goal, then you will never succeed, as there is an absolutely huge conflict of interest between accepting IHV's money (with said IHVs expecting in return that their products are put in a positive light by the benchmark), and accepting medias and end-users money, with those people expecting an objective benchmark. That's not a critic, or a flame : just MHO about why a business model similar to yours cannot succeed in its current form, human nature being what it is. So you could either concentrate on pleasing IHV, thus making the benchmark irrelevant as far as comparing products is concerned, or you could concentrate on pleasing the medias and end-users, which poses the problem of getting stable income sources for your software, especially if some IHVs and their "guys with webpages" proxies are hard at work discrediting your product. Perhaps the solution could be getting money only from Dell/HP (and end-users), as those people can really use some objective performance measuring tools ? But as long as you accept money from any IHV, then your motives will look tainted...
Question: Why was the announcement delayed again? Which particular beta members haven't 'approved' of the announcement?worm[Futuremark said:]Erhm... I must say that I (personally) think that several people (not only here) only see what they want to see, and are ignoring what's really there. As we have said 1000 times before, do not believe any rumors, or some users made up conspiracy theories. Don't read "between the lines" and make up something that doesn't exist. It will get you nowhere...
Some forget that we are really doing our best to "bring balance to the force".
About the "Digitalwhat" thing; I was joking. That "
I don't recall bashing Worm, and I feel my dig at FutureMark is justified after their ping-pong like attitude changes of late.K.I.L.E.R said:Some of you really do need to learn some manners.
I was hoping this wouldn't turn out into a "Bash Worm" thread, thankfully it hasn't turned into one either.
If I were a FM employee would I get bashed too?
I'm sure "Mr DigitalWanderer" would have added me to his shamelessly bash employees of company X list.
The decisions made by FM doesn't mean every FM employee holds that view.
People like "Mr DigitalWanderer" piss me off. If he wants to bash a company over their decisions then by all means go for it but please don't assume that everyone working for the company holds the same view as the CEO/[insert person in charge].
What's even more funny are the people who think they know everything going on behind the curtain.
Such arrogance only makes people look stupid.
Yes, I am defending Worm and I don't see what HE has done wrong. Maybe "Mr DigitalWanderer" can tell me what Worm has done to wrong everyone or him specifically?
This might come as a surprise to you all but I really didn't like FM's(Madonion's) decisions over the last several years.
In no way do I bash ANY FM/MO employee over an entire company's/CEO's decisions.
In fact I wish every FM employee the best of luck and a happy life.
Why should we not be a little unhappy that an announcement that was announced two weeks ago was delayed from coming out on it's announced day because some beta members hadn't finished reading/approving it yet? I really think it's speaks volumes about the level of control/influence the partners have over FM and vice-versa.
Sorry you don't like it K.I.L.E.R., but I'll stand by my words and back them up.
worm[Futuremark said:
Sigh...when will you guys ever understand that it is the absence of verifiable information which causes people to speculate? You say that people are "ignoring what's really there"....Worm, what's really there? Where is it? It is because the information isn't "there" that people speculate.
For instance, your former set of rules clearly designated application optimizations relative to 3dMk03 in an IHV driver as "illegal", did they not? As I recall, this was the premise of your audit report. Why, then, is there a necessity to institute a new set of rules--which nobody in your public has seen yet?
It isn't known what the new rules are, and it isn't known why new rules are necessary. Therefore there is speculation. You guys are the only people who can end the speculation. The only thing people have been able to do up to now is to "read between the lines" because of the lack of information coming out of FM on these issues. Don't blame the public for asking questions about issues you have refused to answer for months.
Some forget that we are really doing our best to "bring balance to the force".
Has it never occurred to you that maybe this is an unrealistic expectation? That maybe you'll wind up pleasing nobody? A wiser man than I once said: "You can please some of the people some of the time, but you can never please all of the people all of the time." You might like to consider that.
The main thing the public expects out of FM is that you guys realize that your benchmark has got to do a lot more than exist as a marketing tool for IHVs. The simplest way to do that is to establish the rule of "No optimizations allowed," enforce it through regular anti-detection patches, and let the chips fall where they may. Continuing to try and straddle the fence between the interests of the public and the interests of the IHVs is very likely to displease both sides here. The loser won't be the public and it won't be the IHVs--it will be FM. That's what the people trying to "read between the lines" are trying to to tell you.
Interview with Patric Ojala at 3D Center, dated Sept 20th:
My thanks to Bert at nVnews for pointing this interview out.
There has been a lot of controversial discussions around 3DMark03, but I would like you to keep one thing in mind. 3DMark03 was developed in cooperation with all the major PC hardware manufacturers, just like 3DMark2001. That enabled us to create a product that was ahead of its time. We understand that many people did not believe the performance figures when 3DMark03 was published back in February. Only now we are beginning to see DX9 games emerge and what do you know, their performance readings seem to correlate well with what 3DMark03 showed you already in February
My thanks to Bert at nVnews for pointing this interview out.
crystalcube
Newcomer
I am not sure how practical it would be. Would the drivers have to be certified both by WHQL & Futuremark ?
I am more keen to see how serious FM is in enforcing their rules. They can publish a guideline every month just to have a new version because previous version could not be enforced after an IHV decided to leave or join FM's beta program.
CorwinB,
Getting back NVIDIA to the program was important to us. We wanted all major players to be in the program in order to have the best knowledge to produce future bechmarks. I thought that would be pretty self-evident?
digitalwanderer,
The announcement was delayed for the reasons I posted. We made some last minute edits & changes, and we wanted all(!) Beta members to have some time to read them thoroughly. Making small edits and changes and simply releasing it without that our Beta members would know about it in advance, would kind of work against the whole idea of our program.
WaltC,
Well, if you think that the absence of verifiable information should and will cause speculation, I can only say that it's sad. The question is, when will people stop speculating, and just wait for official statements and official publications? That would be much better for all of us. And I don't mean this announcement only.
Well, the "wise" quote you posted is true, but is there anything wrong with trying? Of course you can't get everyone to be 100% happy and satisfied with everything you do, but if you don't try, you will never accomplish anything. We think it is utmost important that our users are happy with our products, online and offline media find our tools useful & reliable and that the IHV's see our benchmarks are good and reliable DX benchmarks.
I know that the public (at least here [smile]) is very demanding and want everything out in the open the very second they want and demand. That is simply impossible to do. I think I speak for many other companies as well. Speculating and spreading baseless rumors only makes it harder.
ps. sorry for hasty & short answers, but I am in a big hurry now!
Getting back NVIDIA to the program was important to us. We wanted all major players to be in the program in order to have the best knowledge to produce future bechmarks. I thought that would be pretty self-evident?
digitalwanderer,
The announcement was delayed for the reasons I posted. We made some last minute edits & changes, and we wanted all(!) Beta members to have some time to read them thoroughly. Making small edits and changes and simply releasing it without that our Beta members would know about it in advance, would kind of work against the whole idea of our program.
WaltC,
Well, if you think that the absence of verifiable information should and will cause speculation, I can only say that it's sad. The question is, when will people stop speculating, and just wait for official statements and official publications? That would be much better for all of us. And I don't mean this announcement only.
Well, the "wise" quote you posted is true, but is there anything wrong with trying? Of course you can't get everyone to be 100% happy and satisfied with everything you do, but if you don't try, you will never accomplish anything. We think it is utmost important that our users are happy with our products, online and offline media find our tools useful & reliable and that the IHV's see our benchmarks are good and reliable DX benchmarks.
I know that the public (at least here [smile]) is very demanding and want everything out in the open the very second they want and demand. That is simply impossible to do. I think I speak for many other companies as well. Speculating and spreading baseless rumors only makes it harder.
ps. sorry for hasty & short answers, but I am in a big hurry now!
Actually that's not. Not at all by a long shot. I think that the only thing you could possibly need would be a DirectX 9 SDK, and perhaps the same level of dev-rel a company doing a game could expect. That would quelch the critics that you cater to IHVs whims to rake in some money, because, well, you wouldn't be raking any money from IHVs at all.
Better for who? If we waited for an official statement from nVidia that their hardware is less than they billed it to be we would still be waiting.worm[Futuremark said:
I agree there is a reasonable period that should be allowed for an official response, but right now Futuremark's track record in that department ain't exactly stellar...the last time I remember "waiting on an official response" was when you and nVidia released that press release explaining that cheating was perfectly ok as long as you had enough lawyers to sick on someone.
(<--Frowny face to reflect disapointment, nothing else.)
Why? It indicates to me that you're more concerned with keeping the paying customers happy rather than the enthusiasts.worm[Futuremark said:
Now I can understand that perfectly with you being a business and all, but I really wish you'd quit trying to say that isn't what it is then.
Futuremark should be allowed to make the rules as it sees fit, my whole lack of faith in your product right now is that I think it's being catered a lot more to the IHVs....and needing their approval before putting out a press release just seems a bit overly silly.
Why couldn't you have possibly told them something along the lines of, "We promised everyone we would put out an announcement on the 19th and we absolutely need you to review this before we do. We will be making this announcement at 6pm, if you have any feedback on the changes you must present them before this time."?
I guess I just don't understand how I'm supposed to be inclined to give Futuremark the benefit of the doubt on enforcing their own rules when they can't even enforce a PR deadline.
Gad, talk about shooting the messenger! :? Ya know 'worm' didn't even have to come here & post this as it is posted @ FM. I see this as a courtesy to B3D members & hope some of the replies don't keep it from happening in the future. The easy thing would have been to not post it here & not reply if it was linked from here. I too have had my druthers about some of FM's prior actions/inactions, but 'worm' didn't make those decisions > Tero did. [Keyboard isn't working right :? Sorry for the 1 paragraph post ] I've done my fair share of posting my disappointment in FM & 3DM03, but I can't see what waiting a few days/week is going to make that big of a difference at this stage in the 'game'. Give it a chance & if ya have questions > email Tero as he is the one in charge of this issue & the one FM has referred to answer these questions. Don't shoot 'worm' tho' as he has always been 'up-front' (as far as NDA's allow) with us & only brought the message here ... a message he didn't even have to post here, but did as a courtesy to us. 8) My .02 ... for now.
] I've done my fair share of posting my disappointment in FM & 3DM03, but I can't see what waiting a few days/week is going to make that big of a difference at this stage in the 'game'. Give it a chance & if ya have questions > email Tero as he is the one in charge of this issue & the one FM has referred to answer these questions. Don't shoot 'worm' tho' as he has always been 'up-front' (as far as NDA's allow) with us & only brought the message here ... a message he didn't even have to post here, but did as a courtesy to us. 8) My .02 ... for now.
I apologize for any offense I've caused, but I find this latest delay/snub/whatever to just be the straw that broke the camel's back for me.
I will be quiet.
I will be quiet.
Deathlike2
Regular
"Just me" is correct (it would sound wrong if I didn't put the name in quotes...)
Digi... for reasons beyond my knowledge.. I've really have had little respect for your posts...
You poke fun at NVidia (which is ok at times)... but you go overboard with it like an ATI fanatic (which you may or may not be)... you ARE reading a little bit into things that don't exist...
The majority that are "in the know" of Futuremark's current status are displeased... but worm is right in stating that there weren't any sort of "guidelines written"...
There would be nothing to enforce without them...
Let Futuremark do the talking... and THEN you can digress...
Actions SPEAK louder than words... and that should continue to be the case.. Don't let PR do the talking... let's see what actions HAVE been made... perhaps NVidia and Futuremark will fix themselves.. but don't put them down until they PROVE you wrong BY THEIR ACTIONS... (inaction does NOT mean nothing is being done)
Here's a thought: did MS and the OGL committee decide to hew closer to ATi's specs because they would be easier for other companies to emulate and compete with? Did nVidia produce a design so complex and labor-intensive that essentially only they or ATi could compete? Or did MS just pick ATi's design to spite nV, to balance out nV"s dominance without looking into whether nV had the better spec?
I fully agree that the clip planes, ignored buffer clears, and reduced precision that nV placed in its drivers without the consent of FM were wrong in the context of a benchmark. On the other hand, I'm now unsure whether FM and Valve and Core are being unfair to both nVidia and its large customer base by ignoring the NV3x's pros and cons and blindly hewing to DX9. So I now view DX9 benchmarks with a bit more perspective, and perhaps a bit less ignorant (in my case) invective. Sure, ATi is currently faster at DX9. But is this a case of superior hardware, or everyone else ganging up on big, bad (and profitable and influential) nVidia? Or will updated HLSL or Cg compiler (or drivers) bring regular DX9 code up to acceptable performance levels, thus somewhat justifying people's anger at nV for releasing half a solution (good hardware and bad drivers, akin to ATi in years past)?
Thus, I'm looking forward to FM's statement with increasing interest. I think their approach to 3D benchmarking may change with 3DM04 if nV continues to follow a 3D path parallel yet divergent to ATi. Perhaps separate code paths are justified, provided the developer can be trusted to make every effort to ensure comparable output. Separate code paths should also entail detailing the effort required to achieve equivalent or comparable results, and detailing any IQ differences that arise, no matter how minor.
I fully agree that the clip planes, ignored buffer clears, and reduced precision that nV placed in its drivers without the consent of FM were wrong in the context of a benchmark. On the other hand, I'm now unsure whether FM and Valve and Core are being unfair to both nVidia and its large customer base by ignoring the NV3x's pros and cons and blindly hewing to DX9. So I now view DX9 benchmarks with a bit more perspective, and perhaps a bit less ignorant (in my case) invective. Sure, ATi is currently faster at DX9. But is this a case of superior hardware, or everyone else ganging up on big, bad (and profitable and influential) nVidia? Or will updated HLSL or Cg compiler (or drivers) bring regular DX9 code up to acceptable performance levels, thus somewhat justifying people's anger at nV for releasing half a solution (good hardware and bad drivers, akin to ATi in years past)?
Thus, I'm looking forward to FM's statement with increasing interest. I think their approach to 3D benchmarking may change with 3DM04 if nV continues to follow a 3D path parallel yet divergent to ATi. Perhaps separate code paths are justified, provided the developer can be trusted to make every effort to ensure comparable output. Separate code paths should also entail detailing the effort required to achieve equivalent or comparable results, and detailing any IQ differences that arise, no matter how minor.
- Status
Similar threads
- Locked