Ops,"I" did it again. Why does Futuremark still ke

[banksie]

What interests me more from the extreme tech article is this :-

"Nvidia, meanwhile, has issued statements claiming that the original FutureMark optimizations were the unsupervised work of a programmer, and the company has set down clear public guidelines over the use of such optimizations."

Where have they claimed that? This is the first I have heard of it.

Philip

 

[just me]

So Tero is setting 'optimization' policy & working w/NV. Just as I thought! Of course he would since everyone else @ FM is in Europe & Tero is a few miles away from NV in California.

Odd that ET said 'BETA Program' when NV isn't listed as a member of the "BETA Program" by FM & FM says they joined the "Benchmark Development Program". Semantics? No. Legalese!

Odd that FM hasn't posted here on this. Patric has his own forum even. I did see someone associated w/FM here yeaterday (not worm either) but he didn't post anything.

.02,

 

[digitalwanderer]

What interests me more from the extreme tech article is this :-

"Nvidia, meanwhile, has issued statements claiming that the original FutureMark optimizations were the unsupervised work of a programmer, and the company has set down clear public guidelines over the use of such optimizations."

Where have they claimed that? This is the first I have heard of it.

Philip

I believe ET is referencing those three power-point slides that Kyle put up on hard and how Kyle said nVidia said/implied those things. (I don't remember exactly)

 

[Dave Baumann]

So does the staff at B3D expect to be consulted about these new optimization "guidelines" FutureMark is drafting for our collective benefit?

I've already talked to Tero about the general gist of it, although I'm not sure of how much I should talk about now. Given the situation with NVIDIA ts probably about all they can do (its further muddied by the fact one of the major OEM's allow something as well).

 

[Slides]

(its further muddied by the fact one of the major OEM's allow something as well).

Dave, could you clarify this statement please? What do you mean by "allow something"?

 

[Nick[FM]]

Just to make sure that you (actually all of you) don't get the wrong picture, I'll comment on these "misunderstandings":

So Tero is setting 'optimization' policy & working w/NV. Just as I thought! Of course he would since everyone else @ FM is in Europe & Tero is a few miles away from NV in California.

All/any policies are set in cooperation with all our Beta Members.

Odd that ET said 'BETA Program' when NV isn't listed as a member of the "BETA Program" by FM & FM says they joined the "Benchmark Development Program". Semantics? No. Legalese!

NVIDIA re-joined our Development Program, which is the same as our Beta Program. The NVIDIA logo will appear on the Beta Program page asap. Refresh the page in an hour or two!

Odd that FM hasn't posted here on this. Patric has his own forum even. I did see someone associated w/FM here yeaterday (not worm either) but he didn't post anything.

I was here yesterday, and now I have posted about this.

 

[Mark]

]Just to make sure that you (actually all of you) don't get the wrong picture, I'll comment on these "misunderstandings":

So Tero is setting 'optimization' policy & working w/NV. Just as I thought! Of course he would since everyone else @ FM is in Europe & Tero is a few miles away from NV in California.

All/any policies are set in cooperation with all our Beta Members.

Odd that ET said 'BETA Program' when NV isn't listed as a member of the "BETA Program" by FM & FM says they joined the "Benchmark Development Program". Semantics? No. Legalese!

NVIDIA re-joined our Development Program, which is the same as our Beta Program. The NVIDIA logo will appear on the Beta Program page asap. Refresh the page in an hour or two!

Odd that FM hasn't posted here on this. Patric has his own forum even. I did see someone associated w/FM here yeaterday (not worm either) but he didn't post anything.

I was here yesterday, and now I have posted about this.

I'm sorry, but none of that makes me feel any better about me spending my money on a copy of 3DMark03. Frankly, I feel like I've been cheated out of my money... no not optimized out of it, cheated. Where can I apply for a refund?

 

[Nick[FM]]

I'm sorry, but none of that makes me feel any better about me spending my money on a copy of 3DMark03. Frankly, I feel like I've been cheated out of my money... no not optimized out of it, cheated. Where can I apply for a refund?

I fail to understand why you feel like you have been cheated?

 

[Quitch]

I fail to understand why you feel like you have been cheated?

If you mean that, then I feel very scared.

Oh - so Patric says "don't believe everything you read, and by the way IT'S ALL TRUE!!"

LOL! That's what I was thinking

I think Patric is just concerned that the posted opinions of folks (including Dave, and especially FM's beta members) in public forums may be read as representative of the opinions of personnels at FM. I'm sure many of you trust most of what Dave has to say/post but even Dave will tell you he doesn't know all the details about what he says/posts.

I suspect thay're worried that what Dave says IS representative of how many FM employee's feel.

 

[K.I.L.E.R]

]

I'm sorry, but none of that makes me feel any better about me spending my money on a copy of 3DMark03. Frankly, I feel like I've been cheated out of my money... no not optimized out of it, cheated. Where can I apply for a refund?

I fail to understand why you feel like you have been cheated?

I was going to make an optimisation joke but I believe that's been done to death.
I don't feel cheated spending the money.

Can I have a cookie?

 

[digitalwanderer]

]

I'm sorry, but none of that makes me feel any better about me spending my money on a copy of 3DMark03. Frankly, I feel like I've been cheated out of my money... no not optimized out of it, cheated. Where can I apply for a refund?

I fail to understand why you feel like you have been cheated?

Because you changed the rules of how the benchmark was to be run AFTER they purchased it, and some of us feel that by changing the rules you've invalidated 3dm2k3's usefullness as a tool...as well as all the rampant speculation about upcoming favoritism on FM's part towards nVidia.

Has ATi contacted you people yet today? I'm kind of waiting for that shoe to drop....

 

[just me]

Thanks worm. 8)

So what is this "Benchmark Development Program" Tero spoke of & is quoted in FM's press release for the world to see? Is there such a thing or is Tero making up names as he goes along? :?

Lemme see if I got this part right: 'optimizations' are going to be determined 'en masse' by all the members of the 'BETA Program'? Does that mean 100% must agree or just a 50.0001% majority? Once the BETA Members agree, then Tero will announce the 'new & improved' way it is meant to be benched?

Thanks for stopping by to answer ?'s. Nice to see someone from FM is still posting here. 8)

 

[Doomtrooper]

] I fail to understand why you feel like you have been cheated?

 

[WaltC]

]Just to make sure that you (actually all of you) don't get the wrong picture, I'll comment on these "misunderstandings":

All/any policies are set in cooperation with all our Beta Members.

Does this mean they all have to agree on something before it's implemented by unanimous assent, or that you deal with your IHV partners individually?

Also, are there any policies/standards which FM sets itself independently of your partners which they must adhere to in order to maintain their participation in your program (aside from paying their dues)? IE, do you set the standards or do they?

NVIDIA re-joined our Development Program, which is the same as our Beta Program. The NVIDIA logo will appear on the Beta Program page asap. Refresh the page in an hour or two!

What brought this about--has nVidia changed its demands, or have you simply agreed to theirs? Something has changed nVidia's mind about your company--explaining what that was in your opinion would go a long way towards diffusing misunderstandings.

 

[digitalwanderer]

]NVIDIA re-joined our Development Program, which is the same as our Beta Program. The NVIDIA logo will appear on the Beta Program page asap. Refresh the page in an hour or two!

What brought this about--has nVidia changed its demands, or have you simply agreed to theirs? Something has changed nVidia's mind about your company--explaining what that was in your opinion would go a long way towards diffusing misunderstandings.

Or go a long way towards providing new ones. (I am just NOT feeling very optimistic about this...sorry. )

 

[WaltC]

Or go a long way towards providing new ones. (I am just NOT feeling very optimistic about this...sorry. )

Well, right now I'm fairly ambivalent about the situation. I would at least (speaking for myself) like to provide FM some time to explain this turn of events fully and completely. Hopefully, they won't be coy about it and pretend there's nothing to explain to people and will recognize the value of full disclosure in this case. I'm genuinely puzzled though by the fact that it's not clear the company understands that its whole program rests on the bying public believing their benchmarks have something credible to say about 3d technology. I mean, if the general public loses confidence in their software, surely they know that their partners will follow suit...? I guess we'll see...

I'm hoping they are not so naive as to believe they can turn back the clock to how things were before...not a chance, IMO...The genie's out and there's no getting him back in the bottle.

 

[Patric Ojala]

Wörm told me there is an interesting discussion going on here, since I'm a bit lazy to follow all forums personally

As usual, I'll comment in the nice personal forum Dave and the guys gave me (thanks again). Check it out in a few minutes, I'll start typing right away.

 

[K.I.L.E.R]

I'm waiting Mr Ojala.
BTW: Where's my cookie?

 

[madshi]

As usual, I'll comment in the nice personal forum Dave and the guys gave me (thanks again). Check it out in a few minutes, I'll start typing right away.

I've no problem with NVidia rejoining FutureMark's beta program. I agree, that this is the best solution to get an impartial 3DMark04/05/06.

The main problem I currently have (and I think most of the other guys here think similarly) is about the driver optimizations. First FutureMark said, 3DMark03 specific driver optimizations like shader replacements would not be allowed at all. ATI even took their (mathematical correct) replacement shader(s) out of their drivers. But now after that joint NVidia/FutureMark statement NVidia is putting out one driver after the other which all seem to have most optimizations including replacement shaders back in. And FutureMark sits around and seems to accept that. Then have a look here: http://www.theinquirer.net/?article=11037 . Now where does that leave ATI? NVidia currently has the top performing card in 3DMark03 and everyone knows that this is bullshit, since ATI's cards have a much better shader performance. Could you please add some comments about these problems? Because these are the *real* problems IMHO. Thank you!

 

[Patric Ojala]

The main problem I currently have (and I think most of the other guys here think similarly) is about the driver optimizations. First FutureMark said, 3DMark03 specific driver optimizations like shader replacements would not be allowed at all. ATI even took their (mathematical correct) replacement shader(s) out of their drivers. But now after that joint NVidia/FutureMark statement NVidia is putting out one driver after the other which all seem to have most optimizations including replacement shaders back in. And FutureMark sits around and seems to accept that. Then have a look here: http://www.theinquirer.net/?article=11037 . Now where does that leave ATI? NVidia currently has the top performing card in 3DMark03 and everyone knows that this is bullshit, since ATI's cards have a much better shader performance. Could you please add some comments about these problems? Because these are the *real* problems IMHO. Thank you!

Fair enough, I think I understand you concern.

First of all, we need to be very careful about what public statements we make. This industry is full of eager writers, who seem to spend their days trying to read between the lines and dig out some issues from anything and everything that is published. If we don't immediately comment some new turn in the industry, it might be due to the fact that:
a) We don't know yet the whole extent of the issue and we're still trying to get to the bottom of it. This is usually what's happening when we don't react to something the very next second.
b) We see that the issue in question as not ours to comment. Our goal is to provide the industry with tools for hardware and driver testing. Analyzing the results and judging what's good or bad is what the professional hardware press is for.
c) The whole issue was raised due to some wild rumour, and we don't see any need to comment the issue at all.

The whole shader optimization issue could have gone much smoother if (well, if there wouldn't have been any tension between us and Nvidia, but that's beside the point) the gfx IHVs would have first discussed the issue with us and then implemented the shader replacements. We were asked to do some scrambling of the shader instructions, in order to find out who does direct shader identification and replacement + this was at the time considered an all bad thing. Further discussions have shown, that under some circumstances (that we're trying to define at the moment) shader replacements can be accepted. If these discussions would have been held first, and shader replacement would have been implemented in drivers only afterwards, I doubt there would have been problems like the ones we ran into. Also, the whole shader replacement thing was a brand new issue. Nobody knew at the time how to react to it. This is a fine example of why it is important that we don't impulsively comment all issues immediately, rather first consider them carefully. And this is why our response might take some time, but it does not always mean that we don't give a d---. It does mean that sometime though