Nvidia Against 3D Mark 2003

Silent_One

Newcomer
From VE:
Nvidia has contacted us to say that it doesn't support the use of 3DMark 2003 as a primary benchmark in the evaluation of graphics cards, as the company believes the benchmark doesn't represent how current games are being designed. Specifically, Nvidia contends that the first test is an unrealistically simple scene that's primarily single-textured, that the stencil shadows in the second and third tests are rendered using an inefficient method that's extremely bottlenecked at the vertex engine, and that many of the pixel shaders use specific elements of DX8 that are promoted by ATI but aren't common in current games.

FutureMark's Aki Jarvilehto responded to Nvidia's comments:

"We've been working for the last 18 months to make 3DMark 2003 an accurate and objective benchmark. Nvidia was one of our beta members until December, and it's not our place to comment on why they've decided to withdraw from the program. After working on the project for almost two years with all the leading manufacturers, we do strongly believe that 3DMark 2003 is an accurate representation of game performance for modern 3D accelerators."
Interesting!
 
Sigh. The server ate my message.

To recap: I'll be painted an nvidiot for saying this, but I agree-ish.

From what I have heard, 1.4 isn't being supported in games. Of course, I'm not writing games, so I don't know for sure. NVIDIA works with a lot of developers (but I'm sure they don't get too many 1.4 questions ;)), but supposedly Futuremark has been doing more than simply writing cool looking demos/benchmarks and actually determining what will be the future. Who to believe? I dunno.

I think (personally) that the tests should use HLSL and let the best man win. That would give each vendor the ability to use their card to the best of their abilities. Of course, if the HLSL can't be reduced to work on 1.1, what to do...

Anyways, it is interesting. Its also interesting that HardOCP is following this line of thinking. 2 minds think alike? or a mindless drone?

edit: bold my preference, so it doesn't get lost in the furor that I "agree" with NVIDIA.
 
Russ Said,

From what I have heard, 1.4 isn't being supported in games. Of course, I'm not writing games, so I don't know for sure.

Of course, there is Doom3, where the R-200 path essentially is PS 1.4....and Doom3 has a fallback to NV20 path, which is essentially PS 1.1, multipass.

Just like 3D Mark.

nVidia said:
and that many of the pixel shaders use specific elements of DX8 that are promoted by ATI but aren't common in current games.

Um...so what DX8 elements..AT ALL are common in "today's games?"
 
Can you picture the Nvidia PR machine with buckets venting water frantically out the sides of the sinking ship?
 
Russ:

<shrug> perhaps someone should ask them? I think it's probably a bit of both. I mean, PS 1.4 seems to have some nice advantages over 1.1 when talking about stuff like doom3 (being able to do the lighting in 1 pass) and it seems that it might be atleast somewhat more future proof for that reason. Still, it seems rather odd to me that they didn't include a 1.1 test as well. It seems like it would have been more fair to do something along the lines of:

1 DX7 (Even in a couple of years there will still probably be some DX7 type games hanging around and it's important to atleast verify that the card makes a reasonable effort to perform well with them).

1 DX8 (PS 1.1) this will probably be a pretty likely target simply because there are so many nvidia cards out there that support it that it makes some sense to aim games at it. Still, it's probably not going to be very popular given how much more flexible 2.0 is.

1 DX8.1 (PS 1.4) Maybe not as many cards out there that support it, but there are still a number of them, and it seems to be a good fallback from PS 2.0 in some cases. May be included as a fallback method for non PS 2.0 cards?

1-2 DX9 (PS 2.0) Of course what we are all hoping is the next big thing. Probably should have more weight put on this than the others since PS 1.1 and PS 1.4 seem to really only be stop-gap solutions to get us here.

So I guess I agree with you russ, atleast to a certain extent. Still, they could have some good reasons for setting up the benchmarks the way they did. Perhaps we should wait until they state why it's setup the way it is.

Nite_Hawk
 
Well, nVidia's name has disappeard from the BETA Program Members list on Futuremarks website.

I'm sure it was there the last time I looked, after Worm brought up the BETA program subject in a thread not long ago :?
 
Yeah...in another thread I "predicted" that one set of fanboys (nVidia or ATI) would be claiming "conspiracy" once 3D Mark was released....

...I just didn't think it would be one of the actual companies. :rolleyes:
 
So Joe, changed your mind about your "3DMark might as well be called nVidiaMark" comment yet? :)
 
I thought it was common knowledge (Doomtrooper, HellBinder, Sharkfood, et al) that FutureMark was biased against ATI and that Nvidia had effectively bought off FM. That 3dMark2001 SE's advance pixel shader test was deliberating made to be biased against ATI and pro-NV.

I wonder how this new evidence fits with those conspiracy theories. I wonder if previous 3dmark bashing forum members will now turn around to defend 3dMark (since it shows ATI cards in a favorable light) and argue against anti-NV conspiracy theories that are sure to be put forward.


"The wheel turns, does it not, Ambassador?"
 
Asking to be withdrawn from the list isn't a pretty move if this is what nVidia did. Whether they feel disadvantaged by the new 3DMark or not they were part of the program. They would still be free to critizise the final product of course. This smacks strongly of corporate pressure, and since nVidia is one of the sources of revenue for FutureMark, I wonder about the financial consequences for our Finnish friends, and how much further pressure nVidia can bring to bear on them.

Hmm.
I'll think I'll fork out for the Pro version.

Entropy
 
Joe: http://www.beyond3d.com/forum/viewtopic.php?t=2205

Though you did claim the comment was just testing someone else's patience, so I don't hold you to it ;) That kind of talk has been going on here for the last 2 years though (by various people I don't recall off the top of my head, not necessarily Joe), and it's just funny to watch the evidence mount in the opposite direction.
 
Personally its my guess that some time back Nvidia tried to strongarm 3dmark into making the final tests more complimentary to their cards, when 3dmark refused they backed out.(apparently in December).

Thats total conjecture on my part but it seems to fit in with Nvidias recent track record of overestimating their importance. They probably rationalized totally breaking away from 3dmark by believing they could discredit 3dmark through their fansites and in the press.
Apparently a few 'big boys' have already taken this line. :p
3dmark has never been unbiased, but this release is no more biased than any other really. Theres never going to be a totally unbiased benchmark anyway thats just the nature of the beast.

Nvidia should have realized the power of 3dmark and made the best of it. Graphics companies have been trying to discredit 3dmark since day one and it hasnt worked, I swear these guys are bullet-proof!
 
While I'm somewhat inclinded to believe nVidia's claims, this sounds mighty fishy. Especially when you take into account that their much-hyped GFFX will be hitting shelves soon. This almost seems like a "Cover-Your-Ass" disclaimer;

IE, "If the GFFX gets spanked by the 9700 in 3DMark, blame Futuremark, not us".
 
RussSchultz said:
I think (personally) that the tests should use HLSL and let the best man win. That would give each vendor the ability to use their card to the best of their abilities.

I'll second that.
 
If this isn't the tip of the iceberg, I really don't know what is...

I mean, I feel like I've said this about a dozen times in the last week...Just when you think it couldn't possibly get any worse for the "market leader"...

I mean, they just KEEP outdoing themselves like its going out of style. I can honestly say that my whole perception of this outfit went out the window several months ago, well prior to this whole debacle. But these latest events just completely cement it for me.
 
Back
Top