nVidia's reply to 3dMark

Eolirin

Regular
From CNet...

"A representative at Nvidia questioned the validity of Futuremark's conclusions. 'Since Nvidia is not part of the Futuremark beta program (a program which costs of hundreds of thousands of dollars to participate in), we do not get a chance to work with Futuremark on writing the shaders like we would with a real applications developer,' the representative said. 'We don't know what they did, but it looks like they have intentionally tried to create a scenario that makes our products look bad.'"

http://news.com.com/2100-1046_3-1009574.html?tag=fd_top
 
trust me on this one. Someone at nVidia needs to clean out the PR department as they are really starting to make nVidia look worse than they really are....
 
so B3D (who is a member in the beta program) is paying "hundreds of thousands of dollars to participate" in the Beta Programm? i somehow doubt this...
 
More Smoke and Mirrors from Nvidia? Why am I not surprised. :rolleyes:

Perhaps they can explain why my GF4 ti4400 with the 43.45 drivers lost 400 pts. in 3DMark03 b3.3.0 all else being equal? I dont think this is limited to the 44.xx drivers or the FX video cards. I think this has been going on for some time and it was just never caught. My last 5 purchases have been Nvidia products, and I have recommended their cards to my Customers and friends. That has ended and I sent a nice letter to Nvidia explaining this to them. Words only go so Far. I intend on speaking with my $$$. That's something they WILL understand!



Nvidia...

Fool me Once, shame on you! Fool me Twice, shame on Me!

bye-bye Nvidia!
 
I could comment a few inaccuracies in this quote:

Eolirin said:
From CNet...

"A representative at Nvidia questioned the validity of Futuremark's conclusions. 'Since Nvidia is not part of the Futuremark beta program (a program which costs of hundreds of thousands of dollars to participate in), we do not get a chance to work with Futuremark on writing the shaders like we would with a real applications developer,' the representative said. 'We don't know what they did, but it looks like they have intentionally tried to create a scenario that makes our products look bad.'"

http://news.com.com/2100-1046_3-1009574.html?tag=fd_top

First of all, you can join the Futuremark beta program, if you are in some way a significant player in the PC industry and you pay the membership fee of $5000 USD / year.

And second, Nvidia was a member of our beta program, got pre-release builds, shader source code, and participated actively in the development of 3DMark03. After they left our beta program, there was mostly just bug hunting and product quality assurance left.

I don't see how they could 'not know what we did, in order to intentionally try to create a scenario that makes their products look bad'.
 
Eolirin said:
From CNet...

"A representative at Nvidia questioned the validity of Futuremark's conclusions. 'Since Nvidia is not part of the Futuremark beta program (a program which costs of hundreds of thousands of dollars to participate in).

1. $5000 to join
2. ~$195,000. to develop cheats for the drivers

Since they left they save on number one but they still seem to spend on number 2. Spin, spin., spin. :devilish:
 
It seems CNet only mentioned parts of the response. According to a site called HardAvenue (www.hardavenue.com) the response also continues as follows:

We don't know what they did but it looks like they have intentionally tried to create a scenario that makes our products look bad. This is obvious since our relative performance on games like Unreal Tournament 2003 and Doom3 shows that The GeForce FX 5900 is by far the fastest graphics on the market today.

Unlike a game developer, Future Mark has a motive to make their application run poorly on one IHV's hardware becasuse that IHV refuses to pay them hundreds of thousands of dollars per year to be part of their beta program. A real application or game is best served by running as well as possible on all hardware to give the end user the best experience, but 3DMark03 uniquely is not.

I really think that this must be some sort of misunderstanding from HardAvenue's part (the second chapter must be their own speculations which has been accidentially quoted as part of Nvidia's response). Especially considering that the minimum payment to join Beta Program as a Beta Member is $5.000/year.

Could someone please quote the full Nvidia response here so that this misunderstanding could be sorted out?

Cheers,

AJ

Ps. Besides of the above has anyone received a Technical Response regarding the topic?
 
AJ said:
Ps. Besides of the above has anyone received a Technical Response regarding the topic?
You mean from NVIDIA? Nope, none to me... simply because I never asked them... although I'd been meaning.

I'm going to write Brian Burke later.
 
Classic!!! This is just the sort of disinformation, "cover our arses" responce I have come to expect of big business. The finger pointing when one gets caught does not make one look better either. nVidia's responce is no better then the drunk person who hits the parked car and says "Officer, honestly, I didn't hit the car. The car hit me. It's the cars own fault for being there".

Let me see, if I cheat on some test and then try to accuse those who designed the test of trying to make me look bad, just how much credibility would I have? And no, I wouldn't be unscuplous enough to try it on a college exam, even though there have been a few teachers to load a test with material the course never covered...but that's a different story.

And second, Nvidia was a member of our beta program, got pre-release builds, shader source code, and participated actively in the development of 3DMark03. After they left our beta program, there was mostly just bug hunting and product quality assurance left.

And that's what makes this worse. There's an obvious omission of facts here, in fact that's putting it mildly, as what they say is actually a bending of the truth in saying "because nVidia is not a part of the beta program..." we couldn't have known what they were doing. This is mis-leading at best, as close to deception as one can get without flat out lieing.

Put their statement together with the facts of their prior involvement in the beta program and their dropping out, another picture is painted. Such as they're only being a member so they could figure out how to cheat, dropping out at a point they expected nothing more to change, and then getting all offended and defensive when they get caught and a change is made...as it would need to be, to level the playng field... Finger pointing when one gets caught in an act of wrong doing, how classic!
 
Eolirin said:
From CNet...

"A representative at Nvidia questioned the validity of Futuremark's conclusions. 'Since Nvidia is not part of the Futuremark beta program (a program which costs of hundreds of thousands of dollars to participate in), we do not get a chance to work with Futuremark on writing the shaders like we would with a real applications developer,' the representative said. 'We don't know what they did, but it looks like they have intentionally tried to create a scenario that makes our products look bad.'"

We don't know what they did but it looks like they have intentionally tried to create a scenario that makes our products look bad. This is obvious since our relative performance on games like Unreal Tournament 2003 and Doom3 shows that The GeForce FX 5900 is by far the fastest graphics on the market today.

Unlike a game developer, Future Mark has a motive to make their application run poorly on one IHV's hardware becasuse that IHV refuses to pay them hundreds of thousands of dollars per year to be part of their beta program. A real application or game is best served by running as well as possible on all hardware to give the end user the best experience, but 3DMark03 uniquely is not.



if true
cost=P.R.
pay=slander (if less $100,000)
 
Eolirin said:
From CNet...

"A representative at Nvidia questioned the validity of Futuremark's conclusions. 'Since Nvidia is not part of the Futuremark beta program (a program which costs of hundreds of thousands of dollars to participate in), we do not get a chance to work with Futuremark on writing the shaders like we would with a real applications developer,' the representative said. 'We don't know what they did, but it looks like they have intentionally tried to create a scenario that makes our products look bad.'"

I commented this already above.

We don't know what they did but it looks like they have intentionally tried to create a scenario that makes our products look bad. This is obvious since our relative performance on games like Unreal Tournament 2003 and Doom3 shows that The GeForce FX 5900 is by far the fastest graphics on the market today.

If this is an Nvidia quote, I'm not surprised why they crown their own product as the fastest.
- Hey, it can very well be the fastest card out there, but some game benchmarks, other than those mentioned in the quote, show different numbers compared to the competition. This means that the winner is yet undecided and the exciting race for the fastest card continues.

Unlike a game developer, Future Mark has a motive to make their application run poorly on one IHV's hardware becasuse that IHV refuses to pay them hundreds of thousands of dollars per year to be part of their beta program. A real application or game is best served by running as well as possible on all hardware to give the end user the best experience, but 3DMark03 uniquely is not



if true
cost=P.R.
pay=slander (if less $100,000)

Don't worry, we're not that cruel :LOL:
Even though I would prefer to work with Nvidia also on the next 3DMark version, I respect their wish to leave our beta program. We will continue making benchmarks for high end 3D hardware, and we will keep them impartial since a biased benchmark has no value.
 
How can a GFFX5900 be the "fastest card on the market" if it isn't on the market yet?


at any rate here's a reply to Nvidia's reply...

http://slashdot.org/

"Futuremark replies Posted Tuesday, May 27 by nzaweird
We had a chance to get the follow up remarks of Tero Sarkkinen's, Executive Vice President of Sales and Marketing at Futuremark, regarding the recent claims of Nvidia.


HardAve: In light of the recent struggles between your 3DMark2003 and Nvidia's driver set, we would like to hear your remarks regarding Nvidia's claims your software intentionally put the GeforceFX product in bad light after Nvidia passed on the opportunity in becoming a beta program partner of Futuremark's. On top of this, with the new 330 patch, it appears 3DMark2003 is the only benchmark showing the GeforceFX 5900Ultra well behind the competition, where other apps like UT2003 and Doom3 showcase the exact opposite. Why is this?

Tero Sarkkinen: Any suggestion that Futuremark would intentionally penalize or favor any specific hardware in our products is absurd and NOT TRUE. And please note that we are not attacking here, we are just defending our product.

We are in the business of making objective benchmarks and we intend to keep it that way. We respect deeply our BETA members, which include the biggest in the industry, and they all have participated in the development of 3DMark03. NVIDIA themselves was an active member of the BETA program until December 2002.

NVIDIA's claim that "since they are not a beta partner, they do not get a chance to write shaders like they would with real applications developer" is irrelevant and to us it seems like an attempt to shift discussion to a different topic. The topic here is that the drivers special cased 3DMark03 and resulted in an incorrect score which made false representation of their products' performance in 3DMark03.

3DMark03 was developed strictly according to DirectX9 standard in very close cooperation with Microsoft and other BETA members. If hardware performs well 3DMark03, it performs well in all applications that use DirectX 9. Note that since 3DMark is designed to be an objective evaluation tool, it does _not_ include manufacturer-specific optimizations. This is why it is exceptionally well suitable for objective performance measurement.

Since we all now see how tempting it is to try to cheat in a benchmark, we have started to work with our partners in order to develop new structures and processes to weed out cheating and unfair play in all benchmarks. This is a business opportunity for us, since due to our role and position we are better suited to act in this role than e.g. game benchmark providers. We welcome all interested parties to work with us in this important initiative.

NVIDIA is an extremely capable company with great products. We welcome them to continue the competition in the hardware development with fair means.

In reply to your more specific question, 3DMark03 is a forward looking DirectX 9 benchmark. It stresses the hardware with workloads that will be typical of DirectX 9 games. Thus, hardware's performance in Unreal Tournament 2003 does not necessarily bear relation to that how that hardware will perform in DirectX 9 workloads, which make an extensive use of pixel and vertex shading. Doom 3 is not a published product yet but we have seen few reviews. Based on those reviews, it seems that in Doom 3 there are performance differences depending on which codepath is used for different hardware. Not much more can be said at this time, as the application is not ready and not available for public testing (although we very much would like to see it :)"
 
Just FYI: a poster at Slashdot attributes this Nvidia comment to Derek Perez, so we would have a name to go with that quote, not just a vague reference to "an Nvidia representative".
 
What is the maximum membership fee?
I don't think $5000 is enough for first-tier membership.
Is it confidential?
 
nelg said:
1. $5000 to join
2. ~$195,000. to develop cheats for the drivers

Heh...


Joining 3DMark03 beta programme............$5000
Leaving 3DMark03 beta programme...........$Free
Developing cheats in your driver................$195,000
Hours of free advertising due to bad press and countless forum discussions: Priceless.
 
does benchmarking with 3dmark matter anymore

I didn't see one game that was actually playable online or something that would actually make me want to pay 50 (estimated) for a benchmarking tool. I don't care who cheats. I am more concerned with the actual games. It still anyones game as to who will have the most graphics power. I see that Nvidia has the reign again, but who knows. Maybe ATI will come out with something better and blow them away again. so stop wasting your time and money on such a worthless topic and worry about the real games. (okay I feel better, had to get that off my chest) :oops:
 
Re: does benchmarking with 3dmark matter anymore

vicious said:
I didn't see one game that was actually playable online or something that would actually make me want to pay 50 (estimated) for a benchmarking tool. I don't care who cheats. I am more concerned with the actual games. It still anyones game as to who will have the most graphics power. I see that Nvidia has the reign again, but who knows. Maybe ATI will come out with something better and blow them away again. so stop wasting your time and money on such a worthless topic and worry about the real games. (okay I feel better, had to get that off my chest) :oops:
It's official......I'm no longer the dumbest person on this forum. ;)
 
Re: does benchmarking with 3dmark matter anymore

micron said:
vicious said:
I didn't see one game that was actually playable online or something that would actually make me want to pay 50 (estimated) for a benchmarking tool. I don't care who cheats. I am more concerned with the actual games. It still anyones game as to who will have the most graphics power. I see that Nvidia has the reign again, but who knows. Maybe ATI will come out with something better and blow them away again. so stop wasting your time and money on such a worthless topic and worry about the real games. (okay I feel better, had to get that off my chest) :oops:
It's official......I'm no longer the dumbest person on this forum. ;)

lol - nearly spilled my cup of water (with an Aspirin...)

Dear vicious,

when a hardware vendor intentionally cheats in a benchmark that is widely used to evaluate gfx-performance of different cards at both OEMs and end-users then it IS important to discuss it.
The OEMs can/should feel cheated because they buy the underperforming hardware on the false assumption that it is actually fast. Same goes for end-users.
This IS serious and can involve many $$$$'s.
edit: it doesnt matter a bit if the benchmark is "worthless" or not. NVidia is misleading its customers on purpose. And THAT is important to remember. At least for me.

CU
Mav
 
this is a sad time in 3d graphics for all of us. nvidia and their shor sightedness is not going to do anything to help the industry.


Lets see the important things for me are all equal

1. FSAA QUALITY
2. ANSIO QUALITY
3. PERFORMANCE with 1 AND 2
4. IS RENDERED OUTPUT FAITHFUL TO WHAT DEVELOPER WANTED?


nVidia's 3X hardware cannot compete once you enable one and two mentioned above as the performance hit is too massive for my liking, especially once you factor in the price of the cards in question.

For all of the games released to date there are like 2 or 3 that a Radeon 8500 would have troubles running. I can even get UT 2K3 running on a Penttium III with a Radeon 32MB card and 512MB ram at default clock speeds although you have to tank the resolution down to 512*384*32 for acceptable frames and I think that is most like CPU killing performance there.

Now question is WHO'S CARD WILL BE ABLE TO RUN DOOM III AT MAX IQ SETTINGS

ANSWER ATI as they are using the ARB2 path vs NV3X using NV30 I goota run at 12INT or 16FP JUST TO BE ABLE TO COMPETE.

Carmak already stated that ARB2 IS NOT THE WAY TO DO IT WITH NVIDIA. That single statement explains why you would avoid the nV3X if you are going to do anything that requires HIGH IQ WITH SPEED.
 
Back
Top