Beyond3D Forum

jb · 12-Jun-2002, 17:21

http://www.theinquirer.net/12060214.htm

Not sure if its true or not. So instead of your typical pinch of salt I would suggest a large bag of it. But has anyone else confirmed this?

jb · 12-Jun-2002, 17:24

Hmmm got confirmation over on a NVnews thread:

http://www.nvnews.net/forum/showthre...threadid=15459

Please dont turn this into a anti-nVidia or pro-ATI stuff. Just interesting no?

Doomtrooper · 12-Jun-2002, 17:33

This only appears with some newer driver sets from what I read, which is interesting

Bambers · 12-Jun-2002, 17:38

So is the score too high with the screens or too low without?
That nvnews thread has someone with over an 800 point difference between the two scores. More than 'a few percent' to me

jb · 12-Jun-2002, 17:48

When you dont show the screen the score drops. Again not trying to turn this into a bashing thread. It will be interesting to see of other review sites that put ATIs quake/qauke thing under such a microscope will do the same...or will then just shrug it of like HardOCP did?

LittlePenny · 12-Jun-2002, 17:50

If it turns out to be driver optimizations (or cheating as some call it), big whoop. We know ATI and nVidia custom code their drivers, so I really hope we don't have to make fuss everytime someone finds one of these things.

Doomtrooper · 12-Jun-2002, 18:05

I disagree about the big 'whoop', we need to see about 200 major sites put it on their front page..a couple of articles done at Firingsquad and hundreds of Mad Onion shots blown up 400%

Lets not forget the hundreds of threads on forums

Althornin · 12-Jun-2002, 18:19

Quote:

Originally Posted by LittlePenny

If it turns out to be driver optimizations (or cheating as some call it), big whoop. We know ATI and nVidia custom code their drivers, so I really hope we don't have to make fuss everytime someone finds one of these things.

I agree, but it does seem kinda unfair to have this attitude now, without printing some sort of retraction/apology to ATi about the whole quake/quack thing.

Matt Burris · 12-Jun-2002, 18:27

Haven't really looked into this issue (been too busy working on this particular article and not sounding like a fool. :P) - but the intro screens turns off, your score goes lower in 3DMark 2001SE, and people are calling that cheating? I thought the goal in the benchmark is to get higher scores?

Joe DeFuria · 12-Jun-2002, 18:34

Well, the "potential" issue, Matt, is that by default, the tests are run with the intro screens ON (the higher scores.) So the question is...which scores are "correct". The default scores (which are higher), or the scores when the screens are turned off (which are lower.)

Regardless of which ones are correct, I'm at a loss as to how it could make such a significant difference. It would be interesting to learn the cause of it in any case.

LittlePenny · 12-Jun-2002, 18:36

Quote:

Originally Posted by Althornin

I agree, but it does seem kinda unfair to have this attitude now, without printing some sort of retraction/apology to ATi about the whole quake/quack thing.

True, but I also think HardOCP owed ATi an apology long before this ever happened. Everyone knew the references to Quake were in their drivers before the 8500 came out. What Bennet did was journalistic trash.

Crusher · 12-Jun-2002, 18:38

Is this the same Inquirer that has brought us countless gems of media reporting goodness such as AMD's 10 Year Deal With Intel ?

Dave Baumann · 12-Jun-2002, 18:45

Someone has gone around mailing a number of sites with an Excel sheet displaying differences between the 'splash screen' on and off with the conclusion that nVIDIA is using the splash screen to turn on its 3Dmark 'driver optimisations'; we recieved the mail last night but I didn't actually look into it.

Doomtrooper · 12-Jun-2002, 18:48

Pretty hard to doubt the article when the forum links prove it

I think the concern here is why would disabling of title screens affect scores in the first place , and only on Nvidia based cards.
What is being done, not being done in the drivers as its not a 3Dmark issue when title screens are disabled...

Joe DeFuria · 12-Jun-2002, 18:48

Crusher,

Yes, the very same Inquirer. Despite popular belief, they are not always wrong.

(They couldn't always be wrong, because most of their stories contradict other stories they publish.) Anyway, from the MadOnion NVIDIA Faq:

Quote:

This question applies to: NVIDIA GeForce3, NVIDIA GeForce3 Ti 200, NVIDIA GeForce3 Ti 500, NVIDIA GeForce4 Ti 4200, NVIDIA GeForce4 Ti 4400, NVIDIA GeForce4 Ti 4600
Q: When I disable the loading screens between the benchmarktests, my score drops when run with NVIDIA accelerator. Why?
A: This most likely is a NVIDIA driver issue. We are investigating this.

LittlePenny,

Quote:

True, but I also think HardOCP owed ATi an apology long before this ever happened.

And that's the problem. Double-standards. (Not you, Hard OCP for one.) Lot's of people accused Hard OCP of being nVidia biased based on their "Quack Expose." I Believe Hard OCP defended themselves saying stuff like "they are only after the truth" kinda thing...so what would you expect Hard OCP's apprepriate response to this issue, which clearly requires further "investigation" to learn the truth?
Here's what they said:

Quote:

It is a hot topic via email to us today that there is something afoot with the NVIDIA GeForce4. Seems as though if you disable splash screens between benchmarks in MadOnion's 3DMark 2001 SE, your score will decrease by 500 to 800 points. Some seem to think that this is a sure sign of NVIDIA cheating while others see a 3DMark 2001 bug. I say let's all meet up tonight, get drunk, and argue about it.

Hell, if you are going to beat up on NVIDIA, why not ask them about their Ti GPUs having a green banding issue that we have seen across multiple GPUs and card builders, including our reference design. Another thread. And another.

We specifically questioned NVIDIA on this over a month ago, to never get a reply.

Where's the hard-nosed investigation? An e-mail that was ignored...and no "red alert" to the community about potential shenanigans? Why not re-name 3DMark, or hack in a different splash screen, look at individual tests, compare FPS numbers with the reported 3D Mark score....

Hellbinder · 12-Jun-2002, 18:49

Quote:

Please dont turn this into a anti-nVidia or pro-ATI stuff. Just interesting no?

This, is utterly unacceptable imo. Ati people sit here and get thronged by people like chalnoth(sp?), and many others. Yet suddenly becuase its Nvidia we are all supposed to say, "hmm thats interesting". Somehow Quak's 4 reduced lod textures that resulted in 5fps, and lasted for exactly ONE 2 week driver release is = to a few months of 400% inflated 3dmark scores?

I dont think so, but this is the only post on the subject i will make.

Doomtrooper · 12-Jun-2002, 18:53

Fanatics are dropping like flies

Matt Burris · 12-Jun-2002, 18:55

My take on it is that the default benchmark would be with title screens on, as that's enabled when you start the program up. I really wonder what the option is there for though, why did Madonion put it in there in the first place? Is there any purpose to turning off the intro screens?

This is a pretty interesting issue, but I fail to see the similarities between Quack and this event, as we've yet to see any code that resembles If.3DMark=true then run>optimizations or anything of that sort. If what Inquirer said is true though, then the situation could be the same (the whole "bug in the drivers" routine). I'd ask NVIDIA myself, but my contacts are unavailable for the day, it seems.

jb · 12-Jun-2002, 19:07

There are some similarities:

Both are very popular benchmarking tools
Both showed a difference in FPS scores when run in a different matter (3dmark disable splash & Q3 rename exe)

Agreed that the case is not the same thing. With all of the data gathered I believed that the Q3/ATI was a bug. At this point I believe that the same thing exists with nVidia that is not a driver optimization but a bug. Also people like HardOCP blew that whole ATI thing right out of the water when they failed to realize that Q3 ref had existed way before the launch of the 8500 in their drivers. They over analyzed and provided results based on a sub set of the facts not all of them. Maybe the learned their lesson and will wait until they get all the facts? Maybe they are indeed bias? Who knows but I will bet you that we probably wont see as many if any articles investigation this vrs the 5 or so we saw with the Q3/ATI stuff...

Doomtrooper · 12-Jun-2002, 19:15

Quote:

Originally Posted by LittlePenny

Quote:

Originally Posted by Althornin

I agree, but it does seem kinda unfair to have this attitude now, without printing some sort of retraction/apology to ATi about the whole quake/quack thing.

True, but I also think HardOCP owed ATi an apology long before this ever happened. Everyone knew the references to Quake were in their drivers before the 8500 came out. What Bennet did was journalistic trash.

Hmmm ATI remembers the 8500 launch well...and I think and Hope the Review cards go like this...

1) www.beyond3d.com
2) www.anandtech.com
3) www.tomshardware.com
4) www.firingsquad.com
110)www.hardocp.com ETA Summer 2003

Joe DeFuria · 12-Jun-2002, 19:18

Matt,

We know that the default benchmarks are those with the screens on. That's part of the problem. "By default" the higher scores are reported. (Of course, that's only a problem if the "lower" scores are the correct scores.)

I basically agree with what JB said, to expand one thing he said:
* Both showed a difference in FPS scores when run in a different manner that (3dmark disable splash & Q3 rename exe) and one wouldn't expect this behavior to happen.

It WOULD be interesting, and it's worth "investigating" this unexpected behavior. It's just not worth getting all wrapped up about unless there is some proof of purposeful wrong-doing. But that's what the ATI camp has been saying since the Quack thing...

borzwazie · 12-Jun-2002, 19:31

I don't even own an ATI card but I agree with those who say the way HardOCP hammered the ATI driver "cheat" issue was far from professional. I will also admit, however, that the increased attention to drivers on public websites, as well as the realization by ATI of that attention has done more to improve their drivers than anything else.

The reason I don't own an ATI card is the driver issue, which I see to be less and less of a point at this time. I'm looking for a newer card (as soon as I get the money together for it), and it's a pretty tough call about which card to pick now. The 8500 is a damned good card, and with good drivers and it's current price point, it's a pretty tempting buy. The Ti4200 looks sweet too, oh the decisions

lurtz · 12-Jun-2002, 19:45

who knows whether it is a cheat or an "optimization" but...

http://www.nvnews.net/articles/david...nterview.shtml

"We at Nvidia don't make it a practice to optimize our pipeline for specific benchmarks - we want to provide high quality and high performance on a wide variety of useful and entertaining applications, rather than just getting a good score." David Kirk, Nvidia

muted · 12-Jun-2002, 20:12

I don't know about you , but i just love how The Inquirer never gives out their sources .... since it came from a forum .. and the INQ had "DOCUMENTS"

Quote:

Alain Tiquet, marketing director of Nvidia Europe, told the INQUIRER that both his company and Mad Onion were aware of the problem and they are working together to fix the problem, which he said only made a few percentage points difference.

A new driver to fix the problem will be made available later in the day, he added

now i'm troubled by this ... How is this a problem , and i thought higher was better . If Anyone knows since when this "problem" is active, it would be interesting to see

Vince · 12-Jun-2002, 20:21

Quote:

"We at Nvidia don't make it a practice to optimize our pipeline for specific benchmarks - we want to provide high quality and high performance on a wide variety of useful and entertaining applications, rather than just getting a good score." David Kirk, Nvidia

What BS. Q3A Timedemo anyone?

12-Jun-2002, 17:21	#1
jb Senior Member Join Date: Feb 2002 Posts: 1,636	GF4 has inflated 3dmarks scores so says the INQ..... http://www.theinquirer.net/12060214.htm Not sure if its true or not. So instead of your typical pinch of salt I would suggest a large bag of it. But has anyone else confirmed this?

12-Jun-2002, 18:27	#9
Matt Burris Member Join Date: May 2002 Posts: 110	Haven't really looked into this issue (been too busy working on this particular article and not sounding like a fool. :P) - but the intro screens turns off, your score goes lower in 3DMark 2001SE, and people are calling that cheating? I thought the goal in the benchmark is to get higher scores? __________________ Matt Burris - 3DGPU

12-Jun-2002, 18:38	#12
Crusher Aptitudinal Constituent Join Date: Mar 2002 Posts: 869	Is this the same Inquirer that has brought us countless gems of media reporting goodness such as AMD's 10 Year Deal With Intel ? __________________ Crusher The metric system is the tool of the devil! My car gets 40 rods to the hogshead, and that’s the way I likes it!

12-Jun-2002, 18:45	#13
Dave Baumann Gamerscore Wh... Join Date: Jan 2002 Posts: 12,947	Someone has gone around mailing a number of sites with an Excel sheet displaying differences between the 'splash screen' on and off with the conclusion that nVIDIA is using the splash screen to turn on its 3Dmark 'driver optimisations'; we recieved the mail last night but I didn't actually look into it. __________________ Expand. Accelerate. Dominate. Tweet Tweet!

12-Jun-2002, 18:55	#18
Matt Burris Member Join Date: May 2002 Posts: 110	My take on it is that the default benchmark would be with title screens on, as that's enabled when you start the program up. I really wonder what the option is there for though, why did Madonion put it in there in the first place? Is there any purpose to turning off the intro screens? This is a pretty interesting issue, but I fail to see the similarities between Quack and this event, as we've yet to see any code that resembles If.3DMark=true then run>optimizations or anything of that sort. If what Inquirer said is true though, then the situation could be the same (the whole "bug in the drivers" routine). I'd ask NVIDIA myself, but my contacts are unavailable for the day, it seems. __________________ Matt Burris - 3DGPU

12-Jun-2002, 17:24	#2
jb Senior Member Join Date: Feb 2002 Posts: 1,636	Hmmm got confirmation over on a NVnews thread: http://www.nvnews.net/forum/showthre...threadid=15459 Please dont turn this into a anti-nVidia or pro-ATI stuff. Just interesting no?

12-Jun-2002, 17:33	#3
Doomtrooper Senior Member Join Date: Feb 2002 Location: Ontario, Canada Posts: 3,328	This only appears with some newer driver sets from what I read, which is interesting

12-Jun-2002, 17:38	#4
Bambers Member Join Date: Mar 2002 Location: Bristol, UK Posts: 781	So is the score too high with the screens or too low without? That nvnews thread has someone with over an 800 point difference between the two scores. More than 'a few percent' to me

12-Jun-2002, 17:48	#5
jb Senior Member Join Date: Feb 2002 Posts: 1,636	When you dont show the screen the score drops. Again not trying to turn this into a bashing thread. It will be interesting to see of other review sites that put ATIs quake/qauke thing under such a microscope will do the same...or will then just shrug it of like HardOCP did?

12-Jun-2002, 17:50	#6
LittlePenny Member Join Date: Feb 2002 Location: Rolla, Missouri, USA Posts: 276	If it turns out to be driver optimizations (or cheating as some call it), big whoop. We know ATI and nVidia custom code their drivers, so I really hope we don't have to make fuss everytime someone finds one of these things.

12-Jun-2002, 18:05	#7
Doomtrooper Senior Member Join Date: Feb 2002 Location: Ontario, Canada Posts: 3,328	I disagree about the big 'whoop', we need to see about 200 major sites put it on their front page..a couple of articles done at Firingsquad and hundreds of Mad Onion shots blown up 400% Lets not forget the hundreds of threads on forums

12-Jun-2002, 18:34	#10
Joe DeFuria Regular Join Date: Feb 2002 Posts: 5,951	Well, the "potential" issue, Matt, is that by default, the tests are run with the intro screens ON (the higher scores.) So the question is...which scores are "correct". The default scores (which are higher), or the scores when the screens are turned off (which are lower.) Regardless of which ones are correct, I'm at a loss as to how it could make such a significant difference. It would be interesting to learn the cause of it in any case.

12-Jun-2002, 18:48	#14
Doomtrooper Senior Member Join Date: Feb 2002 Location: Ontario, Canada Posts: 3,328	Pretty hard to doubt the article when the forum links prove it I think the concern here is why would disabling of title screens affect scores in the first place , and only on Nvidia based cards. What is being done, not being done in the drivers as its not a 3Dmark issue when title screens are disabled...

12-Jun-2002, 18:53	#17
Doomtrooper Senior Member Join Date: Feb 2002 Location: Ontario, Canada Posts: 3,328	Fanatics are dropping like flies

12-Jun-2002, 19:07	#19
jb Senior Member Join Date: Feb 2002 Posts: 1,636	There are some similarities: Both are very popular benchmarking tools Both showed a difference in FPS scores when run in a different matter (3dmark disable splash & Q3 rename exe) Agreed that the case is not the same thing. With all of the data gathered I believed that the Q3/ATI was a bug. At this point I believe that the same thing exists with nVidia that is not a driver optimization but a bug. Also people like HardOCP blew that whole ATI thing right out of the water when they failed to realize that Q3 ref had existed way before the launch of the 8500 in their drivers. They over analyzed and provided results based on a sub set of the facts not all of them. Maybe the learned their lesson and will wait until they get all the facts? Maybe they are indeed bias? Who knows but I will bet you that we probably wont see as many if any articles investigation this vrs the 5 or so we saw with the Q3/ATI stuff...

12-Jun-2002, 19:18	#21
Joe DeFuria Regular Join Date: Feb 2002 Posts: 5,951	Matt, We know that the default benchmarks are those with the screens on. That's part of the problem. "By default" the higher scores are reported. (Of course, that's only a problem if the "lower" scores are the correct scores.) I basically agree with what JB said, to expand one thing he said: * Both showed a difference in FPS scores when run in a different manner that (3dmark disable splash & Q3 rename exe) and one wouldn't expect this behavior to happen. It WOULD be interesting, and it's worth "investigating" this unexpected behavior. It's just not worth getting all wrapped up about unless there is some proof of purposeful wrong-doing. But that's what the ATI camp has been saying since the Quack thing...

12-Jun-2002, 19:31	#22
borzwazie Junior Member Join Date: Feb 2002 Posts: 37	hmm I don't even own an ATI card but I agree with those who say the way HardOCP hammered the ATI driver "cheat" issue was far from professional. I will also admit, however, that the increased attention to drivers on public websites, as well as the realization by ATI of that attention has done more to improve their drivers than anything else. The reason I don't own an ATI card is the driver issue, which I see to be less and less of a point at this time. I'm looking for a newer card (as soon as I get the money together for it), and it's a pretty tough call about which card to pick now. The 8500 is a damned good card, and with good drivers and it's current price point, it's a pretty tempting buy. The Ti4200 looks sweet too, oh the decisions

12-Jun-2002, 19:45	#23
lurtz Registered Join Date: Jun 2002 Posts: 3	who knows whether it is a cheat or an "optimization" but... http://www.nvnews.net/articles/david...nterview.shtml "We at Nvidia don't make it a practice to optimize our pipeline for specific benchmarks - we want to provide high quality and high performance on a wide variety of useful and entertaining applications, rather than just getting a good score." David Kirk, Nvidia

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
G70 scores 7700 3DMarks, says Inq	_xxx_	3D Hardware, Software & Output Devices	143	22-Jun-2005 12:21
3DMark05 scores courtesy the Inq	trinibwoy	3D Hardware, Software & Output Devices	422	01-Oct-2004 14:13
questions about 3dmark scores	AOS	Hardware & Software Talk	6	30-Apr-2004 22:54
My 3Dmark Scores are normal on my ATI 9800 Pro ?	rainz	3D Hardware, Software & Output Devices	18	11-Feb-2004 19:18
Hype and review scores.	Geeforcer	Console Technology	67	05-Dec-2002 07:31

Beyond3D Forum

Welcome, Unregistered.