Man...you are way off base, are you for real?....been around the industry long?......Ah waite....you own an Nvidia product dont you......Richthofen said:Doomtrooper said:Solomon said:When is Aquamark 3 coming out so we can just forget about 3DMark03?
So doing similar things to Aquamark makes that a better 'benchmark'
If they need to hack a Benchmark to get frames, they need to hack game titles to get frames...the hardware doesn't magically fix itself when a game is ran.
why do they? What has 3DMark to do with typicall games.
3dMark is more kind of PS1.4 test than anything else. Guess which company colaborated with them to do this?
I don't care if Nvidia cheats in 3DMark at all. I know they do it and i know others will do the same as long as these stupid synthetic benchmarks exist. Time do drop that crap soon.
Same with CPUs. I am sick of reading reviews with over 50% of the tests done with benchmarks like sysmark, pcmark, sandra and so on.
ExtremeTech Article Up
- Thread starter TempestX
- Start date
Interesting (if not suprising) lines being drawn here...... IF you subtract those that favor one IHV over another, what you see is the middle being as outraged by this as much as anyone is. I have to look at what Type says(as a former big time nVidia supporter) and wonder when the rest of you will get it.... nVidia is cheating on you, too.
This is the whole point of being ethical. While many here have complained for years about nVidia's ethics, their supporters would reply "it's business". Would any of you accept thid kind of lying & deceit from a loved one or friend? Then why accept it from a company.
This has nothing to do with the products nVidia makes. It has everything to do with how they view you, as their customer - with absolutely NO RESPECT! And, the sad thing is they will continue this because their fans allow it.
This is the whole point of being ethical. While many here have complained for years about nVidia's ethics, their supporters would reply "it's business". Would any of you accept thid kind of lying & deceit from a loved one or friend? Then why accept it from a company.
This has nothing to do with the products nVidia makes. It has everything to do with how they view you, as their customer - with absolutely NO RESPECT! And, the sad thing is they will continue this because their fans allow it.
THe_KELRaTH
Regular
It's certainly a cheat. Nvidia thought they would get away with this one easily but were unaware of the other modes in the 3Dmark software that showed up what they are up to.
I don't think "cheat" is the right word either. Creating drivers to deliberately supply false information so as to sway reviewers and ultimately end users into thinking their product is the fastest with the outcome of selling more product and altering market share is not just "cheating" - rather deliberate deception.
I bet there's more to come too.
The only thing Nvidia have demonstrated to me is that as a company they fall into the same catagory as Double Glazing Salesmen and are no better than athletes taking steroids.
I don't think "cheat" is the right word either. Creating drivers to deliberately supply false information so as to sway reviewers and ultimately end users into thinking their product is the fastest with the outcome of selling more product and altering market share is not just "cheating" - rather deliberate deception.
I bet there's more to come too.
The only thing Nvidia have demonstrated to me is that as a company they fall into the same catagory as Double Glazing Salesmen and are no better than athletes taking steroids.
What is needed is some way to 'fool' the Nvidia drivers that it is not running 3DMark2003 so that it defaults to its generic 3d rendering path in the driver (which will remove both cheats and acceptable optimizations). Perhaps a 3D wrapper program of sorts or something similar. I believe that the detection method employed by Nvidia and ATI deals with detecting specific textures belonging to a benchmark, so the wrapper would have to slightly alter all textures.
If the wrapper works successfully on negating all optimizations and cheats on ATI and Nvidia cards (not saying ATI has any cheats, optimizations for sure though) then we can have a true apples to apples comparison between the generic DX9 3d path rather than some tweaked 3DMark2003 path. This is desireable since many games will also using the generic DX9 path.
If the wrapper works successfully on negating all optimizations and cheats on ATI and Nvidia cards (not saying ATI has any cheats, optimizations for sure though) then we can have a true apples to apples comparison between the generic DX9 3d path rather than some tweaked 3DMark2003 path. This is desireable since many games will also using the generic DX9 path.
Hmm, thats an interresting idea, just look for a set of check sums or CRC's in the textures that you've loaded, quite cheep to do. Other suggestions that the driver could look for state sequences and specific shaders can all end up costing quiet a bit of perf in the driver, but this... App could easily defeat it by adding noise into texture LSB's so that you're unlikley to ever get the same chechsum for anyone texture, might mess with DXTn nastily though, maybe it would be enough to change the order in which you load your textures randomly, neither effect repeatability of the BM, which is good...
THe_KELRaTH said:
Not to pick nits, but the definition of cheating is "deliberate deception"...
I agree there's more to this story--I want to know more about what they are doing with their FSAA modes and the post filter, myself. If it's just some kind of "poor man's gamma correction" or something that wouldn't be so bad...but I do wonder if they aren't using post-filter blending in conjunction with a lower-than-expected FSAA mode in order to inflate their QC-4x FSAA mode performance numbers.
Any fixed camera timedemo benchmark is now suspect.
Throw them all out, including the Doom 3 scores. Anything that has a fixed camera can have those clip planes added in.
This leaves review sites with one option at the moment with timedemos. Record a new one each time (variable) and review the cards then.
Get a new card or test? Redo the timedemos in a different area.
And to people complaining, its just a synthetic benchmark, what about this? Say a card gets 50FPS in a game benchmark, (using those artifical clip planes) yet only gets around 30FPS without them at certain settings? How is this not cheating or lieing to the consumer?
Throw them all out, including the Doom 3 scores. Anything that has a fixed camera can have those clip planes added in.
This leaves review sites with one option at the moment with timedemos. Record a new one each time (variable) and review the cards then.
Get a new card or test? Redo the timedemos in a different area.
And to people complaining, its just a synthetic benchmark, what about this? Say a card gets 50FPS in a game benchmark, (using those artifical clip planes) yet only gets around 30FPS without them at certain settings? How is this not cheating or lieing to the consumer?
Richthofen said:Doomtrooper said:Solomon said:When is Aquamark 3 coming out so we can just forget about 3DMark03?
So doing similar things to Aquamark makes that a better 'benchmark'
If they need to hack a Benchmark to get frames, they need to hack game titles to get frames...the hardware doesn't magically fix itself when a game is ran.
why do they? What has 3DMark to do with typicall games.
3dMark is more kind of PS1.4 test than anything else. Guess which company colaborated with them to do this?
Bullsh*t. 3dm2k3 is the first usable 3dmark ever. (I mean for VGA testing. Pure testing.)
Let me turn this thing back: why are they against 2k3 since its started? Because they've seen there is no more space left for cheating?
So - how you like this idea?
I hope you're just joking - it's about our moral...
I don't think so. Stupid for you maybe - but it could be very useful for measuring technical capabilities and hidden flaws/goods.
Probably you should try to interpret those datas...
Demalion:
Well, I knew some people weren't gonna like this post when I typed it. But anyway...
First, how is this excessively complicated? I believe you're significantly overestimating the work which has gone into this.
Doing it by hand is *impossible*. You'd be insane to do that. However, simply writing a driver revision writing some test results to a file, and then another driver able to interpret that data, should be quite easy. I think many people think nVidia wasted a lot of time on this. I don't think so, more like several days I guess.
Furthermore, nVidia might see it as a "long term investment": now that they've written this app, which isn't THAT hard to program, they might be able to apply it to any other benchmark.
And yes, that part is very ugly, and I'd get quite disgusted at them if they began doing that. So if they're busted in other benchmarks for similar things, I can promise you I won't have the same attitude anymore. I mean, if they used it in all timedemos, but couldn't figure out a way to do anything similar in real games.
Alos, as I said, I'm against cheating. I think it'd be better if those companies didn't cheat. But my point is that this issue is being exagerated. When I first heard that some sites were gonna expose it all soon, I supposed it was gonna be a lot more nasty than this. I guess I've got a subjective issue on this whole story for this very reason: I thought it was gonna be MUCH worse.
Also, while the current cheating methods are special-purpose, the DIP caching I propose can easily be applied to other things even without a very general system. That means using it for any game which use illogical draw orders for things like the sky should be rather easy, although it's still special purpose code: but if you use special purpose code everywhere, then it isn't as annoying as if you use special purpose code in only one game/benchmark to lie to your customers.
Anyway, to summarize, sorry if this seemed like I'm accepting that nVidia cheats. I'm not. But what I'm saying is:
1. This isn't as big of a deal as the forums, news sites, ... seem to indicate - although it's still important.
2. If those cheats are possible, it's because 3DMark isn't optimized enough. It would be more representative of real games if you had a version of 3DMark with sky rendered after everything, for example.
Any serious game isn't going to render it before everything else. Saying it's supposed to stress the GPU makes no sense, because this is supposed to be a GAME test, not a stress test!
Of course, nVidia is still guilty for acting like that - the best thing they could have done is try to insist to get 3DMark to publish an update with those optimizations for all cards. And things like the "no Color clear" should have never happened.
Okay, I hope my POV is clearer now...
Uttar
Well, I knew some people weren't gonna like this post when I typed it. But anyway...
First, how is this excessively complicated? I believe you're significantly overestimating the work which has gone into this.
Doing it by hand is *impossible*. You'd be insane to do that. However, simply writing a driver revision writing some test results to a file, and then another driver able to interpret that data, should be quite easy. I think many people think nVidia wasted a lot of time on this. I don't think so, more like several days I guess.
Furthermore, nVidia might see it as a "long term investment": now that they've written this app, which isn't THAT hard to program, they might be able to apply it to any other benchmark.
And yes, that part is very ugly, and I'd get quite disgusted at them if they began doing that. So if they're busted in other benchmarks for similar things, I can promise you I won't have the same attitude anymore. I mean, if they used it in all timedemos, but couldn't figure out a way to do anything similar in real games.
Alos, as I said, I'm against cheating. I think it'd be better if those companies didn't cheat. But my point is that this issue is being exagerated. When I first heard that some sites were gonna expose it all soon, I supposed it was gonna be a lot more nasty than this. I guess I've got a subjective issue on this whole story for this very reason: I thought it was gonna be MUCH worse.
Also, while the current cheating methods are special-purpose, the DIP caching I propose can easily be applied to other things even without a very general system. That means using it for any game which use illogical draw orders for things like the sky should be rather easy, although it's still special purpose code: but if you use special purpose code everywhere, then it isn't as annoying as if you use special purpose code in only one game/benchmark to lie to your customers.
Anyway, to summarize, sorry if this seemed like I'm accepting that nVidia cheats. I'm not. But what I'm saying is:
1. This isn't as big of a deal as the forums, news sites, ... seem to indicate - although it's still important.
2. If those cheats are possible, it's because 3DMark isn't optimized enough. It would be more representative of real games if you had a version of 3DMark with sky rendered after everything, for example.
Any serious game isn't going to render it before everything else. Saying it's supposed to stress the GPU makes no sense, because this is supposed to be a GAME test, not a stress test!
Of course, nVidia is still guilty for acting like that - the best thing they could have done is try to insist to get 3DMark to publish an update with those optimizations for all cards. And things like the "no Color clear" should have never happened.
Okay, I hope my POV is clearer now...
Uttar
GraphixViolence
Newcomer
When multi-million $ OEM deals are at stake, nothing is impossible. You could put a dozen engineers on the project for several months, and it would probably still be worth it if you got one major design win as a result.Uttar said:
I think it's telling that in ExtremeTech's Gamegauge tests, which use a wide variety of games that are not typically used as benchmarks, the DetonatorFX drivers provide little or no benefit. 3DMark03, however, gets a big boost.
Now ask yourself:
a) Which of these apps make more sense for Nvidia to optimize from a financial perspective?
b) Which of these apps are easier to optimize for (due to a fixed flight path)?
c) Which optimizations would provide more benefit to users of the product?
and finally...
d) Do you still think this isn't a big deal?
Hellbinder
Banned
I will just say this. If i were a review site. I would find a way pronto to test a large group of fixed timedemo benchmarks. Especially ones like Codecreatures. Where Nvidia has a significant lead, yet their synthetic shader performance is way down.
things id test.
1. Codecreatures
2. Serious Sam Timedemos. (Id really look at this one)
3. Q3 timedemo 1-3
Im sure there are others, but these are the ones that have alwyas made me scratch my head. After all in SS a GF4 ti 4600 miraculously scores the same as a 9700pro with 4xaa +8xAF. This has bugged me for the longest time.
things id test.
1. Codecreatures
2. Serious Sam Timedemos. (Id really look at this one)
3. Q3 timedemo 1-3
Im sure there are others, but these are the ones that have alwyas made me scratch my head. After all in SS a GF4 ti 4600 miraculously scores the same as a 9700pro with 4xaa +8xAF. This has bugged me for the longest time.
Hellbinder
Banned
Um, How did extremetech and B3D find out about this Clip plane issue? Was this already addressed somewhere and i missed it?
Or do you guys always make a habbit out of freezing timedemos and flying around in them? Who tipped you all off that this was happening?
Or do you guys always make a habbit out of freezing timedemos and flying around in them? Who tipped you all off that this was happening?
It is a big f**king deal, whether ET came up with this article or not, or whether forums heat it up. It challenges every preconceptions we may have had about reading and trusting reviews.Uttar said:
This really isn't about how X is rendered before Y or whther a game is following an optimized route. It's about what a IHV can do wrt a benchmark with a fixed camera mode as the basis of its benchmark. You don't appear to grasp the gravity of the situation.
Doomtrooper
Veteran
Richthofen said:
Hmm Doom 3 comes to mind, S.T.A.L.K.E.R...stencil shadows and Pixel Shader 2.0.
You don't seem to get it, it doesn't matter what benchmark is used, Synthetic or Quake 3, IHV's will optimized the crap out of their drivers to get sales from OEMs.
So does heavy optimizing in a single game engine make it better for the end user, with visual anomolies etc...I hope not
The latest patch can force two year old PS 1.1 to stop that comment
Well I do, there is something called ethics that alot of online reviewers lack. There is games shipped that state they require a certain '3Dmark', it s by far the most popular benchmark ever, and should not be used to 'mislead' potential customers.
OK, it's obvious that some deception is occurring here. The question is, who (directly) is to blame? I'm hearing lots of of "Nvidia is cheating", "Nvidia is deliberately deceiving us". And of course the usual suspects are holding this incident up as proof that Nvidia as a whole is somehow ethically bankrupt.
How many employees in the Nvidia driver-writing group? How many levels in the organization below CEO are the driver writers? How large is the driver-writing group in relation to the whole company? Think Jen-Hsun Huang is directing driver writers in their tasks? I'd bet that he has other things to worry about, like running a Fortune 500 company.
Is it possible that a low-level lead engineer in the driver writing group directed his team to get the benchmark scores up, and it became a minor side-project of a lone driver writer? Is it possible that noone else was aware of the cheats he implemented?
Every past attempt to cheat at benchmarks has been uncovered and has blown up into a media fiasco. Surely the decision-makers at Nvidia know this and are not dumb enough to encourage this sort of behavior. The idea that driver cheating is a some sort of "corporate policy" endorsed by Nvidia is absurd (or Nvidia leadership is just fantastically stupid).
Again, it's clear that the Nvidia drivers have been manipulated to unfairly optimize for a benchmark. I'm just saying that there's no need to prosecute an entire organization based on what could be the actions of a very few individuals.
How many employees in the Nvidia driver-writing group? How many levels in the organization below CEO are the driver writers? How large is the driver-writing group in relation to the whole company? Think Jen-Hsun Huang is directing driver writers in their tasks? I'd bet that he has other things to worry about, like running a Fortune 500 company.
Is it possible that a low-level lead engineer in the driver writing group directed his team to get the benchmark scores up, and it became a minor side-project of a lone driver writer? Is it possible that noone else was aware of the cheats he implemented?
Every past attempt to cheat at benchmarks has been uncovered and has blown up into a media fiasco. Surely the decision-makers at Nvidia know this and are not dumb enough to encourage this sort of behavior. The idea that driver cheating is a some sort of "corporate policy" endorsed by Nvidia is absurd (or Nvidia leadership is just fantastically stupid).
Again, it's clear that the Nvidia drivers have been manipulated to unfairly optimize for a benchmark. I'm just saying that there's no need to prosecute an entire organization based on what could be the actions of a very few individuals.
YeuEmMaiMai
Regular
we are all getting screwed over on this one.
For all of you crying about ATi and the Quack thing all the time claiming that nVidia would NEVER cheat,I got one question for you, How does that foot in your mouth taste?
For all of you crying about ATi and the Quack thing all the time claiming that nVidia would NEVER cheat,I got one question for you, How does that foot in your mouth taste?
Hellbinder
Banned
SteveG..
Having said that ATi in this case did the same kind of thing with Quake a couple years ago. Thus for me personally i am not going to blow this out of perportion. I simply think some research should be done to verify the integrity of all the major Bencharking applications and demos.
My good man they ARE ethically bankrupt and have been for a couple years now.
Having said that ATi in this case did the same kind of thing with Quake a couple years ago. Thus for me personally i am not going to blow this out of perportion. I simply think some research should be done to verify the integrity of all the major Bencharking applications and demos.
I think we should just ditch 3DMark as a benchmark. It's just a glitzy way of showing off what your shiny new card can do, and how good the driver teams are at optimising for it. Nothing more and nothing less.
It allows flagrant abuse to occur and if it is going to come back as a serious application then some more thought needs to go into how it actually performs and scores it's benchmarks.
It allows flagrant abuse to occur and if it is going to come back as a serious application then some more thought needs to go into how it actually performs and scores it's benchmarks.
Yes, a lot of games might skip clearing the color buffer, but this decision should be left up to the game. There might only be a few scenes that require clearing and it is nearly impossible for the driver optimizers to know this.Ostsol said:I don't really think that not clearing the colour buffer is really critical. . . UT by default doesn't clear it, at least in OpenGL. That was one of the problems I encountered while trying to create a "wallhack" cheat (never actually used online).
Similar threads
