Cheating and its implications
- Thread starter Humus
- Start date
- Status
The real implication here is that people are going to buy cards like the 5600 and the 5200 based on special effects (FX) benchmarks and find their games run like slideshows.
OEM's might look at the benchmarks and select Nvidia over ATI because of their rankings.
Cheating without getting caught can mean big $$$.
OEM's might look at the benchmarks and select Nvidia over ATI because of their rankings.
Cheating without getting caught can mean big $$$.
I see there is no response to my ideas? Bleh. No wonder the cheating is going to continue. Nobody wants to do any of the broad changes like the ones I suggested. Until something drastic changes we're going to continue seeing NVIDIA and others cheat.
Anyway, all this talk about cheating and WHQL reminded me of some testing that I did back in 1997 for Jon Peddie as part of some litigation against companies using the "GDI Bypass" hack/cheat. I had to purchase about $3000 worth of graphics boards and test every one of them for the "GDI Bypass". The tool I used was very small and basically gave a yes or no answer. I'm not sure who developed the program, but it worked very well and I was surprised to see how many board manufacturers were using cheat in their drivers. The connection to WHQL was that Microsoft was outraged with this hack and started testing for it in its WHQL certification process. They wouldn't give any WHQL certification to any driver that used it. So that at least shows that Microsoft is concerned with cheating, but only if it affects Windows.
I doubt they would want to police cheating in benchmarking, but if cheating is being done in the driver you would think they wouldn't allow such drivers to at least get certification. And as Doomtrooper explained this is going to be even more of an issue with the release of Longhorn. Provided they will require WHQL certified drivers for running the OS.
Tommy McClain
Anyway, all this talk about cheating and WHQL reminded me of some testing that I did back in 1997 for Jon Peddie as part of some litigation against companies using the "GDI Bypass" hack/cheat. I had to purchase about $3000 worth of graphics boards and test every one of them for the "GDI Bypass". The tool I used was very small and basically gave a yes or no answer. I'm not sure who developed the program, but it worked very well and I was surprised to see how many board manufacturers were using cheat in their drivers. The connection to WHQL was that Microsoft was outraged with this hack and started testing for it in its WHQL certification process. They wouldn't give any WHQL certification to any driver that used it. So that at least shows that Microsoft is concerned with cheating, but only if it affects Windows.
I doubt they would want to police cheating in benchmarking, but if cheating is being done in the driver you would think they wouldn't allow such drivers to at least get certification. And as Doomtrooper explained this is going to be even more of an issue with the release of Longhorn. Provided they will require WHQL certified drivers for running the OS.
Tommy McClain
YeuEmMaiMai
Regular
I live here in the USA after all I was born here. Here are some typical examples of American companies
MCI WORLD COM, AMERICAN AIRLINES, CHRYSLER, ENRON, AOL TIME WERNER, GE, etc. those are just a few of the companies that lied about their debt and tried to make it look like a profit. how about CitiGroup or Merril Lynch? Those guys were cought lying to investors so that they could dump crap stock on them.
How about S3 Inc they craped on everyone with the S2000 and it's hardware flaws just so they could make a buck for the HOLIDAY season
Do you actually know anything about the automotive industry of the late 60 through the early 80's The have internal memos from all 3 automakers stating that the japanese were not a threat and no matter what they made the american puplic would buy it.
How about the cigarette companies? They all lied their collective butts off about wether or not they knew smoking was bad and that they did not ever market towards kids..... just a quick question who owns Kraft foods? ANSWER PHILLIP MORRIS and now you know why a box of kraft mac and cheese cost $100 and the store brand that is just as good cost $.33
So why don't you actually do some research before you open your mouth and then come back and make an Intelligent statement.
MCI WORLD COM, AMERICAN AIRLINES, CHRYSLER, ENRON, AOL TIME WERNER, GE, etc. those are just a few of the companies that lied about their debt and tried to make it look like a profit. how about CitiGroup or Merril Lynch? Those guys were cought lying to investors so that they could dump crap stock on them.
How about S3 Inc they craped on everyone with the S2000 and it's hardware flaws just so they could make a buck for the HOLIDAY season
Do you actually know anything about the automotive industry of the late 60 through the early 80's The have internal memos from all 3 automakers stating that the japanese were not a threat and no matter what they made the american puplic would buy it.
How about the cigarette companies? They all lied their collective butts off about wether or not they knew smoking was bad and that they did not ever market towards kids..... just a quick question who owns Kraft foods? ANSWER PHILLIP MORRIS and now you know why a box of kraft mac and cheese cost $100 and the store brand that is just as good cost $.33
So why don't you actually do some research before you open your mouth and then come back and make an Intelligent statement.
micron said:
Joe DeFuria
Legend
Not to get too sidetracked, but:
No, Kraft Mac and Cheese costs a lot more than "store brand" mac and cheese, because consumers are willing to pay a higher price for the brand. It's called "capitalism." Capitalism is not based on the "actual" best product (if such a thing could be defined) commanding the highest price.
It's about supply and demand. The product that is perceived to be the best....for whatever reason, logical or illogical, will command a higher price due to higher demand.
YeuEmMaiMai said:
No, Kraft Mac and Cheese costs a lot more than "store brand" mac and cheese, because consumers are willing to pay a higher price for the brand. It's called "capitalism." Capitalism is not based on the "actual" best product (if such a thing could be defined) commanding the highest price.
It's about supply and demand. The product that is perceived to be the best....for whatever reason, logical or illogical, will command a higher price due to higher demand.
YeuEmMaiMai
Regular
and you definately can bank on the fact that Phillip morris is using the brand reconition to squeeze all of the $$ they can to pay for that lawsuit that they lost.... Do you honestly believe that they are just going to pay that out of their own pocket?
LOL that would definately be something new.....
I save a lot of $$$ by buying store brands over name brands in the good department.....
LOL that would definately be something new.....
I save a lot of $$$ by buying store brands over name brands in the good department.....
Joe DeFuria said:
Joe DeFuria
Legend
YeuEmMaiMai said:
What's to stop a driver from recognizing timedemo file formats, and having code to analyze them when they are loaded? The only thing that occurs to me is that it might be too difficult and bloat driver code too much, or the timedemo file format might take too long to load and analyze effectively to this end. This particular practice could already be well established, and the 3dmark issue would be nVidia taking it to an entirely new level of misrepresentation due to the lack of competitiveness of their hardware for floating point fragment processing.
Will applications that load timedemos need key encryption for timedemo files now ? Have some already taken this step? Can drivers be denied access to files effectively?
These are questions I'd expect someone has considered already, and aside from the obvious issue of what nVidia in particular is doing in this regard, I'm just curious if there are any answers to be shared along this line.
Perhaps the simplest test option available for now is to compare in game framerates and image quality to the timedemo, and for the 3dmark de-rail option to be offered more widely (something we've been asking for even before this was fully disclosed). People keep harping on 3dmark, not realizing that any other benchmark tool is only (perhaps) more protected because they are less universally recognized and therefore (perhaps) not targetted yet...Futuremark atleast is actively focusing on hindering this type of thing and has so far achieved some success, and following nVidia's party line is just a way of circumventing that.
Will applications that load timedemos need key encryption for timedemo files now ? Have some already taken this step? Can drivers be denied access to files effectively?
These are questions I'd expect someone has considered already, and aside from the obvious issue of what nVidia in particular is doing in this regard, I'm just curious if there are any answers to be shared along this line.
Perhaps the simplest test option available for now is to compare in game framerates and image quality to the timedemo, and for the 3dmark de-rail option to be offered more widely (something we've been asking for even before this was fully disclosed). People keep harping on 3dmark, not realizing that any other benchmark tool is only (perhaps) more protected because they are less universally recognized and therefore (perhaps) not targetted yet...Futuremark atleast is actively focusing on hindering this type of thing and has so far achieved some success, and following nVidia's party line is just a way of circumventing that.
I downloaded the Cat3.4s yesterday. A total of 20 meg, (compressed, I assume.)demalion said:
EDIT: I went back and checked the NVIDIA drivers on their site. They're ~18 meg also. Does that make you f'n happy that I mentioned both companies?
You could easily slip all sorts of analysers into something that big without being noticed.
Yes, but isn't that multi-language, etc (in the windows tradition, localization seems to not just be the text data, but entire libraries and executable code...lovely redundancy)? My download was about 7 MB.
Anyways, the actual driver disk space occupation before setup is 8.12 MB for me. What you'd need to measure to get an actual start on this (though still be a far way from finishing) is memory footprint, and then establish a rough idea of the ballpark expectation given various concerns.
For example, nVidia has been the developer standard for a while...all other drivers (especially the Kyro if it is still trying to compete with as many recent games) would have reason for more work-around bloat. This would presumably shift to ATI adding less new bloat after the first DX 9 level games are introduced.
It is adding more on top of this, as well as monitoring and analysis workload during game execution, that I am thinking about with bloat. What I'm primarily interested is analysis related to isolating that and the other things I mentioned.
Anyways, the actual driver disk space occupation before setup is 8.12 MB for me. What you'd need to measure to get an actual start on this (though still be a far way from finishing) is memory footprint, and then establish a rough idea of the ballpark expectation given various concerns.
For example, nVidia has been the developer standard for a while...all other drivers (especially the Kyro if it is still trying to compete with as many recent games) would have reason for more work-around bloat. This would presumably shift to ATI adding less new bloat after the first DX 9 level games are introduced.
It is adding more on top of this, as well as monitoring and analysis workload during game execution, that I am thinking about with bloat. What I'm primarily interested is analysis related to isolating that and the other things I mentioned.
You can get around pre analysis by playing a game while writing the FPS, # of enemies on screen, amount of polygons, shaders being used, sound, amount of network traffic being passed, et cetera once every X frames. Afterwards, you run statistics correlating the amount of stuff going on with FPS, and use that as a measure of performance.
No way for a driver to preanalyze the timedemo file since you have a person providing controlling input. If the person plays long enough in similar enough conditions, statistics should be valid for comparing one card to another and one site's analysis to another's.
I don't think you'll find very many developers willing to devote time to doing this though, and the resulting data would require some understanding of statistical data analysis to interepret.
No way for a driver to preanalyze the timedemo file since you have a person providing controlling input. If the person plays long enough in similar enough conditions, statistics should be valid for comparing one card to another and one site's analysis to another's.
I don't think you'll find very many developers willing to devote time to doing this though, and the resulting data would require some understanding of statistical data analysis to interepret.
boobs,
Regarding your commentary, I direct you to do a search on the handle "vogel" and look up some Unreal Tournament discussions, or run the UT2k3 demo/retail game, if you have them, and load the cvs files into a spreadsheet program.
You'll note a significant parallel to your suggestion, I believe.
I believe Splinter Cell (also based on the Unreal Engine) has implemented similar functionality, and I think this remains a strong suite of the basic engine (UT2k3 doesn't use shaders extensively itself, but the engine seems to allow them, as shown by SC).
Regarding your commentary, I direct you to do a search on the handle "vogel" and look up some Unreal Tournament discussions, or run the UT2k3 demo/retail game, if you have them, and load the cvs files into a spreadsheet program.
You'll note a significant parallel to your suggestion, I believe.
I believe Splinter Cell (also based on the Unreal Engine) has implemented similar functionality, and I think this remains a strong suite of the basic engine (UT2k3 doesn't use shaders extensively itself, but the engine seems to allow them, as shown by SC).
demalion said:
That's interesting. Thanks for the info.
Now that I think about it, there is a good reason for developers to put this functionality in since it helps them optimize code.
Logistically speaking, doing this type of analysis is feasible for in depth reviews, but people like Anand and [H], who place an emphasis on getting reviews and previews out first, probably wouldn't have the time to do this.
Maybe B3D can blaze the trail in this area? 8)
OpenGL guy
Veteran
You caught us!!!! OMG! Call the FBI! The CIA! The Boy Scouts!RussSchultz said:
Why don't you try looking at what's contained in the installer before spouting crap.
OpenGL guy said:
I caught you doing what? Making large installers? Don't be so defensive.
RussSchultz said:
In all fairness, when I read your post my initial thought was, "That's awful FUDish sounding."
- Status
Similar threads
