ExtremeTech Article Up

Nods. But i think we need SS:SE corrupted image to support this.
If we find it only on timedemo, there'd be no escape for nvidia.
But if it happens on real games not only on timedemo, it's probably a bug.
(though not necessarily a bug for it might have both bugs and cheats)
And i'm not sure nvidia would mind to cheat on old games like SS I.
 
binmaze said:
Nods. But i think we need SS:SE corrupted image to support this.

We won't see any corrupted SS SE screenshots, the point is:

a) Det 44.03 showed big performance increase in SS SE.
b) SS (non-SE) that uses the same engine as SS SE has serious clipping issues with Det 44.03.

Possible (likely?) explanation is that the driver believes it runs in benchmark mode when playing SS (non-SE), thus using hardcoded clippingplanes.
 
Tim said:
Possible (likely?) explanation is that the driver believes it runs in benchmark mode when playing SS (non-SE), thus using hardcoded clippingplanes.
Quite possible. But it's also possible that it's just due to a bug.
I think we need 1 more evidence. Then it'd be 100% apparent.
Is there no way to cut-in the time-demo and move around?
 
The problem is, with the similar sky effects popped on real gaming, nvidia can claim the whole cheat matter is a bug.
For it happens not only on bench demo but on real gaming.
 
Rev,

I for one would definately approve of B3D running their own timedemo's in order to ensure that this sort of thing doesn't happen. You were talking about it being a problem in not being able to compare results across websites, however, I don't see this being a problem, AS LONG AS you compare the product you are reviewing to the competitor's product. This would be to ensure that, at the end of the day, the reader could determine what is the best for them to buy. I mean, this is the major reason that we read reviews in the first place isn't it? I know that most of us have the most belief in B3D reviews, so who needs to go to another website anyway? :D

If you were worried about people saying you were biased, would it be possible to then share the timedemo's with a couple of TRUSTWORTHY people from other websites, and get them to run it in their reviews? I know that logistically this would be difficult, but if you really wanted to do it......

Sorry if this has already been said, but I haven't got time to read through all of the posts.
 
demalion said:
I think nVidia has succeeded in stigmatizing 3dmark for many people. The mindset currently encouraged is "anything but 3dmark" and the technical validity of that propostion seems to not matter at all.

I've never been a big fan of static benchmarks. But it strikes me that in this case nVidia has done far more to stigmatize nVidia than to stigmatize 3DMark. nVidia's official disdain for 3DMark 03, as expressed when it voluntarily resigned from the 3D Mark program last December, was, paprphrased:

"We don't think 3DMark 03 accurately portrays the state of 3D games of the future."

OK, 3dMark is a DX9 benchmark. It's not clear what nVidia is saying here--is it saying it doesn't think there will be any DX9 games written in the future? 3DMark 03 is *just* a DX9 benchmark. That's all it is. It's one of many, many benchmarks--and a very popular one with a lot of people. It would be interesting for nVidia to compile a list of the benchmarks which it thinks *do* reflect the state of future gaming....*chuckle* (I would imagine their recent Doom III demo would be one of which they would heartily approve...*snicker*)

But all things considered '03 really is just a benchmark. Nothing else. There's just no reason for nVidia to feel so threatened by it. Nobody else is, right?

Someone brought up an interesting idea somewhere in all of these threads: he said he thought nVidia was getting scared that eventually its 3D Mark "optimizations" were going to be exposed, so they decided to get out before the dross hit the fan. That idea seems eminently plausible to me.

As I mentioned earlier in all of this, I also think nVidia is tiring of the "universal API" approach to doing things and wants more power in shaping the future of the industry--all the power, most likely...Heh-heh...which is going to be hard for nVidia as long as the major API in use, D3d, remains controlled by M$. IE, I don't think nVidia can be satisfied anymore with providing "input" into the formulation of the API--I think it wants to control it, and that this is at the bottom of everything we're seeing related to DX9 and nVidia. Apparently nVidia is not confident in its ability to remain numero uno on the level playing field of a universal API and wants to branch off and go its own way hoping everyone else will attempt to follow. *chuckle* After R3xx, can you really blame them? They see the writing on the wall.
 
I think it may be prudent to attempt to take a look at the UT2003 Antalus and CTF-Face3 demos as well, if it's even possible to do so. Not to say that they've been 'optimized' but since the two show a marked improvement over the 9800 scores, it'd definately be good idea to verify the scores. Especially if it turns out that SS:SE was cheated with. If both were to prove to have been subject to anything similar to what's going on with 3dMark, the 5900 would basically have its performance values knocked out from under it.
 
Hellbinder[CE said:
]Nvidias Cheating in 3dmark made TechTV. Now thats bigtime.. Guess pretty much everyone knows now. :LOL:

The beginning of the end?

I wonder if this could technically be labled as intent to defraud customers... :devilish:
 
Doomtrooper said:
MuFu said:
ssclip.jpg


MuFu.

sam3-a.gif

How does this support the cheating contention?

The 3DMark problem was suspicious precisely because people thought that 3dMark was the only instance when the clipping problem occured, thus making the clipping problem an unlikely bug. Finding this problem in other D3D games would indicate that this error is not unique to 3DMark and hence much more likely to be a bug with overly agressive optimizations, just as Nvidia initially speculated.

The way to find out whether that's true is to answer two questions.

Does this problem only occur during timedemo?

Is this problem only restricted to games commonly used in benchmarking?
 
boobs said:
How does this support the cheating contention?

The 3DMark problem was suspicious precisely because people thought that 3dMark was the only instance when the clipping problem occured, thus making the clipping problem an unlikely bug. Finding this problem in other D3D games would indicate that this error is not unique to 3DMark and hence much more likely to be a bug with overly agressive optimizations, just as Nvidia initially speculated.

The way to find out whether that's true is to answer two questions.

Does this problem only occur during timedemo?

Is this problem only restricted to games commonly used in benchmarking?

If I understand what's going on properly with the rendering issues it's not so much that the problem is unique to 3dMark as that the rendering issue only occurs if you're not in the predicited pathline for the camera.

In otherwords, to the end user nothing would really seem to be wrong, but if you pause the timedemo and step off the camera's 'rail' using tools only available to beta members the screen messes up. It's just way too strange for the bug to only occur outside of the normally rendered frames.

[sarcasm]If overly aggressive optimizations are responsible for these problems then the DetFX drivers have some out of this world predictive algorithms as to where the next frame is going to be.[/sarcasm]

The thought with SS is that the DetFX drivers are misdetecting SS1 as the SS:SE timedemo, which is one thing that could cause the problems that are being seen if cheating is going on with that time demo. It's somewhat odd that the same problems are not present in SS:SE in normal gameplay too, the engine is pretty much the same isn't it?
 
Eolirin said:
boobs said:
How does this support the cheating contention?

The 3DMark problem was suspicious precisely because people thought that 3dMark was the only instance when the clipping problem occured, thus making the clipping problem an unlikely bug. Finding this problem in other D3D games would indicate that this error is not unique to 3DMark and hence much more likely to be a bug with overly agressive optimizations, just as Nvidia initially speculated.

The way to find out whether that's true is to answer two questions.

Does this problem only occur during timedemo?

Is this problem only restricted to games commonly used in benchmarking?

If I understand what's going on properly with the rendering issues it's not so much that the problem is unique to 3dMark as that the rendering issue only occurs if you're not in the predicited pathline for the camera.

In otherwords, to the end user nothing would really seem to be wrong, but if you pause the timedemo and step off the camera's 'rail' using tools only available to beta members the screen messes up. It's just way too strange for the bug to only occur outside of the normally rendered frames.

[sarcasm]If overly aggressive optimizations are responsible for these problems then the DetFX drivers have some out of this world predictive algorithms as to where the next frame is going to be.[/sarcasm]

The thought with SS is that the DetFX drivers are misdetecting SS1 as the SS:SE timedemo, which is one thing that could cause the problems that are being seen if cheating is going on with that time demo. It's somewhat odd that the same problems are not present in SS:SE in normal gameplay too, the engine is pretty much the same isn't it?

Don't know. Somebody needs to sit down and run through a bunch of games with their own recorded demos, default demos, running around in the game, etc.

Rigorous experimentation is much more useful than speculation. ;)
 
Rev. MaximumPC magazine has been using a custom Quake 3 timedemo for years. I think if each site uses their own timedemo it wouldn't need to change often because people would notice if performance improved at B3D, but not at Anand for example.
 
3dcgi said:
Rev. MaximumPC magazine has been using a custom Quake 3 timedemo for years. I think if each site uses their own timedemo it wouldn't need to change often because people would notice if performance improved at B3D, but not at Anand for example.

If we could assume that sites had a fairly overlapping and equal distribution of viewers yeah... However just messing with Tom's and Anand's could probably cause a considerable number of people to become affected by the scewed benches. People who wouldn't cross reference properly with other sites. Even then there aren't that many high profile review sites that it wouldn't be fairly easy to mess around with the time demos from all of them if you only had to do it once.
 
boobs said:
How does this support the cheating contention?

Finding this problem in other D3D games would indicate that this error is not unique to 3DMark and hence much more likely to be a bug with overly agressive optimizations, just as Nvidia initially speculated.

Well, if that shot is of Serious Sam (the first one) and it is getting the same thing, then since it is running in OpenGL (no D3D option in the first one), then you have two API's and code bits with this same "bug". Don't know how NVIDIA manages their drivers, but getting the same oops twice in two separate pieces of code would be more in support of something deliberate than accidental.

[Edit] Just saw the SE in the pic, oops. :) Well, was it run in D3D mode or OpenGL mode?

It's either a cheat or crappy drivers, but since it's been proven that NVIDIA's drivers are always perfect, then it must be a cheat. :)
 
Eolirin said:
I think it may be prudent to attempt to take a look at the UT2003 Antalus and CTF-Face3 demos as well, if it's even possible to do so. Not to say that they've been 'optimized' but since the two show a marked improvement over the 9800 scores, it'd definately be good idea to verify the scores. Especially if it turns out that SS:SE was cheated with. If both were to prove to have been subject to anything similar to what's going on with 3dMark, the 5900 would basically have its performance values knocked out from under it.

I think you can change the viewpoint freely when running UT2003 benchmarks so they'd be pretty stupid to software cull for the rail.

MuFu.
 
covermye said:
I have no problem at all with this, Rev, for the reasons others have stated above. Nobody questions your bias.
That is wrong. There a few individuals who participates at Beyond3D's forums as well as elsewhwre that thinks I'm NVIDIA-biased simply because I used to interview more NVIDIA personnels than any other IHVs and review a lot of NVIDIA-based products. :rolleyes:

In any case, I have received a couple of emails from Kyle about this whole thing and I will risk Kyle's anger at my posting his private email (as well as my replies) to me for the public to read. You should read bottom to top as this is cut-and-pasted ad verbatim :

Anthony Tan wrote:


> I'd like to tell the whole world how pissed I am at some sites for the
> kind of things they practice but I won't. I am above mud slinging. There
> is an expression of opinion in a responsible manner and there is
> mudslinging. I always choose the latter.


Whoops, I meant the "former" :)

--
Anthony

============================

Kyle Bennett wrote:

> I realize the implications of the issue Anthony, AND I will still stick to
> my opinion. I think you miss the point. It may well already be >happening.
> I would rather see your time and effort put into helping lobby for better
> benchmarks and tools than talking about something you will have not >control over in the end.

Like I stated, I think it is perfectly fine for you to state your opinion (and continue to stand by it regardless of the crap you get). However, I think you know the pull you/[H] has and I think you may perhaps have worded your opinion regarding ET a little bit better and responsibly. I have stated in the past that while websites are in competition with each other, this doesn't mean we should be slinging mud at each other (it is precisely what you did). We need to act a little more responsibly. We should stick to talking about facts instead of rumors and speculations, regardless of who the source of such rumors/speculations may be.

And I have put my time into thinking about how best to tackle the situation, as evidenced by my post as per the link I gave you below.

> Do you honestly think I am so dense that I do not see the implications > of
> this? I would argue that you are focusing a bit too much on the facts >and
> not taking time to look at the big picture.....you have no control over >what
> they do, but we can have control over the tools we use. That of course
> makes the job harder, but I suspect that those that can't handle that >should
> not be sharing their work with others.


Actually, when I read your opinion on your front page (I did not go around looking for whatever forum posts you may have made to validate/back up your opinion), I must admit I was incredulous that a seasoned guy like you would say what you did. But it would've helped if you had stated what you stated below on your front page in more bold terms than the accusations you threw at ET. Why on earth did you do that, dude? You *must* know what that would lead to. If you are concerned about improving things, then cut out the accusations and just say what you said below.

I don't know you all that well, so forgive me for thinking you're "dense" (I don't) but your opinions on your front page did not lead me to believe you don't see the big picture as I expressed them below. Perhaps that would've been a better post for you to make instead of the accusations you threw at ET. Regardless of whether you do see the big picture or not, people will harp on you because you threw the accusations. I don't like throwing mud at another website even though I may know for a fact that they're doing something with ulterior motives. That's not my way. I don't like shitting on other sites. I stick to disproving things.

I act responsibly by providing my own findings instead of silly little accusations with no fact.

> We do everything we do for a reason, and it is seldom to things to directly
> drive site hits. We have all of those we could want. I assure you that my motives
> in this situation are very pure even if you can't stand back and see the overall situation.
>I would ask that you think a bit longer
>and a bit harder before you paint me with such a wide and ignorant >brush.


Like I said, given the way you worded your post, what else am I supposed to think? It's not my being ignorant... it's an expression of my reaction based on your post... because you have offered nothing concrete to back up your accusations... all you did was accuse a website of ulterior motives when the same can be said based on your priveleged status of being offered by NV to bench Doom3. Tit for tat, so to speak.

If I painted you with a wide ignorant brush, the same can be said of the way you painted ET.

Kyle, I'm not here to make enemies with you. On the contrary, for ages I have tried to get websites to work together for the good of the consumers. I just wished you had acted/posted with a litlle more responsibility, especially with the kind of weight your website carries.

Mud slinging is immature/irresponsible/unprofessional action. Rebuttals backed up with facts are mature/responsible/professional.

I'd like to tell the whole world how pissed I am at some sites for the kind of things they practice but I won't. I am above mud slinging. There is an expression of opinion in a responsible manner and there is mudslinging. I always choose the latter.

Peace bro.

--
Anthony

=====================================
_____________________________

-----Original Message-----
From: Anthony Tan [mailto:reverend@beyond3d.com]
Sent: Friday, May 16, 2003 4:31 AM
To: Kyle Bennett
Subject: Re: Hi Kyle... Reverend here


Kyle,

I'd wanted to add a "point" in that post of mine whereby I thought I
felt you have every right to state your opinion on your own website but
given the number of times I edited and re-edited my post, it somehow got
forgotten [:)] I do not, however, agree with the things you said but you
have every right to express your opinion on your website.

This is actually more serious than you think, Kyle.

Forget that this is NVIDIA and forget that this is happening in
3DMark03. Here's a later post I made in that thread, which should
explain why I think this is a serious issue with wide-reaching
implications :

http://www.beyond3d.com/forum/viewtopic.php?p=116656#116656

The point is that if NVIDIA can do this with a synthetic benchmark,
*any* IHV can do the same with game recorded demos that is available to
all. And this means so much more extra work for a reviewer in terms of
verifying game demo benchmark results is the same as actual game playing
results *regardless of what hardware is being reviewed*, be it NVIDIA
video cards or ATi video cards. Do you understand what I'm getting at?
Again, forget that this is NVIDIA and 3DMark03 -- the two are the
unfortunate debut examples.

--
Anthony Tan
"Reverend"
Beyond3D : http://www.beyond3d.com

"The brighter future has always been about how many things can be done,
not how fast such things can be done. Competition, though, always result
in the exact opposite."

==================================

Kyle Bennett wrote:


Hehe, just read that thread in the B3D forums.....great stuff. Too bad
those guys would not stop talking about me and focus on the damn issue.

Anyway, I wanted to address you post in that thread.

I am not pissed at anyone or any situation. I just have my opinion be it
right or wrong. Actually, in this case, I really don't care as 3Dmark is
worthless anyway. I actually find all of this rather funny. [:)] Laterz
bro.

_____________________________

Kyle Bennett
Editor-in-Chief @ HardOCP.com
Owner @ Ratpadz.com
_____________________________
 
Back
Top