View Full Version : Help me understand the AF/[H] controversy
Park the Shark
31-Jul-2003, 18:00
Dang it I tried to post earlier and it didn't go through.
Here's the situation as I understand it thus far:
1. nVidia release some drivers that are not doing true AF for UT2K3.
2. It has shown obviously noticeable IQ degradation in the HallOf Anubis add-on map.
3. It has not shown obviously noticeable IQ degradation for other maps.
4. It does show up in mip map highlights.
5. nVidia has promised to resolve the issue by providing the end-user control over AF.
6. Some people are VERY mad at [H] about this.
Now, for some people, this is difficult, but I personally haven't chosen sides in some sort of for/against [H] or B3D "war". It seems like many people have done so.
I just look at what I have been able to find, and make conclusions and ask questions from that. I guess I'm setting myself up to be attacked by BOTH sides, lol.
Two main things I'm curious about:
1. Has there been any map besides this add-on HallOfAnubis that shows decreased image quality from actual in game screenies? I have looked and haven't seen anyone posting them.
2. Why is everyone mad at [H]? It seems [H] ran their tests, showed that the actual in-game IQ is not affected noticeably, AND STILL went to nVidia expressing concerns over the lack of true AF. [H] reports that nVidia intends to resolve this.
It seems everyone (even nVidia themselves) agrees that nVidia should allow the end-user full control over graphics features/settings.
So what exactly is causing the big rift between B3D and [H] or at least their audiences?
If B3D, [H], and nVidia all agree that nVidia needs to change their drivers, what's left to argue about?
I just don't get it...
The thing is that [H] doesn't care about the IQ issue and just say that it's not noticeable when playing. But back at 2001 they raised big woowoo for the Quack thing. Double standards.
And it's on every map on UT2003. Nvidia applies the hacked filtering based on the executable name.
Bouncing Zabaglione Bros.
31-Jul-2003, 18:12
You really have to go back to last year when Nvidia promised all kinds of great things with NV30 (which kept getting delayed and delayed before finally being killed off) at the same time as ATI suddenly brought out a very competitive card.
You need to search for old threads on 3Dmark2003 and Nvidia cheating on it to get higher scores and badmouthing 3Dmark2003 after leaving Futuremark membership just before the new version got released, the subsequent response from Futuremark about the Nvidia cheats, and what various industry websites said (or didn't say about it).
There's loads of info, but it will be a lot of reading.
Park the Shark
31-Jul-2003, 18:29
The thing is that [H] doesn't care about the IQ issue and just say that it's not noticeable when playing. But back at 2001 they raised big woowoo for the Quack thing. Double standards.
First off--is there any IQ issue? Is [H] wrong when they say it's not noticeable when playing? This is why I asked if there is any map besides the HallOfAnubis (which is an add-on map) that shows IQ issues? I would think if there were, screenies would be plastered all over the place. I've looked for the screenies, and can't find them. So I'm beginning to think that either it doesn't affect in game IQ (HallOfAnubis being an exception, and being a 3rd party map, there may be other issues involved), or that the places it does present itself are too limited to notice.
Second... If [H] doesn't care about IQ, why would they do such a lengthy article at ATI's prompting? Also, why then would they ask nVidia to implement proper AF controls in the driver, DESPITE not seeing a noticeable visual difference?
Are there double standards on the Quack issue? I've seen both sides point to this to both bolster and refute their stances. On the one hand, we have the ATI people saying this is hypocritical treatment from [H]. On the other hand, we have nVidia people saying "ATI did it, so can we". Then we have people on the ATI side saying, "This is totally different, and not comparable because the ATI issue was a bug that was fixed and the IQ was repaired with no performance penalty." Yet we have people on the nVidia side saying, "This is totally different, nVidia is not denying the optimization and they aren't affecting image quality".
I can see some BASIC similarities: both companies did in fact optimize drivers for a specific application.
But from what I've seen, the ATI scenario did affect IQ much more drastically. Meanwhile, ATI denied the situation, despite being presented proof. So I think that [H] has treated the situatino differently because it IS DIFFERENT. They raised a "big woowoo" I think because ATI was lying about the optimization while hurting IQ. nVidia has handled it differently, and thus so have [H]. Is that not appropriate?
ATI did fix the "problem" and nVidia has reportedly promised to fix their "problem" as well.
I can see, though, where people come off saying [H] is hypocritical here, b/c part of their point with ATI was that Q3 was widely used as a benchmark. Now the same is true of UT2K3, and I think [H] hasn't tried to drive that same point home. However, it's somewhat different this time because nVidia is promising to fix the situation which will make the benchmarking "fair" again.
It's my opinion that benchmarking should be done in two modes, though. I would like to see straight benches that compare technique to technique (i.e. 4XAA/8XAF from both cards) as well as a comparison of benches at relatively equal IQ levels (which is much more subjective). For instance, from what I've seen, ATI AA is superior to nVidia AA at this point. Some say 2X ATI == 4X nVidia. I'd like to see benches and screenies that demonstrate this. The AF issue may fall into a similar category. We can compare nVidia optimized IQ to ATI (optimized) IQ when the IQ is comparable. I think this would have a lot of value to potential consumers when evaluating cards.
Park the Shark
31-Jul-2003, 18:30
And it's on every map on UT2003. Nvidia applies the hacked filtering based on the executable name.
Where are the screenies that show it in game? Looking at [H]'s mip map highlights, it's obviously affecting every map, but when you compare screenshot to screenshot, the difference is negligible.
The only map I've seen show a distinctive and obvious difference is the HallOfAnubis.
Dang it I tried to post earlier and it didn't go through.
Here's the situation as I understand it thus far:
1. nVidia release some drivers that are not doing true AF for UT2K3.
2. It has shown obviously noticeable IQ degradation in the HallOf Anubis add-on map.
3. It has not shown obviously noticeable IQ degradation for other maps.
4. It does show up in mip map highlights.
5. nVidia has promised to resolve the issue by providing the end-user control over AF.
6. Some people are VERY mad at [H] about this.
<snip>
You are confusing two different issues that currently reside with UT2003 and NVIDIA 44.03 drivers.
1.) The issue of the 44.03 not allowing the application, i.e., UT2003, to use full trilinear, but is rather applying their own mix of filtering that is currently between Bi/Trilinear filtering. While the ATI cards have an App Preference option that lets the app decide the filtering. The result of lesser Trilinear filtering can result in reduced transitions between mip-maps.
2.) The other issue is regarding Anisotropic filtering being enabled with the 44.03 drivers on the GFFX 5600. So far testing indicates it is restricted to that card specifically in one map specifically, CTF-HallOfAnubis which is a map that is not included with the game, but can be downloaded. This so far appears to be a bug.
Are there double standards on the Quack issue?
Most definitely.
I can see some BASIC similarities: both companies did in fact optimize drivers for a specific application.
Yes.
However, it's somewhat different this time because nVidia is promising to fix the situation which will make the benchmarking "fair" again.
It's been proven that [H] didn't even ask ATi about the thing before posting the woowoo.
The quack thing didn't affect all textures. Is there a noticeable IQ difference when playing?
Park the Shark
31-Jul-2003, 18:49
You really have to go back to last year when Nvidia promised all kinds of great things with NV30 (which kept getting delayed and delayed before finally being killed off) at the same time as ATI suddenly brought out a very competitive card.
You need to search for old threads on 3Dmark2003 and Nvidia cheating on it to get higher scores and badmouthing 3Dmark2003 after leaving Futuremark membership just before the new version got released, the subsequent response from Futuremark about the Nvidia cheats, and what various industry websites said (or didn't say about it).
There's loads of info, but it will be a lot of reading.
What does nv30 delays have to do with being mad at [H]? When they preview the nv30, they had this to say:
As we’ve seen, the GeForce FX is no slouch in the 3D accelerator world, however it is not the "9700-killer" many have expected. It is, at best, mildly faster in most games, and the same or slightly worse in a few. Had it arrived when most of us thought it should, there is no doubt it would be much better received.
However, its best performance was in 3dmark2001, a synthetic benchmark, and its worst in ut2003, an actual game compared to the 9700 Pro.
There was also commentary about how hot and loud it was. I don't think they were sugar coating the nv30.
RE: [H] on 3dMark2K3... They wrote an article about how worthless it was. It had nothing to do with "cheating" or optimizations. It had to do with 3DMark2K3 using mostly DX7 and DX8 games and calling it a DX9 benchmark. Particularly, this page:
http://www.hardocp.com/article.html?art=NDI4LDM=
I've never seen anyone disagree with the accuracy of the characterization of the DX tests there. In that regard, I can agree with [H[H] that the overall 3DMark score is worthless. I use it for stress testing, to test fillrate, and to make sure a card is performing comparably to others. For gaming purposes, it simply doesn't reflect much in the real world. Now, if [H] was truly nVidia biased, wouldn't they have been singing the virtues of 3DMark because nVidia's nv30 was the leader in 3DMark when [H] was blasting it.
Who honestly thinks 3DMark's overall score is a good measure of video card performance in real world applications or even future DX9 titles? Even Extremetech, who found nVidia was cheating, made comments about how 3DMark's game tests were done in strange ways that would not likely ever be seen in a real game title.
blah blah blah... What does this have to with the AF issue? I just don't see where you're going with this?
Now I know I'm starting to sound like an [H] defender, but I just don't think 3DMark is a good tool, and I don't think [H] was EVER singing the praises of the nv30. I honestly never cared if nVidia cheated at it or not, I didn't trust the scores ANYWAY! lol
The thing is that NV cheats at games too. LOL :roll:
Park the Shark
31-Jul-2003, 18:54
You are confusing two different issues that currently reside with UT2003 and NVIDIA 44.03 drivers.
1.) The issue of the 44.03 not allowing the application, i.e., UT2003, to use full trilinear, but is rather applying their own mix of filtering that is currently between Bi/Trilinear filtering. While the ATI cards have an App Preference option that lets the app decide the filtering. The result of lesser Trilinear filtering can result in reduced transitions between mip-maps.
2.) The other issue is regarding Anisotropic filtering being enabled with the 44.03 drivers on the GFFX 5600. So far testing indicates it is restricted to that card specifically in one map specifically, CTF-HallOfAnubis which is a map that is not included with the game, but can be downloaded. This so far appears to be a bug.
Thanks for clearing that up. So there is no AF controversy, that's just a bug? The bi/tri controversy remains, though...
The big question for me, then, is has ANYONE produced any screenshots showing the bi/tri issue as actually degrading in game image quality? (obviously, mip map highlights DO show a difference).
digitalwanderer
31-Jul-2003, 19:00
Now I know I'm starting to sound like an [H] defender, but I just don't think 3DMark is a good tool, and I don't think [H] was EVER singing the praises of the nv30. I honestly never cared if nVidia cheated at it or not, I didn't trust the scores ANYWAY! lol
That's the thing, a lot of people did and do think that nVidia cheating on 3dm2k3 is still wrong even if you don't happen to like the benchmark and that it was newsworthy enough that [H] should have covered it rather than just say, "Well, we don't like that benchmark so it doesn't matter". :roll:
A whole lot of people don't agree with that attitude, and Kyle has never directly addressed the issue of cheating. :(
Someone said in the [H] forums that Kyle was doing Nvidia a disservice by not bringing their "optimizations" to the forefront. He did it with ATI and ATI then improved dramatically(it's amazing what a little bad press can do). By not putting pressure on Nvidia he is giving them a green light to continually go down hill when it comes to their drivers.
I do give Kyle credit for at least having an opinion on the matter even though the vast consensus disagrees. At least its making people aware of the issues. There are far too many other sites that are so silent on the matter you could hear a pin drop. For instance, if you go to Anandtech forums, 99% know nothing about the latest UT2k3 saga. I don't believe anandtech has made an official statement on anything regarding nvidia drivers, the silence is deafening
Bouncing Zabaglione Bros.
31-Jul-2003, 19:12
What does nv30 delays have to do with being mad at [H]?
Because that is when the PR war that Nvidia has been waging started. That's when websites chose sides instead of being objective. That's when Nvidia decided to trash 3Mark2003 because they couldn't compete, and recruited websites to voice their PR documents. That's when cheating on benchmarks became a viable alternative to having the fastest part out there.
It's the direct source for why Nvidia is touting a huge increase in speed on one of the most used benchmarked programs (UT2K) by cheating on the filtering, and why sites like [H] are supporting them.
Park the Shark
31-Jul-2003, 19:14
It's been proven that [H] didn't even ask ATi about the thing before posting the woowoo.
The quack thing didn't affect all textures. Is there a noticeable IQ difference when playing?
So [H] was flat out lying when they said:
ATi engineers were asked last week if ATi drivers used any game specific instructions and we were told "No."
I don't know how many textures were affected by "Quack", but I have seen screenshots that show obvious differences. I would definitely say there is a noticeable IQ difference.
digitalwanderer
31-Jul-2003, 19:16
So [H] was flat out lying when they said:
ATi engineers were asked last week if ATi drivers used any game specific instructions and we were told "No."
According to ATi, yup. :)
Joe DeFuria
31-Jul-2003, 19:27
The big question for me, then, is has ANYONE produced any screenshots showing the bi/tri issue as actually degrading in game image quality? (obviously, mip map highlights DO show a difference).
That's only part of the issue. Image quality is indeed subjective, and what one person might notice, others may not. Though you have to ask, why would nVidia make this a UT specific setting, and not a global setting, if it really doesn't "impact" image quality?
The other part of the issue, is that you CAN get a similar quality level on ATI cards by setting (I believe) the texture slider lower.
So why isn't that done on ATI cards when comparing performance to nVidia ones?
digitalwanderer
31-Jul-2003, 19:30
The other part of the issue, is that you CAN get a similar quality level on ATI cards by setting (I believe) the texture slider lower.
So why isn't that done on ATI cards when comparing performance to nVidia ones?
"Psssssst, it's because then the 9800 would absolutely KILL the 5900 in all the reviews!", whispers the Dig quietly to Mr.DeFuria
;)
demalion
31-Jul-2003, 19:31
Dang it I tried to post earlier and it didn't go through.
Here's the situation as I understand it thus far:
1. nVidia release some drivers that are not doing true AF for UT2K3.
Actually, it's "not true trilinear filtering".
2. It has shown obviously noticeable IQ degradation in the HallOf Anubis add-on map.
Well, perhaps that is one place where it is evident even in selected still screenshots? In any case, trilinear filtering being absent is most evident in motion, not screenshots. Also, it doesn't show as readily on dark/dark detailed or low contrast surfaces (the darker a surface and its details, the less color brightness values available for contrast) and irregular surfaces (i.e., not flat, or "floor-like").
There are plenty of maps with "flat" surfaces in UT 2k3 (I played the "ball" maps a lot, and I know they are highly evident there), but there are also plenty of spots in almost every map that are irregular, or where the surfaces are dark/low contrast. Picking screen shots can make it look like a map is completely affected by the issue even if it is only the one spot picked for a screen shot where the issue showed up, as well as making it look like a map is completely unaffected by the issue even if it has the one spot you picked where the issue isn't evident.
Also, with increasingly detailed texture usage (as in UT2k3), using a still screenshot as "proof" of trilinear not being needed is a fallacy. That's backwards...seeing mip map transitions with the naked eye in a still screen shot is proof that it would be evident in (most types) of motion, all their absence proves is that they haven't risen above a certain threshold of image degradation.
3. It has not shown obviously noticeable IQ degradation for other maps.
No, I don't think that's right. More accurate to say that "[H] has not shown obviously noticeable still screenshot IQ degradation for the maps selected", or anyone involved with posting screenshots you may have looked at.
I've used bilinear/16xAF, and with boosted LOD (makes mip map transitions worse), and I can certainly like it when playing UT2k3, but that's different than saying that one card can arbitrarily switch trilinear off if the IHV wants to boost fps because the selected still screenshots don't show it. It's not like the nVidia cards can't do trilinear...nVidia decided to disable it for fps boosting.
4. It does show up in mip map highlights.
Yes, this is an analysis tool. It highlights the issue regardless of the significance for the textures used, as that's its purpose. Without being able to feasibly deliver pixel accurate (no encoding artifact) video of in game motion, this allows the issue to be made evident in still screenshots.
5. nVidia has promised to resolve the issue by providing the end-user control over AF.
Well, [H] indicates that they have, I haven't seen a specific statement from nVidia...again, I'm correcting your "AF" to mean "trilinear filtering". Note that nVidia's review guidelines already propose that the end-user has control over trilinear filtering ("Quality" mode), though through application detection as evidenced in UT2k3 that is not the whole truth. The whole truth about the change remains to be seen.
6. Some people are VERY mad at [H] about this.
Well, no, ATI is "concerned" with [H]'s review that this article addresses. I'd presume they are still "concerned", because the central issue of unfair fps comparison, as determined by nVidia's decisions for application detection alone, does not seem successfully addressed by the article.
Why "some people are VERY mad at [H]" are for a long list of reasons leading up to this article, all having the pattern of giving nVidia and their interests special treatment. Actions associated with this include dismissing ExtremeTech, Beyond3D, and other sites involved in the nVidia/3dmark 03 benchmarking image quality degradation issues as "police" or lackeys, including saying that the issue's exposure was "payback" for not having access to Doom 3 granted by nVidia (he's only expressed "regret" concerning ExtremeTech, which happens to be the particular site which is part of a large media group); attacks on Futuremark for nVidia's decisions to cheat, apparently based on a "technical article" that seems exclusively based on a 'technical' PR document nVidia circulated to several websites (though this was not indicated in the article, IIRC); and having thoroughly exposed the Quack issue, for it to be later found out that nVidia provided the info and tools (again, this was not indicated in the article...the source for them was listed as "friends").
This seems a stark contrast in IHV treatment, and "some people" find it cause for disappointment. Also, there is the issue of post deletions and banning of posters in this issue, and other issues, such as when faulty benchmark results were posted, changed, people at the site denied they were changed, and posters in the forums insisted and showed screenshots for comparison to the changed benchmark results.
Now, for some people, this is difficult, but I personally haven't chosen sides in some sort of for/against [H] or B3D "war". It seems like many people have done so.
Hmm? I don't get this B3D/[H] "war" thing. I have problems with specific actions of specific people at [H]...there are lots of forum posters at [H] I haven't met, so why would I want to "war" with them? B3D is just a place where people who are aware of these issues can discuss them, because the [H] forum's policy of banning, deletions, and restricted registering prevents it from being possible there. That policy is that tolerable disagreement is determined by Kyle, and the listed objective standards of conduct are arbitrarily dismissed as being necessary for the listed actions when it suits Kyle. Deleting posts serves to obscure the frequency of this issue.
I just look at what I have been able to find, and make conclusions and ask questions from that. I guess I'm setting myself up to be attacked by BOTH sides, lol.
There's a lot to look at.
Two main things I'm curious about:
1. Has there been any map besides this add-on HallOfAnubis that shows decreased image quality from actual in game screenies? I have looked and haven't seen anyone posting them.
See above.
2. Why is everyone mad at [H]? It seems [H] ran their tests, showed that the actual in-game IQ is not affected noticeably, AND STILL went to nVidia expressing concerns over the lack of true AF.
They also left a review standing that showed an nVidia card leading an ATI card in a mode that is represented as being equivalent. nVidia cards can do trilinear filtering, they do trilinear filtering in other apps, including those they recommend to show whether they are doing trilinear filtering or not, they do not do trilinear filtering in UT2k3, which happens to be an application they reported as having a performance boost in the driver set, and which happens to be often used for fps comparisons. In response to ATI apparently indicating they had a problem with this, this image quality article concluded that "it didn't matter anyways", and let the fps comparisons stand.
You see no problem in that sequence of events?
[H] reports that nVidia intends to resolve this.
Some people have issues with considering [H] and nVidia as trustworthy in their statements, a partial list of reasons listed above. Once the next batch of "fixes" (there have been many, including the one that put this UT2k3 specific behavior in) is released and can be evaluated, we can check for ourselves.
It seems everyone (even nVidia themselves) agrees that nVidia should allow the end-user full control over graphics features/settings.
Yes, but it has seemed this way before too.
So what exactly is causing the big rift between B3D and [H] or at least their audiences?
Is there a rift between the audiences? :-? There might be a rift between those who think [H] has serious problems and those who don't, but I'm pretty sure there are members of both groups in at least B3D's audience. The apparent rift between [H] and B3D is, IMO, simply a rift between what can be posted without deletion/bans, with those who think [H] has some serious problems therefore disproportionately represented in one of those places.
If B3D, [H], and nVidia all agree that nVidia needs to change their drivers, what's left to argue about?
I just don't get it...
Hmm...you seem to be missing a few months and a few issues in your consideration.
Park the Shark
31-Jul-2003, 19:31
That's the thing, a lot of people did and do think that nVidia cheating on 3dm2k3 is still wrong even if you don't happen to like the benchmark and that it was newsworthy enough that [H] should have covered it rather than just say, "Well, we don't like that benchmark so it doesn't matter". :roll:
A whole lot of people don't agree with that attitude, and Kyle has never directly addressed the issue of cheating. :(
Well, you nailed me there... I'm pretty much in the boat of "that benchmark sucks, who cares" lol...
I can understand and appreciate the point, though, of disliking nVidia cheating on a benchmark, period. Obviously, I do read [H], and sometimes I disagree with Kyle, but as far as 3DMark03 goes, he posted numerous links and stories regarding the cheating issue. I did think it was pretty odd the way he linked to ExtremeTech's article and at the same time roasted them a bit, questioning their motives. It's not like he was in some conspiracy to deny nVidia was cheating.
What you're saying, though, is essentially that people are made at [H] because they didn't proclaim at the top of their longs that nVidia is the devil? What do you expect them to add? They posted the links to other works that clearly demonstrated what was happening, as well as concluded it was done intentionally to "cheat". Put another way, are you not saying that [H] has a duty/responsibility to not just link to other people's findings, but independently verify everything they link to and add their own commentary?
If someone has already done the work, what would be the point in wasting resources only to say, "In conclusion, ExtremeTech was right!"
Finally, what bearing does this really have on the bi/tri issue?
It's been proven that [H] didn't even ask ATi about the thing before posting the woowoo.
The quack thing didn't affect all textures. Is there a noticeable IQ difference when playing?
So [H] was flat out lying when they said:
ATi engineers were asked last week if ATi drivers used any game specific instructions and we were told "No."
I don't know how many textures were affected by "Quack", but I have seen screenshots that show obvious differences. I would definitely say there is a noticeable IQ difference.
http://www.beyond3d.com/forum/viewtopic.php?t=7073
There is some info on the "Quack" issue. It's not even clear if [H] asked ATi. The wording indicates that it may have been someone else (possibly Nvidia) who asked ATi and the [H] just took someone elses (possibly Nvidias) word for it :wink:
It affected total of 5 textures in the game.
Joe DeFuria
31-Jul-2003, 19:37
What you're saying, though, is essentially that people are made at [H] because they didn't proclaim at the top of their longs that nVidia is the devil?
No, people are upset with [H] because he does not treat IHVs equally, nor does he have a consistent approach to benchmarks.
What do you expect them to add?
We expect fairness and consistency.
Finally, what bearing does this really have on the bi/tri issue?
Because benchmarking:
1) A card that CAN do trilinear, and doesn't, against.
2) A card DOING trilinear, even though it CAN do bi/tri...
Is in no way an attempt to fairly compare products.
Dave Baumann
31-Jul-2003, 19:39
Some UT2003 Screenshots with 44.03 drivers:
http://www.beyond3d.com/forum/viewtopic.php?p=145076#145076
The mip maps are visible - the transitions are blended, but I can make out there is a difference between one mip level and the next (look at the bottom if the image upwards - there is a difference between the detail texture to the next mip level down). With full Trilinear all mip levels should be blended so you wouldn't be able to notice one mip from the next.
Park the Shark
31-Jul-2003, 19:41
Someone said in the [H] forums that Kyle was doing Nvidia a disservice by not bringing their "optimizations" to the forefront. He did it with ATI and ATI then improved dramatically(it's amazing what a little bad press can do). By not putting pressure on Nvidia he is giving them a green light to continually go down hill when it comes to their drivers.
What more should he do? He showed the mip map highlights, they published an entire article dedicated to the issue, and he directly contact nVidia and nVidia has promised to fix it. Either he's lying and you have some proof regarding that, or he doesn't have to "pressure" nVidia b/c nVidia is already working on the resolution. What am I missing there?
digitalwanderer
31-Jul-2003, 19:44
What you're saying, though, is essentially that people are made at [H] because they didn't proclaim at the top of their longs that nVidia is the devil?
No, people are upset with [H] because he does not treat IHVs equally, nor does he have a consistent approach to benchmarks.
What do you expect them to add?
We expect fairness and consistency.
Finally, what bearing does this really have on the bi/tri issue?
Because benchmarking:
1) A card that CAN do trilinear, and doesn't, against.
2) A card DOING trilinear, even though it CAN do bi/tri...
Is in no way an attempt to fairly compare products.
Yup. ("Thanks Joe!" :) )
digitalwanderer
31-Jul-2003, 19:48
Someone said in the [H] forums that Kyle was doing Nvidia a disservice by not bringing their "optimizations" to the forefront. He did it with ATI and ATI then improved dramatically(it's amazing what a little bad press can do). By not putting pressure on Nvidia he is giving them a green light to continually go down hill when it comes to their drivers.
What more should he do? He showed the mip map highlights, they published an entire article dedicated to the issue, and he directly contact nVidia and nVidia has promised to fix it. Either he's lying and you have some proof regarding that, or he doesn't have to "pressure" nVidia b/c nVidia is already working on the resolution. What am I missing there?
"What more should he do?"?!? :shock:
His article on it was just a PR justification piece for nVidia, what he SHOULD have done was find some drivers that use trilinear in UT2k3 or used a bilinear setting on the ATi card or both and posted those results also. The way he presented it was basically a condescending dismissal of all the logically presented evidence with a big f-ing "Meh.". :(
He also shouldn't just be posting this stuff buried in his forums, it deserves to be openly posted on his front page same place as his PR BS was.
ATi is still not really pleased with [H] and did NOT consider that follow up article vindication, but I can't comment any further on that right now.
Dave Baumann
31-Jul-2003, 19:49
Thanks for clearing that up. So there is no AF controversy, that's just a bug? The bi/tri controversy remains, though..
There is a separate issue with AF in that only the AF selected is applied to texture stage 0, any other stages recieve a maximum of 2x AF - and this overides any application settings.
(If you are looking at comparisons with ATI there is an issue that ATI applied only Bilinear AF to textures beyond stage 0 - although this can be overridden by the application. link (http://www.beyond3d.com/reviews/ati/9800_256/index.php?p=13))
Park the Shark
31-Jul-2003, 19:52
Though you have to ask, why would nVidia make this a UT specific setting, and not a global setting, if it really doesn't "impact" image quality?
Why did ATI release a driver update specifically to improve NeverWinter Nights play? Popular games get special driver treatment though specific optimizations and fixes.
I don't think the "crime" here is that nVidia optimized... the "crime" is that they over-rode a user selectable setting. No one seems to like that. They all seem to agree. That's why I don't understand the ill will.
The other part of the issue, is that you CAN get a similar quality level on ATI cards by setting (I believe) the texture slider lower.
So why isn't that done on ATI cards when comparing performance to nVidia ones?
That's what I was talking about previously when I mentioned IQ equivalent benchmarking, in addition to technically equivalent benchmarking. This leads to a very valid point, though, and I posted the follow message directly to Kyle:
1. From what you've written, I think you're of the opinion that nVidia should not be overriding user selected Trilinear filtering with this Bi/Tri mix. That would be the reason you consulted nVidia about changing that aspect of the driver, correct?
2. In light of point #1, do you think it's fair to compare benchmarking numbers from ATI's full tri filtering to nVidia's bi/tri mix? Has [H] published any reviews using the so called "cheat" drivers, making such a direct comparison, and if so, will those numbers be pulled or updated when nVidia delivers the drivers that enable full Tri filtering in UT2K3?
Park the Shark
31-Jul-2003, 19:55
What does nv30 delays have to do with being mad at [H]?
That's when Nvidia decided to trash 3Mark2003 because they couldn't compete, and recruited websites to voice their PR documents. That's when cheating on benchmarks became a viable alternative to having the fastest part out there.
It's the direct source for why Nvidia is touting a huge increase in speed on one of the most used benchmarked programs (UT2K) by cheating on the filtering, and why sites like [H] are supporting them.
In fact, though, the nv30 performed BETTER in 3dMark03 upon release. So that kind of shoots a hole in that theory....
How is [H] supporting cheating in UT2K3? They've openly stated that nVidia needs to provide a driver that allows the user to select FULL tri filtering.
There is a separate issue with AF in that only the AF selected is applied to texture stage 0, any other stages recieve a maximum of 2x AF - and this overides any application settings.
Is that what the issue raised on AMDMB.com turned out to be? And where were the specifics of this AF problem first talked about? I'd love to read from the source.
Some UT2003 Screenshots with 44.03 drivers:
http://www.beyond3d.com/forum/viewtopic.php?p=145076#145076
The mip maps are visible - the transitions are blended, but I can make out there is a difference between one mip level and the next (look at the bottom if the image upwards - there is a difference between the detail texture to the next mip level down). With full Trilinear all mip levels should be blended so you wouldn't be able to notice one mip from the next.Dave, are you saying that under trilinear filtering in the 44.03 control panel, that mip transitions shouldnt be evident?
Due to an accident, I had to recently go from an R300 to an NV25, and the mip banding is killing me. I'm using the 44.03's. If I set the mipmap detail level to 'best image quality' in the control panel, my framerate is nearly halved. I never saw banding like this on my R200, or R300, at any driver setting. If it wasnt for this problem, I would be pleased with this particular card.
Actions associated with this include dismissing ExtremeTech, Beyond3D, and other sites involved in the nVidia/3dmark 03 benchmarking image quality degradation issues as "police" or lackeys, including saying that the issue's exposure was "payback" for not having access to Doom 3 granted by nVidia (he's only expressed "regret" concerning ExtremeTech, which happens to be the particular site which is part of a large media group) .
Actually, to clarify this point. Kyle only said that he was sorry for making it public. He is not sorry for making it.
Bouncing Zabaglione Bros.
31-Jul-2003, 20:05
Well, you nailed me there... I'm pretty much in the boat of "that benchmark sucks, who cares" lol...
But *why* does it "suck"? Do you understand that it is meant to be a synthetic loading test that hammers every card as hard as possible to find it's breaking point? This is a quite different test from most other tests.
If it is issues with cheatability or "it's not a game", those same issues also "invalidate" any other test you care to mention.
I can understand and appreciate the point, though, of disliking nVidia cheating on a benchmark, period. Obviously, I do read [H], and sometimes I disagree with Kyle, but as far as 3DMark03 goes, he posted numerous links and stories regarding the cheating issue. I did think it was pretty odd the way he linked to ExtremeTech's article and at the same time roasted them a bit, questioning their motives. It's not like he was in some conspiracy to deny nVidia was cheating.
That's just a standard technique for disparaging your opponent. Instead of ignoring or rebutting the points raised, you point to the information, vaguely disparage it and the source with veiled comments. [H] rarely linked to any of the "cheating" stories, except with a bit of namecalling and vague accusations of ulterior motives to explain the articles as anything other than factual reporting.
What you're saying, though, is essentially that people are made at [H] because they didn't proclaim at the top of their longs that nVidia is the devil? What do you expect them to add? They posted the links to other works that clearly demonstrated what was happening, as well as concluded it was done intentionally to "cheat". Put another way, are you not saying that [H] has a duty/responsibility to not just link to other people's findings, but independently verify everything they link to and add their own commentary?
I expected them to tell the truth, not to say things were okay when they were not. They still don't admit to Nvdia doing any cheating, and still say that even if they did cheat, it does not matter because "you can't see it" or they don't like that particular benchmark.
It's nonsensical. Can you imagine a racing magazine saying it's okay to cheat during a car race because they (and the one particular manufacturer that they are close friends with) don't like that track?
If someone has already done the work, what would be the point in wasting resources only to say, "In conclusion, ExtremeTech was right!"
Where did thay say that? AFAIK, [H] has always said ET was wrong, stating ET had some kind of grudge because they did not get one of the exclusive Doom3 Nvidia PR machines to test.
Finally, what bearing does this really have on the bi/tri issue?
Shows a pattern of behaviour. You need to see the bigger picture to understand the context of the UT2K filtering issue.
swanlee
31-Jul-2003, 20:05
Even if you and Kyle find 3dmark useless the fact remains it is used in almost every benchmark and several oem's use it to determine what graphic chips to put in their pc's. Nvidia flat out cheated to make their cards do better so more people and companies would buy their chips. That's consumer fraud in the simplest terms. Also the ironic part of this is it was Nvidia that pushed 3dmark in our face when they were doing good, they were using 3dmark scores in pr statements and advertising and linking scores on nvidia's official site. It was Nvidia that created the idea that 3dmark was an important all purpose benchmark, and now that there cards are doing badly they try and destroy 3dmark.
Now with Kyle what is happening is Nvidia is using him as an extension of their marketing dept. and for a site that claims to be a fair and unbias hardware review site that is a big no no. Kyle is now seen as a co conspirater in Nvidia's attempt to steal money from people by making there crappy new cards seem better than they really are.
All of these side issues of UT2k3, shadermark cheats, uwinder anti cheat script and now the encryption of their drivers are just side efects that show just how bad Nvidia was cheating and it seems to have been present in 3dmark 2001 also.
I mean please people every week new things are learned about how extensively nvidia has been cheating in their drivers aren't we beyond the point of trying to defend them and people like Kyle who are simply corporate extensions of Nvidia?
Dave Baumann
31-Jul-2003, 20:09
Let try and keep the thread to the context of the title please.
Park - did you see the images referenced earlier?
Bouncing Zabaglione Bros.
31-Jul-2003, 20:12
In fact, though, the nv30 performed BETTER in 3dMark03 upon release. So that kind of shoots a hole in that theory....
But much, much worse than R300. It only got vaguely competative when it was overclocked (hence the noisy heatsink) and still had to cheat badly in the drivers.
How is [H] supporting cheating in UT2K3? They've openly stated that nVidia needs to provide a driver that allows the user to select FULL tri filtering.
Where? Not in the review where they crown the NV35 as faster than the R350, even though the NV35 is secretly dropping IQ (whenever the user or application requests trilinear) in order to do less work, and thus get higher benchmark scores. If that's not misleading and worth pointing out in a competent review, I don't know what is.
The unwillingness to change faulty reviews even after it has pointed out to Kyle and Brent can be nothing but complicity in the deception.
In fact, though, the nv30 performed BETTER in 3dMark03 upon release. So that kind of shoots a hole in that theory....
Which driverset for the NV30 are you comparing with which driverset for the 9700? And are you sure that the drivers you're referring to were devoid of invalid opimizations such as shader replacement, forced precision below the DX9 spec, etc.? URL, please?
How is [H] supporting cheating in UT2K3? They've openly stated that nVidia needs to provide a driver that allows the user to select FULL tri filtering.
By allowing an improper comparison to stand, they are supporting Nvidia's efforts to deceive consumers about the performance of their video cards. at equivalent image quality settings vs. ATI. It wasn't too long ago that [H] was saying the only valid comparison to ATI's QualityAF was NV's Application setting, which did full Trilinear("Apples to Apples at last", I believe they referred to it as). Their next setting down(Balanced) performed a bi/tri blend that is curiously similar to the pseudo-trilinear we're seeing now. [H] is now saying however, that ATI's QualityAF and Nvidia's pseudo-trilinear CAN be compared to each other.
One might argue that since Nvidia's drivers don't allow for full Trilinear filtering, that a valid comparison is impossible, so this is the best that can be done(even if this is the case, it should be clearly noted in any review that makes use of the flawed comparison). This is crap. ATI's scores should not be penalized because they can go up to the proverbial "Eleven". And just to mix another metaphor, "if Mohammed will not go to the mountain, the mountain must go to Mohammed". In other words, if you can't raise Nvidia's image quality up to what ATI is capable of, bring ATI's down to Nvidia's level and then run the numbers. Then at least you can say "with Nvidia you can get $SCORE, and with ATI doing the same thing you get $OTHERSCORE". I could be wrong, but I believe it was Doomtrooper who has stated that using the Texture Quality slider it is possible to get the Catalysts to do the same pseudo-trilinear filtering. For a site that made such a big deal about bucking the trend and finding proper settings for a valid comparison between the NV3x and R3x0 not too long ago, they sure don't seem to place a very high priority on it anymore.
Nazgul (or anyone else who cares :)),
The anisotropic filtering issue we/I raised with the article at AMDMB is a different issue than the bi/tri issue.
The AF issue looks more to be a bug. I've done more tests and will publish an update that will be released after my next article is published (which will hopefully be tonight). The AF problem is inconsistent. It appears in only one of the maps I use, but is fine on other maps.
The timing of the AF issue, the [H] article, and B3D's tri/bi threads, and NVIDIA's per-application optimizations all came at pretty much the same time. It therefore seemed that all of this was the same problem. However that's not the case.
What we talked about at AMDMB appears to be a bug in v44.03 drivers. Nothing more.
The Bi/Tri and application-detection issue still stands.
- Jonathan.
digitalwanderer
31-Jul-2003, 20:25
Nazgul (or anyone else who cares :)),
The anisotropic filtering issue we/I raised with the article at AMDMB is a different issue than the bi/tri issue.
The AF issue looks more to be a bug. I've done more tests and will publish an update that will be released after my next article is published (which will hopefully be tonight). The AF problem is inconsistent. It appears in only one of the maps I use, but is fine on other maps.
The timing of the AF issue, the [H] article, and B3D's tri/bi threads, and NVIDIA's per-application optimizations all came at pretty much the same time. It therefore seemed that all of this was the same problem. However that's not the case.
What we talked about at AMDMB appears to be a bug in v44.03 drivers. Nothing more.
The Bi/Tri and application-detection issue still stands.
- Jonathan.
So has nVidia agreed to fix the tri/bi issue or the AF bug or both?
As far as I know, NVIDIA determined that the AF issue was a bug before I concluded as such last weekend. In their IRC chat, they referred to the AF issue as a "bug". So I would assume that NVIDIA has done their own testing in isolation, because they never contacted me about it.
If NVIDIA has already identified it as a bug, it may not necessarily mean it will be fixed (lots of bugs go unfixed).
The bi/tri issue I don't know what's happening with that (guys here have a better idea than I do). The only official statements I know regarding that are in that chat last week. As we all know, their replies were very general, so who knows if that will ever get fixed before NV36/NV38.
- Jonathan.
Park the Shark
31-Jul-2003, 20:44
Actually, it's "not true trilinear filtering".
Yes Brent resolved that for me above...
2. It has shown obviously noticeable IQ degradation in the HallOf Anubis add-on map.
Well, perhaps that is one place where it is evident even in selected still screenshots?
Brent also points out that the HallOfAnubis issue is AF, not tri filtering related. It apparently is limited to this one map. For the purpose of discussing the bi/tri filtering issue, it would seem the AMDMB article is irrelevant. This was what confused me as well. I saw the AMDMB article and thought all the "woowoo" was regarding AF. The real issue is not AF, but tri vs bi/tri.
Clear as mud, right?
You make a point about "cherry picking" screen shots, motion involved with tri filtering, etc. You do realize that you're essentially saying there's no way to prove that nVidia is damaging IQ, short of posting raw video captures. No one is going to do that for bandwidth reasons.
More accurate to say that "[H] has not shown obviously noticeable still screenshot IQ degradation for the maps selected", or anyone involved with posting screenshots you may have looked at.
Well, that's what I'm asking... Has ANYONE posted a screenshot showing decreased image quality? I have already said that I haven't seen any. I've looked. Haven't found them yet. Got a link for me?
It's not like the nVidia cards can't do trilinear...nVidia decided to disable it for fps boosting.
I don't think anyone has disagreed with THAT point.
5. nVidia has promised to resolve the issue by providing the end-user control over AF.
Note that nVidia's review guidelines already propose that the end-user has control over trilinear filtering ("Quality" mode), though through application detection as evidenced in UT2k3 that is not the whole truth. The whole truth about the change remains to be seen.
So is the bottom line that people want [H] to put on their front page something like "nVidia lied, saying they would give us true tri filtering, but gave us a bi/tri mix".
Actually, I think I'm seeing your point... Essentially, many people are feeling that [H] is supporting the "cheating" because despite KNOWING that nVidia is not doing full tri filtering, they are maintaining that it is fair to compare directly to ATI who IS performing full tri filtering. I do think that is a valid point. Furthermore, since ATI does have a very comparable setting, the two should be compared at that setting. Certainly it is NOT fair to set both to full tri-filtering when we know the nVidia card is going to override that.
I'm going to post this directly over at [H] and see if Kyle will respond.
You had some other points on why, in general, may do not like Kyle. I'm not into all that, hell just this bi/tri issue is complicated enough. I'll just let that go by saying that I think there must be some happy medium between allowing folks to go off flaming and deleting something just because you don't like it. I personally have no knowledge of the situation and don't care to get involved with it--unless he starts deleting MY posts. :) Seriously, though, I have noted he sometimes does have an emotional chip on his shoulder.
They also left a review standing that showed an nVidia card leading an ATI card in a mode that is represented as being equivalent. nVidia cards can do trilinear filtering, they do trilinear filtering in other apps, including those they recommend to show whether they are doing trilinear filtering or not, they do not do trilinear filtering in UT2k3, which happens to be an application they reported as having a performance boost in the driver set, and which happens to be often used for fps comparisons. In response to ATI apparently indicating they had a problem with this, this image quality article concluded that "it didn't matter anyways", and let the fps comparisons stand.
Yes, I can see a problem there. I think you articulated it better than anyone previously. It seems there is so much other fluff thrown into the mix by so many different people, it's been difficult to find the crux of the matter.
Park the Shark
31-Jul-2003, 20:47
Finally, what bearing does this really have on the bi/tri issue?
Because benchmarking:
1) A card that CAN do trilinear, and doesn't, against.
2) A card DOING trilinear, even though it CAN do bi/tri...
Is in no way an attempt to fairly compare products.
You're points are good... but what you're responding to was in relation to the 3dMark issue the DW brought up... I just don't see what 3DMark03 has to do with this bi/tri filtering business.
Park the Shark
31-Jul-2003, 20:50
[quote="DaveBaumann"]
http://www.beyond3d.com/forum/viewtopic.php?p=145076#145076
[quote]
It would be more relevant if we had these same screenshots taken with either a different driver set at the same settings, or a 9800 card at the same settings.
As it is, I have nothing to compare these images to.
Dave Baumann
31-Jul-2003, 20:55
Can you see the different mipmap levels?
Park the Shark
31-Jul-2003, 20:58
Where did thay say that? AFAIK, [H] has always said ET was wrong, stating ET had some kind of grudge because they did not get one of the exclusive Doom3 Nvidia PR machines to test.
In in his forum, under the video card section... There's a lengthy posting there on the issue... Their forum is down right now, but I think this is the link to the thread:
http://www.hardforum.com/showthread.php?s=&postid=1025103810#post1025103810
Bouncing Zabaglione Bros.
31-Jul-2003, 21:04
Where did thay say that? AFAIK, [H] has always said ET was wrong, stating ET had some kind of grudge because they did not get one of the exclusive Doom3 Nvidia PR machines to test.
In in his forum, under the video card section... There's a lengthy posting there on the issue... Their forum is down right now, but I think this is the link to the thread:
http://www.hardforum.com/showthread.php?s=&postid=1025103810#post1025103810
This has actually been discussed in another thread: http://www.beyond3d.com/forum/viewtopic.php?t=7164&highlight=
Many people (myself included) felt that the (grudging) half apology should have been in a front page editorial, where the original unfounded accusations took place. Making a forum post is akin to a newspaper tucking a retraction away on the bottom of page 19. Note that several people posted their thoughts here because they have been banned at [H] for trying to discuss similar matters there.
There's certainly nothing in there that even admits to Nvidia cheating, let alone claims it as a "bad thing".
Park the Shark
31-Jul-2003, 21:06
Can you see the different mipmap levels?
It looks odd to me... like it gets clear then blurry, then clear... But I still find it hard to make any judgement without a comparison.
Either a comparison to the screenshots taken with a different driver, or a different card, with the settings *set* the same.
Still, I've finally grasped the point here. People are mostly upset because they feel benchmark comparisons between the two are NOT valid, REGARDLESS of resultant IQ. I can certainly appreciate that standpoint.
I'll say again what I would like to see is two sets of benchmarks in all video card comparisons--technique to technique, and optimization to optimization with comparable IQ.
Joe DeFuria
31-Jul-2003, 21:06
Why did ATI release a driver update specifically to improve NeverWinter Nights play? Popular games get special driver treatment though specific optimizations and fixes.
Right, but bi-tri optimizations can be applied globally. It actually takes MORE effort to limit such optimizations to one application than to just force it on all the time. So why limit it to just UT?
Did ATI release a driver update to improve NWN play, by REDUCING image quality in that single game?
I don't think the "crime" here is that nVidia optimized... the "crime" is that they over-rode a user selectable setting. No one seems to like that. They all seem to agree. That's why I don't understand the ill will.
There are two crimes.
1) That nVidia overrode an application setting.
2) That [H] does not compensate for this by equally treating the competition with respect to performance benchmarks.
1. From what you've written, I think you're of the opinion that nVidia should not be overriding user selected Trilinear filtering with this Bi/Tri mix. That would be the reason you consulted nVidia about changing that aspect of the driver, correct?
2. In light of point #1, do you think it's fair to compare benchmarking numbers from ATI's full tri filtering to nVidia's bi/tri mix? Has [H] published any reviews using the so called "cheat" drivers, making such a direct comparison, and if so, will those numbers be pulled or updated when nVidia delivers the drivers that enable full Tri filtering in UT2K3?
So, what was Kyle's response to that?
Park the Shark
31-Jul-2003, 21:09
1. From what you've written, I think you're of the opinion that nVidia should not be overriding user selected Trilinear filtering with this Bi/Tri mix. That would be the reason you consulted nVidia about changing that aspect of the driver, correct?
2. In light of point #1, do you think it's fair to compare benchmarking numbers from ATI's full tri filtering to nVidia's bi/tri mix? Has [H] published any reviews using the so called "cheat" drivers, making such a direct comparison, and if so, will those numbers be pulled or updated when nVidia delivers the drivers that enable full Tri filtering in UT2K3?
So, what was Kyle's response to that?
He banned me from their forums.
lol j/k!! Their forums are down right now... or are they? :shock:
digitalwanderer
31-Jul-2003, 21:12
Still, I've finally grasped the point here. People are mostly upset because they feel benchmark comparisons between the two are NOT valid, REGARDLESS of resultant IQ. I can certainly appreciate that standpoint.
No, I'd go a step further than that. The clear/fuzzy bit you saw were mipmap lines and they're even more noticable in motion.
I haven't seen a 5900 in action, but I'd be willing to be dollars to donuts that you CAN tell they are there while playing...and that Kyle as a guy "who can tell the difference between 200fps and 250fps in Quake" should be able to see a visual difference between them.
I think the man is lying and playing around with words to avoid saying so.
Their forums are down right now... or are they? :shock:
They are down for me as well currently
digitalwanderer
31-Jul-2003, 21:22
Their forums are down right now... or are they? :shock:
They are down for me as well currently
Oooooh, you should read what Kyle is posting up about you Brent! :shock:
j/k! Down here too. :(
Park the Shark
31-Jul-2003, 21:44
http://www.hardforum.com/showthread.php?s=62a1e13f91a011a5b85efab7adeee1de&postid=1025110743#post1025110743
This is what I ended up posting:
1. From what you've written, I think you're of the opinion that nVidia should not be overriding user selected Trilinear filtering with this Bi/Tri mix. That would be the reason you consulted nVidia about changing that aspect of the driver, correct?
2. In light of point #1, do you think it's fair to compare benchmarking numbers from ATI's full tri filtering to nVidia's bi/tri mix? Has [H] published any reviews using the so called "cheat" drivers, making such a direct comparison, and if so, will those numbers be pulled or updated when nVidia delivers the drivers that enable full Tri filtering in UT2K3?
3. Knowing that nVidia is not doing full trilinear filtering (despite being capable of it and choosing to over-ride the setting), do you think a direct comparison to ATI while doing full trilinear is a valid performance comparison?
4. It has come to light that ATI has a very similar bi/tri filtering mix to what nVidia is doing. Do you think that it would be fair to compare that bi/tri mix to nVidia's bi/tri mix, rather than comparing the bi/tri mix of nVidia to the full tri of ATI?
5. IOW, as one person said it to me very succintly, he said he did not feel a direct comparison was valid between:
A. a card capable of full trilinear, but using a bi/tri mix
and
B. a card capable of a bi/tri mix, but using full trilinear.
What is your comment on this?
demalion
31-Jul-2003, 22:03
...
You make a point about "cherry picking" screen shots, motion involved with tri filtering, etc. You do realize that you're essentially saying there's no way to prove that nVidia is damaging IQ, short of posting raw video captures.
No that is wrong. I said you can't prove they are not damaging image quality short of video captures (or tools like colored mip levels), but said you can prove that they are damaging image quality. I also explained why: in short, seeing it in still screen shots means you see it in motion, but seeing it in motion does not mean you will see it in still screen shots...and games are played in motion.
Example of the logical issue here:
You have two metal junction boxes, each with several bare wires. In one box, you see sparks. In the other, you do not. The sparks in the one box is proof that it is dangerous. The absence of sparks in the other box is not proof that the box is safe. Colored mip map levels are like checking if the wires are bare or not (or maybe whether there are wires at all) in the first place...depends on other factors in the situation for whether there will be a problem when sticking your hands in the junction box.
No one is going to do that for bandwidth reasons.
I think Wavey (DaveBaumann) has been pointing to example shots where he maintains the mip map transition is evident even in still screenshots. Please note, if you are still under a misinterpretation of my comments, that this does not contradict what I actually said. Please direct some discussion towards those examples directly for discussing your questions about being able to see the issue.
More accurate to say that "[H] has not shown obviously noticeable still screenshot IQ degradation for the maps selected", or anyone involved with posting screenshots you may have looked at.
Well, that's what I'm asking... Has ANYONE posted a screenshot showing decreased image quality?
I believe Wavey has been tackling this...?
I have already said that I haven't seen any. I've looked. Haven't found them yet. Got a link for me?
I've seen such pictures, and noticed the transitions in them, and I believe Wavey provided them at the time. I presume he is linking to such pictures in his above discussion to you. Does that serve?
Also, does an accurate reading of my statements about still screenshots help concerning understanding this issue?
It's not like the nVidia cards can't do trilinear...nVidia decided to disable it for fps boosting.
I don't think anyone has disagreed with THAT point.
Well, there was a failure to represent the issue to readers of atleast one of the site's reviews.
5. nVidia has promised to resolve the issue by providing the end-user control over AF.
Note that nVidia's review guidelines already propose that the end-user has control over trilinear filtering ("Quality" mode), though through application detection as evidenced in UT2k3 that is not the whole truth. The whole truth about the change remains to be seen.
So is the bottom line that people want [H] to put on their front page something like "nVidia lied, saying they would give us true tri filtering, but gave us a bi/tri mix".
Actually, I think I'm seeing your point... Essentially, many people are feeling that [H] is supporting the "cheating" because despite KNOWING that nVidia is not doing full tri filtering, they are maintaining that it is fair to compare directly to ATI who IS performing full tri filtering. I do think that is a valid point.
That is the issue with regard to the recent UT2k3 review and the UT2k3 bi/tri article. The controversy is related to quite a bit more besides just that issue, but to which the issue is related, as I outlined.
Furthermore, since ATI does have a very comparable setting, the two should be compared at that setting. Certainly it is NOT fair to set both to full tri-filtering when we know the nVidia card is going to override that.
I'm going to post this directly over at [H] and see if Kyle will respond.
Well, FYI, similar things have been posted there. One polarizing issue, specifically, is that Wavey posted such at [H], in a post that seems by any measure not to have broken the rules of conduct of the forums, but was pointed in factuality and details presented (I'd choose the term "inconvenient" in illustrating issues with Kyle's "take" on trilinear filtering). What is polarizing about this instance is that Kyle banned Wavey as a result of it.
This is (one) incident that adds to the other factors I already mentioned.
You had some other points on why, in general, may do not like Kyle. I'm not into all that, hell just this bi/tri issue is complicated enough.
Well, you did ask. You might not be interested in all the details, but all of the details relate to the "controversy", though you perhaps atleast understand the specific flaws as far as just this issue to see why it is related to a broader issue of controversy.
I'll just let that go by saying that I think there must be some happy medium between allowing folks to go off flaming and deleting something just because you don't like it.
Well, the thing is Kyle allows himself to go off "flaming" and deletes things because he doesn't like it. :-?
I personally have no knowledge of the situation and don't care to get involved with it--unless he starts deleting MY posts. :) Seriously, though, I have noted he sometimes does have an emotional chip on his shoulder.
Well, you are free to have no knowledge of it, or not to get involved in it, but you can't change that it is involved in the answer to the question about the "controversy" with [H] you asked. However, I don't recommend trying to conduct a discussion by dictating that everyone else must view things in similar isolation.
swanlee
31-Jul-2003, 23:34
Gees he pretty much here condones what Nvidia has done as being responsible to their sotck holders. He has a very twisted and warped mind. Imagine what's gonna happnes once he figures out Nvidia is using him. reposted here in case it's nuked on the hardforum
"I think NVIDIA does everything they can do to beat the competition on every level that they can. ATI should be doing the same thing. If neither is, then they are not being responsible to their stockholders and that is simply the fact of the matter."
In the current context of what Nvidia has done to keep up with the competition this statement is pretty awful. This is one step away from you actually condoning what Nvidia has done lately.
They have cheated in 3dmark, cheated in shader mark we've seen that common timedemo's they've cheated in and now the whole bi/tri issue.
Being competitive is one thing but out right trying to fool your customers and oem's by artificially bumping up numbers you know are going to be used in reviews is simply fraud.
No matter what anyone thinks of 3dmark the reality is it is used a lot in benchmarks and oem's to determine the outcome of reviews and what graphic chip a company uses in their pc's.
To try and play it off like these drivers cheats were just nvidia trying to help their stock holders is condoning what would be considered in other industries a federal crime. I'm quite saddened that you have chosen to condone Nvidia's recent actions are mearly the actions of a company trying to stay competitive is a serious flaw in ethics of a company offering a product to it's customers.
I've got to agree with Swanlee 100% this is clearly consumer fraud at work and Hard is supporting it.
They still don't admit to Nvdia doing any cheating, and still say that even if they did cheat, it does not matter because "you can't see it" or they don't like that particular benchmark.
I think that the optimization techniques that we have recently seen used by NVIDIA in 3DMark03, in principle violate the very fabric that our community and industry is held together with.
http://www.hardforum.com/showthread.php?s=&threadid=647147&perpage=15&pagenumber=3
:wink:
StealthHawk
01-Aug-2003, 03:17
Can you see the different mipmap levels?
It looks odd to me... like it gets clear then blurry, then clear... But I still find it hard to make any judgement without a comparison.
That is because it is not doing trilinear. If it was, you wouldn't be seeing the different mipmap levels(ie, where it goes from blurry to clear to blurry, etc).
Take a look at this:
http://www.3dcenter.de/artikel/ati_nvidia_treiberoptimierungen/screenshot05+06.php Top shot is bi/tri, bottom shot is full trilinear. Can you see a difference? You can see a mouseover version here http://www.3dcenter.de/artikel/ati_nvidia_treiberoptimierungen/index3.php
This is one of the big problems of [H]'s article. They did not compare NVIDIA's tri/bi optimization to NVIDIA's normal full trilinear. They compared ATI's tri/bi to NVIDIA's tri/bi, and somehow came across the conclusion that NVIDIA's tri/bi did not decrease image.
Or perhaps what they wanted to compare was ATI's trilinear AF to NVIDIA's tri/bi AF. Which AFAIK they did not do. Now, why did they not do this? Because although trilinear was set in the game, AF was set via the control panel, which AFAIK overrides the game setting. So ATI was not using trilinear AF.
Regardless, obviously the best comparison would be NVIDIA to NVIDIA, not NVIDIA to ATI. Interestingly enough, Brent concludes that NVIDIA tri/bi 0xAF looks better than ATI trilinear 0xAF.
StealthHawk
01-Aug-2003, 03:21
Now, if [H] was truly nVidia biased, wouldn't they have been singing the virtues of 3DMark because nVidia's nv30 was the leader in 3DMark when [H] was blasting it.
No, because NVIDIA wrote up a nice presentation on why 3dmark03 was useless and unfair. If [H] is NVIDIA biased as many believe, then they will agree with NVIDIA, not disagree.
Also, NVIDIA only won with cheating drivers. You can see NV30's real 3dmark03 performance in articles that benched it when the 330 patch first came out, which eliminated cheats. Or, you can do it yourself by downloading RivaTuner and running the anti-detection script.
They still don't admit to Nvdia doing any cheating, and still say that even if they did cheat, it does not matter because "you can't see it" or they don't like that particular benchmark.
I think that the optimization techniques that we have recently seen used by NVIDIA in 3DMark03, in principle violate the very fabric that our community and industry is held together with.
http://www.hardforum.com/showthread.php?s=&threadid=647147&perpage=15&pagenumber=3
:wink:
Micron, I got a cavity just from reading that candy coated remark. :wink: My wife is a teacher. There have been times when she was forced to write things like "we will continue to help Bobby with his peer to peer interactions." when she wanted to say "Bobby really needs to stop hitting his classmates." Why it is it so difficult for Kyle to tell his readers that Nvidia cheated in plain and simple language.
digitalwanderer
01-Aug-2003, 03:37
Why it is it so difficult for Kyle to tell his readers that Nvidia cheated in plain and simple language.
A. Because he's afraid Brian and Derek wouldn't be his friends anymore.
B. He has a severe truth impedement.
C. He has actually started to believe the BS he's been spouting out so anything that is contrary to it is automatically wrong or doesn't exist in his mind.
D. He pretty much bet his reputation on his ability to help nVidia pull this one off and it would be counter-productive to that agenda to say anything like that.
Pick the one you like best. :)
Park the Shark
01-Aug-2003, 07:01
All I can say is I came here trying to see what people were upset about, and I can clearly see it at this point... To my direct questions, Kyle responded:
You have asked some extremely broad questions here that do not make specific references so I will have to make some assumptions to answer....
1. That is not my opinion and I don't think I have ever stated that. I think there should be an option in the driver to let the more experience user allow the game to set the different levels of AA/AF/Tri/Bi if that is what he wants to do. That is part of the reasoning behind asking for the change. Another part of that is to serve our own selfish needs to facilitate apples-to-apples benchmarking. Still, the user should be the main reason the change was made IMO.
2. I will have to assume this question is in direct reference to UT2K3. This article will directly answer your question and also inform you of why it was written.
3. That would need to be evaluated on a case-by-case basis. I don't think there is a black and white answer to this question.
4. I think IQ would need to evaluated to make that determination.
5. I really don't have any comment on that person's opinion. Everyone is certainly entitled to their own.
My response was:
I'm going to start with point 5, since that was just incredible.
5. IOW, as one person said it to me very succintly, he said he did not feel a direct comparison was valid between:
A. a card capable of full trilinear, but using a bi/tri mix
and
B. a card capable of a bi/tri mix, but using full trilinear.
What is your comment on this?
quote:
--------------------------------------------------------------------------------
Originally posted by FrgMstr
You have asked some extremely broad questions here that do not make specific references so I will have to make some assumptions to answer....
5. I really don't have any comment on that person's opinion. Everyone is certainly entitled to their own.
--------------------------------------------------------------------------------
I don't think the questions could be much more specific, unless you are intentionally LOOKING to interpret them as vaguely as possible. I would certainly say that the questions were more specific than the answers.
To this direct and specific point:
It is invalid to compare benchmarks for:
A. a card capable of full trilinear, but using a bi/tri mix
and
B. a card capable of a bi/tri mix, but using full trilinear.
Your answer is "no comment"....
Now, this is getting plain odd... On point 1... I said "I think you're of the opinion that nVidia should not be overriding user selected Trilinear filtering with this Bi/Tri mix." and you say "That is not my opinion and I don't think I have ever stated that. I think there should be an option in the driver to let the more experience user allow the game to set the different levels of AA/AF/Tri/Bi if that is what he wants to do."
WTF is the difference here? If you think the user should be allowed to set the damn settings, then you obviously don't think nVidia should be overriding the user selected settings.
On point 2... In response to "do you think it's fair to compare benchmarking numbers from ATI's full tri filtering to nVidia's bi/tri mix?" you do not find that specific enough to say simply yes or no? Further on point 2, in response to "will those numbers be pulled or updated when nVidia delivers the drivers that enable full Tri filtering in UT2K3?" you do not find that specific enough to say simply yes or no?
On points 3 and 4... I think what would solve this issue is simply to complete your homework, so to speak. If you are going to let nVidia have a CHANCE to run a bi/tri mix, why is ATI not given that same chance?
It's obvious that Kyle does not intend to change his mind. Since he is not going to admit the comparison is invalid, we are left only to speculate as to his reasoning. I'm often one to give a person the benefit of the doubt...
I think he's probably more stubborn than willingly biased towards nVidia. You see what you want to see, right?
Thanks for coming here Park the Shark. I think this thread has been very positive, largely due to your willingness to see all points of view. I hope you continue to hang out here. :D
Bouncing Zabaglione Bros.
01-Aug-2003, 08:23
They still don't admit to Nvdia doing any cheating, and still say that even if they did cheat, it does not matter because "you can't see it" or they don't like that particular benchmark.
I think that the optimization techniques that we have recently seen used by NVIDIA in 3DMark03, in principle violate the very fabric that our community and industry is held together with.
http://www.hardforum.com/showthread.php?s=&threadid=647147&perpage=15&pagenumber=3
:wink:
And straight afterwards, the kicker is:
We have suggested that reviewers stop using 3DMark03 as is obviously represents nothing pertaining to real world gaming.
Why? Because Nvidia "optimse" for it. Nvidia also "optimise" for UT2K, but I don't see Kyle advocating that people stop using UT2K as a benchmark. Kyle either does not understand what 3Dmark2003 does, and why it is different from gaming benchmarks, or is entrenched in his position after swallowing and parroting the Nvidia anti-3Dmark document at the beginning of the year.
To lambast 3Dmark2003 as useless whilst letting Nvidia continue to "violate the very fabric that our community and industry is held together with" is nonsense. A couple of recent, qualified, grudging statements tucked away in the forums does not counterbalance the damage done by his implicit and explicit support of Nvidia's actions over a long period in the main articles, reviews and news features of [H].
That grudging concession, where he can't even bring himself to use the word "cheat", continuing to sugar coat it with the word "optimisation", is brought to us by the man who says:
I can't really say that that optimization is "wrong" or "right" and I don't think that is the way to go about characterizing it. We are in this huge gray area and I don't think you will see any lines drawn that are specific. Game specific optimizations are here to stay, that is something I believe. It is our job as a review site to make sure that a certain level of image quality is maintained. I think we fully demonstrated this earlier this year with our 5200/5600 article. I think this is something we do with every review we write. I think from reading the linked article you will see that we thought the IQ was sliding a bit to far towards what you might say is "wrong". We made huge efforts to get that fixed and overall the problem was solved to some extent, although this UT2K3 mipmap issue has clouded the air.
I look at it this way and have hinted at this above. NVIDIA and ATI have the right to do whatever they want to with their products and drivers. If you do not like the IQ that NVIDIA or ATI is producing, then you should not buy their card. This is a free market and unless any specific contracts are signed between game developers and IHVs regarding IQ they are free to do what they wish, whether we like it or not.
We feel that IQ and Performance are the two foremost issues that we should be representing to our readers for them to make an informed decision when buying a video card.
It's very little, very late, and the words still do not represent the *actions* that [H] take. For instance, look at the last sentence above, then compare it with the lowering of IQ in both synthetic and game tests (along with other cheats) that Nvidia have been doing, and how these have been ignored, or even championed in recent [H] articles.
StealthHawk
01-Aug-2003, 08:54
5. IOW, as one person said it to me very succintly, he said he did not feel a direct comparison was valid between:
A. a card capable of full trilinear, but using a bi/tri mix
and
B. a card capable of a bi/tri mix, but using full trilinear.
What is your comment on this?
Ok, once again I will stubborningly bring up this point about [H]'s UT2003 filtering comparison.
Based on the information layed out in this thread,
http://www.beyond3d.com/forum/viewtopic.php?t=6719, trilinear AF was not enabled on the r9800. Based on Brent's own test settings, this is what he did:The ATI driver control panel was set to “Application Preference” on both AA and AF so that we could test with AF disabled. Then we set the AF slider to Quality AF when AF was tested.
I have seen this brought up once, and no one denied or agreed with whether or not the testing was done correctly or not. When I have asked, I have gotten no answer. I don't want to spread false information. Based on what I have read about ATI's control panel and application settings, Brent did not set up the test correctly if he in fact wanted to use trilinear AF. Then again, while perusing the article, I did not actually find it written anywhere where he explicitly states ATI was using trilinear AF, although it is certainly implied, and I have seen people all over different forums saying that the [H] article "proves there is no difference between trilinear AF and tri/bi AF because of the comparison between ATI and NVIDIA performed in the article."
Now, I am reasonably sure I am correct and [H] did not test trilinear AF in UT2003. I am also astonished that none of ATI's staunch defenders have brought up this point. This makes me believe I may be in error.
Here are the facts about NVIDIA:
1) there is no way to get full trilinear filtering in UT2003 via the control panel or by changing the game settings. This optimization is triggered by the detection of UT2003.exe.
2) Full trilinear can be enabled by using RivaTuner and the anti-detection script.
Here are the facts about ATI:
1) By selecting 'Quality AF' in the control panel, trilinear is performed in stage 0 and bilinear is performed on other stages. Normally this is fine and provides full IQ, but not in UT2003.
2) With AF off, full trilinear is performed, unlike with NVIDIA cards.
3) Full trilinear is attainable by enabling AF in UT2003 via the ini file and by setting AF to 'application' in the control panel <- Brent did not do this.
There are a few possibilities here.
1) Brent mistakingly listed the test setup and trilinear AF was used for ATI.
2) Brent knowingly did not test trilinear AF for ATI.
3) Brent unknowingly did not test trilinear AF for ATI.
The firinqsquad Asus FX 5900 U review has some words and screenshots about the tri/bi issue
Link (http://firingsquad.gamers.com/hardware/asus_v9950_ultra_review/page4.asp)
Dave Baumann of Beyond3D discovered that GeForce FX cards render a form of quasi trilinear filtering, even with the driver running in quality mode. This is important because by using a mix of bilinear and trilinear, performance is enhanced. When you factor in the significance of Unreal Tournament 2003 not just as a game, but as a performance benchmark used by numerous publications, quite a few people became concerned about the legitimacy of testing results taken on this website as well as countless others. ;)
Reverend
01-Aug-2003, 11:07
We have suggested that reviewers stop using 3DMark03 as is obviously represents nothing pertaining to real world gaming.
He has either missed the point about 3DMark03 or he believes/accepts FM's "The Gamers Benchmark" slogan about 3DMark03 (which is not quite understandable given that I assume he is a matured guy) or he is just being anal-retentive (i.e. "FM says their 3DMark03 is a Gamers' Benchmark when we all know it is not").
3DMark03 is nothing more than a suite of technology demos. I, as part of a website that is FM's beta partner, have said publicly (http://www.beyond3d.com/forum/viewtopic.php?t=6225&start=20) that I do not agree with FM's "The Gamers' Benchmark" slogan (which I have voiced out to FM in their private/NDA'ed solicitation of their beta members for the next 3DMark). We all know that it is not "a gamers benchmark"... stressing on this shows that a person is just ignoring the obvious while also wanting to knock on FM for their obviously-incorrect slogan.
If FM is not allowed, or is not supposed, to make a benchmark that attempts to show the benefits of the latest 3D technologies as advertised prominently by FM and IHVs themselves, and reported so very prominently by 99% of the websites out there in their p/reviews of the latest video cards, then pray tell what is the purpose of websites in featuring the latest 3D technologies so prominently in their previews of the latest video cards?
Because the latest technologies is exciting (surely this can't be argued... why else would websites talk about such latest technologies in their previews of the latest video cards?). Because the only way to show them off is via technology demos (ditto) and not games. Because FM happens to show them off probably the earliest. Because FM do not care if their benchmark that utilizes the latest 3D technology makes as much money or has the same financial consideration as developers that make games, nor does FM even need to care as much about support/compatibility as game developers with their games -- FM makes it a point that either you have the latest video card or not and that the "score" will be affected accordingly and that the performance will be affected accordingly.
Kyle/public is right for knocking FM about 3DMark not being representative of games. Because FM never believes it to be so, regardless of their marketing slogan for 3DMark. But when I questioned Kyle about his opinion that "3DMark03 is useless" while he also told me via email that he would not object to using synthetic benchmarks in any of their articles/reviews, and that I think 3DMark03 really is nothing more than a suite of synthetic benchmarks and hence I am a little puzzled by his statement that 3DMark03 is useless yet he agrees that synthetic benchmarks have their place at [H]... so why not use 3DMark03... he did not reply.
Kyle is bent on, and publizing the fact, that he/[H] will not use 3DMark03... because he has said so, is not willing to retract this... but at the same time act very strange about the whole matter. It is strange because he knows 3DMark03 is nothing more than a suite of synthetic demos (but he will knock FM/3DMark03 for its "Gamers benchmark" slogan) yet he has acknowledged that synthetic benchmarks have their place. Huh? Someone say "An agenda... or a refusal to acknowledge/admit mistakes".
Kyle will never admit he is wrong wrt FM/3DMark03. Because NVIDIA will never admit the same.
Hanners
01-Aug-2003, 11:28
So has nVidia agreed to fix the tri/bi issue or the AF bug or both?
As I understand it from Kyle, what nVidia have promised to do is reintroduce the 'Application' setting in their driver control panel, so that an application can force it's own aniso settings.
However, what hasn't been promised is that using this 'Application' setting will override nVidia's application specific optimisation in UT2003. Kyle seems to be guessing that it will (which is the reason why he says that the tri/bi problem was 'fixed before it was started'), but at no point have nVidia promised to remove this specific optimisation or make it user-controllable (at least not publically).
Evildeus
01-Aug-2003, 13:39
We have suggested that reviewers stop using 3DMark03 as is obviously represents nothing pertaining to real world gaming.
He has either missed the point about 3DMark03 or he believes/accepts FM's "The Gamers Benchmark" slogan about 3DMark03 (which is not quite understandable given that I assume he is a matured guy) or he is just being anal-retentive (i.e. "FM says their 3DMark03 is a Gamers' Benchmark when we all know it is not").
Well i don't know about kyle, but shouldn't FM remove that slogan? I would say false publicity.... I don't mind, i'm not using 3DMark, but well it's a "gamer benchmark" with "game test" knowing that in fact it isn't.
digitalwanderer
01-Aug-2003, 13:49
Kyle is bent on, and publizing the fact, that he/[H] will not use 3DMark03... because he has said so, is not willing to retract this... but at the same time act very strange about the whole matter. It is strange because he knows 3DMark03 is nothing more than a suite of synthetic demos (but he will knock FM/3DMark03 for its "Gamers benchmark" slogan) yet he has acknowledged that synthetic benchmarks have their place. Huh? Someone say "An agenda... or a refusal to acknowledge/admit mistakes".
Kyle will never admit he is wrong wrt FM/3DMark03. Because NVIDIA will never admit the same.
Yeah, I'm afraid you're right on that one.
I never knew Kyle wanted to be a cop or a politician when he grew up, it explains a LOT! The man is pretty much incapable of giving a straight answer on this issue since the truth would be embarressing.
I'm just wondering if yesterday he had an actual epiphany that he was wrong, he was drunk, or someone slipped some sodium pentathol into his drink....either that or he thought he could wordsmith his way out of it by redefining the word "is". :roll:
I don't get what the point of his thread was about if it was just gonna be a song-n-dance again. :( ("Pssssssst! Dig, it was damage control!" )
Park the Shark
01-Aug-2003, 15:36
On 3DMark03, maybe he's wrong, but he listed:
http://www.hardocp.com/article.html?art=NDI4LDM=
If that information is in fact correct, I agree that 3DMark is not very useful... You have DX7 and 8 tests, done in apparently weird ways:
The sky in Game Test 4 uses Pixel Shaders 2.0, and so having to first draw the entire sky, only to have scene objects occlude it later is an expensive way to render, almost like a back-to-front Painter's Algorithm. Using clip planes to ignore drawing non-visible parts of the sky can save considerable processing time and bandwidth -- although we don't know exactly how much performance benefit nVidia receives. Oddly enough, nVidia hasn't published a registry setting bit to enable/disable this "feature." And while one could take FutureMark to task for drawing the scene in this manner, it is nonetheless what the application does, and we think it should be executed correctly in the hardware.
There's a bit of DX9 pieces (and DX8 as well) thrown in only on the mothernature "game" (and apparently the 4 game tests are all that matter for the overall score).
So I think the reason 3DMark was concluded as useless is because if you want to test DX7 and 8, do it with real games... If you want to test DX9... Don't you need more than a few seconds of a DX9 test, and isn't it muddling to the make that only ~30% of the overall score?
The problem is not neccessarily that 3DMark is synthetic--it just doesn't have much basis in reality--particularly the overall score.
I think it's a good utility for measuring fillrate. I could understand using the mothernature score as a forward looking comparison... But there are simply much better tools for DX7 and 8 testing out there.
I happen to agree with Kyle that synthetics have their place. For me personally, that place is to test things like fillrate for video cards, or memory bandwidth on a motherboard, or sustained transfer rate on a hard drive. Those are all example of trying to find a limit of the actual throughput to compare to a mfgr listed specification.
The 3DMark overall score is pretty pointless, then... I still like the fillrate testing, though.
Bouncing Zabaglione Bros.
01-Aug-2003, 16:04
If that information is in fact correct, I agree that 3DMark is not very useful... You have DX7 and 8 tests, done in apparently weird ways:
3Dmark2003 is different from other tests in that it is a loading test. It is like measuring the breaking strain of a steel cable. In the real world, you don't use a cable to the point where it breaks straight away, but the breaking strain is a useful metric if what you want to know is the strength of the thing (and thus it's SWL).
To claim it is usless is like saying the top speed of a car is a useless metric because "people don't actually drive around town at 150 mph". That doesn't make knowing the top speed any less useful a metric to compare cars if you understand the context of the results. Can you imagine a car magazine telling it's readers "well, we don't believe that top speed or 0-60 times are worth telling you, because no one drives like that"?
Now you may claim that what 3DMark2003 measures is pointless to you (which is a different discussion) but that doesn't invalidate it as a tool, any more than testing a car's top speed is invalidated by the fact that you won't ever be driving that fast.
3DMark2003 is designed to measure certain things in a certain way so that you can get a comparison between cards. To complain that it doesn't measure *other* things in *different* ways is a pointless exercise. It does what it is designed to do, and tests things which give us different types of information from running UT2K or Quake.
Don't forget that 3Dmark2003 was written this way because of input from many manufacturers, *including* Nvidia. They wanted a heavy, loading based test that was divorced from whether there was a fast/slow CPU in the PC. It's just that Nvidia has managed to muddy the waters by reeling in people like Kyle to give out a confusing message to users so that they can hide the poor performance of the GFFX cards behind this "it's not a real test, it's not a real game" smokescreen.
No it's not a real game - that's why it was developed the way it is.
The 3DMark overall score is pretty pointless, then... I still like the fillrate testing, though.
Actually, if a benchmark is done right and no IHV's are actually cheating, I believe that an overall score is not useless but instead potentially valuable. It could show you relatively speaking that Card A is relatively faster than Card B on Machine C. I also believe that 3DMark 99/Max and to a certain degree 3DMark 2001 was successful in doing that. You could look at the scores in those tests and actually say that a GeForce3 was faster than a Radeon 7500 in almost every application. Unfortunately, the major changes to 3DMark03 like no longer using the Max Payne engine and more GPU intensive tests have clouded the use of the overall score in some people's eyes. It also didn't help that the performance of the R3xx and NV3x series are a probably too close to call and thus the need to look at subjective tests like image quality. Chalk it up to bad timing. However, can you still use the 3DMark03 overall score for simple comparisons? I think so, if you understand how the benchmark works, are using it under the right conditions and provided drivers are no longer optimizing for the tests.
Tommy McClain
swanlee
01-Aug-2003, 16:25
What is most ironic here is it was Nvidia themselves that greatly helped 3dmark be the standard in 3d benchmarks. They are the ones that used their scores in every pr release and linked to them off there main website.
Whaen tehre cards were doing good Nvidia promoted the heck out of 3dmark and they are mostly responsible for how highly used 3dmark is.
Now all of a sudden there cards look like crap on 3dmark because they are crap and do not follow standard api's and need cheats to do well. Nvidia got ticked off that 3dmark actually made a good gpu bench that stricktly followed the DX api while Nvidia's new cards need customn paths to perform well because nvidia is trying to control the api much like 3dfx did with glide.
Nvidia is pretty much following 3dfx's blue print to disaster to a T, excpet now they are taking a step further and purposley mis-leading people.
The 3DMark overall score is pretty pointless, then... I still like the fillrate testing, though.
Actually, if a benchmark is done right and no IHV's are actually cheating, I believe that an overall score is not useless but instead potentially valuable. It could show you relatively speaking that Card A is relatively faster than Card B on Machine C. I also believe that 3DMark 99/Max and to a certain degree 3DMark 2001 was successful in doing that. You could look at the scores in those tests and actually say that a GeForce3 was faster than a Radeon 7500 in almost every application. Unfortunately, the major changes to 3DMark03 like no longer using the Max Payne engine and more GPU intensive tests have clouded the use of the overall score in some people's eyes. It also didn't help that the performance of the R3xx and NV3x series are a probably too close to call and thus the need to look at subjective tests like image quality. Chalk it up to bad timing. However, can you still use the 3DMark03 overall score for simple comparisons? I think so, if you understand how the benchmark works, are using it under the right conditions and provided drivers are no longer optimizing for the tests.
Tommy McClain
The "funny" part for me is that one of my long-standing objections a couple of years ago to using 3DMk01 was that it was based on the Max Payne engine, which was only one game engine out of the myriad game engines 3D games are built around. IE, running Max Payne wasn't going to help me determine how Q3 or UT were going to run on my hardware. So I feel that if anything, coming up with a synthetic vpu-stressing 3D benchmark makes all the sense in the world, especially considering the fact that vpus are continuing to distance themselves from cpus in terms of 3D functionality and capability. To my eyes being vpu-dependent serves to legitimize 3DMk03 as a real "3D" benchmark for the present and future, which is just what I think FM intended it to be.
I mean, if you want to get a measure of current in-game frame rates, then run the games you want to know about--ie, don't run Max Payne if you are interested in how your system runs Q3--run Q3. Or, if you are interested in looking at the performance of other games, run them and use either their inbuilt utilties to measure performance, or run something like Fraps. But if you are interested in investigating your hardware's features which are unsupported or else partially supported in your current games--just because you are interested--then something like '03 fits the bill perfectly, I think. This is the general purpose for which all synthetic benchmarks exist. Are all synthetics "bad" because they don't describe Q3 performance? I can't see how or why, since you are using the software for different purposes. Certainly, it is exactly the same thing as someone running Q3 in a given hardware environment and declaring that the Q3 results extrapolate to all other 3D games running on the same hardware--not so...
That's why I think the whole "anti-3dMk03" crusade is dishonest and misrepresentative on its face.
digitalwanderer
01-Aug-2003, 18:44
That's why I think the whole "anti-3dMk03" crusade is dishonest and misrepresentative on its face.
I agree, but I don't disagree with Kyle's take on benching entirely either. I think that pushing for in-game benching is a GOOD THING(tm), I just don't see why you can't use 3dm2k3 in addition to real world testing....the way it was meant to be used. :(
Reverend
01-Aug-2003, 18:47
Well i don't know about kyle, but shouldn't FM remove that slogan?
Like I said, I have suggested/told FM to drop this for the next 3DMark. Of course, it all remains to be seen how successful I will have been in this aspect :)
Park the Shark, read through this thread.
http://www.beyond3d.com/forum/viewtopic.php?t=5874
Most of the issues that you are curious about are adressed in it.
CorwinB
01-Aug-2003, 18:56
, or run something like Fraps
WaltC, don't forget that according to Nvidia, FRAPS is evil too... :)
The "funny" part for me is that one of my long-standing objections a couple of years ago to using 3DMk01 was that it was based on the Max Payne engine, which was only one game engine out of the myriad game engines 3D games are built around.
Actually I thought it was kind of nice because it gave some kind of relevance to real game and although you and others will disagree I believe you can extrapolate how your machine/card would do in other games(relatively speaking). Since 3DMark03 no longer uses a game engine it can have no relevance to any game at all. I'm not complaining about that. I like that fact it's a more of a synthetic GPU benchmark and that it puts more of a stress on the video card. However, I can understand why people would complain that it has no relevance to any games sans game engine. What Futuremark needs to do is build a games engine just for the 3DMark and then sell or give the engine to game developers. Provided they still want 3DMark to have relevance with games.
That's why I think the whole "anti-3dMk03" crusade is dishonest and misrepresentative on its face.
Agreed.
Tommy McClain
Well, I'd only say to look at the difference between the performance of games built around the UT2K3 engine--like Unreal2 and UT2K3. Same engine, big performance differences in some cases. Look at all of the various games built on the Q3 engine--big differences between the games. And using the Max Payne engine for 3DMK01 is not the same thing at all as running Max Payne the game, for a number reasons, as I'm sure you'll agree. At best you can say that 01 was "based on" the Max Payne engine, not that it performs identically with Max Payne the game.
Different games can use varying degrees of rendering precision available in the same game engine, for instance. Game engines themselves differ widely in performance because of such factors. Then there are differences in texture use, resolution, ad infinitum--all of which will impact performance to one degree or another.
Let's also not forget the differences in IHV's drivers when it comes to one game or another, even if based on the same engine. Is running Soldier of Fortune II going to help me figure out how my system will run Q3? Will running UT2K3 OpenGL allow me to extrapolate my system running Q3, etc.? That's what I'm talking about, basically, which is why I was never much impressed that running 3DMk01 would be "more realistic" because it was "based on" a game engine used in a single 3D game.
Hey, though, it's OK if we disagree on this one...;)
Bolloxoid
01-Aug-2003, 20:09
I agree, but I don't disagree with Kyle's take on benching entirely either. I think that pushing for in-game benching is a GOOD THING(tm)(
It is a bad thing if this ideology carries with it the simplistic and superficial mentality of reporting only the performance within certain applications and not making analyses, generalisations and predictions about the cards potential in general. My main objection with Hardocp's approach is just this, that they plan (and encourage others) to just give readers impressions of gameplay in certain select titles.
This kind of an approach itself is ok for less in-depth reviews, but more professional reviewers must continue to analyse the cards performance, both strenghts and weaknesses, beyond individual applications. Hardocp's suggestions are completely at odds with this.
A more sophisticated approach to reviewing video cards requires the understanding that the benchmark results are indicators of the cards potential in different situations, and based on these indicators you make educated inferences of the cards performance in general. I don't want reviewers to disregard this and just be content with describing gameplay in a couple of titles.
Well, I'd only say to look at the difference between the performance of games built around the UT2K3 engine--like Unreal2 and UT2K3. Same engine, big performance differences in some cases. Look at all of the various games built on the Q3 engine--big differences between the games. And using the Max Payne engine for 3DMK01 is not the same thing at all as running Max Payne the game, for a number reasons, as I'm sure you'll agree. At best you can say that 01 was "based on" the Max Payne engine, not that it performs identically with Max Payne the game.
Different games can use varying degrees of rendering precision available in the same game engine, for instance. Game engines themselves differ widely in performance because of such factors. Then there are differences in texture use, resolution, ad infinitum--all of which will impact performance to one degree or another.
Let's also not forget the differences in IHV's drivers when it comes to one game or another, even if based on the same engine. Is running Soldier of Fortune II going to help me figure out how my system will run Q3? Will running UT2K3 OpenGL allow me to extrapolate my system running Q3, etc.? That's what I'm talking about, basically, which is why I was never much impressed that running 3DMk01 would be "more realistic" because it was "based on" a game engine used in a single 3D game.
Hey, though, it's OK if we disagree on this one...;)
;)
Walt, I'm not totally disagreeing with you though. I agree that each game, each card, each machine and each drivers have varying effects on the performance. So yes, if you wanted to know the performance in Quake 3, then yes, it's BEST to test that game, that card, that machine and that driver. What I was trying to get at was that I don't believe the 3DMark overall score is totally useless in this regard. For those, that don't or can't get performance on the game or games they want, the 3DMark overall score can be a general indicator of performance. However, I will say that the latest cards do more to make this type of comparison difficult since their performance relative to each other is too close to call. So in a way, the latest hardware is making it more and more difficult, if not impossible, to use the overall score as a general indicator of performance in games. Eventually, I believe that if both and ATI and NVIDIA continue this route, then yes the overall score will become useless and Futuremark will either have to abandon it or change it in a way where it will continue to have relevancy.
Tommy McClain
Myrmecophagavir
03-Aug-2003, 00:07
WaltC, don't forget that according to Nvidia, FRAPS is evil too... :)
Really? What do they have to say about it?
WaltC, don't forget that according to Nvidia, FRAPS is evil too... :)
Yes, I saw that, too... I think nVidia's position on benchmarks is pretty clear--as long as a benchmark reflects what nVidia wishes it to reflect, it's fine. When it doesn't it's banished to the "enemies" list :D What an uncomplicated view nVidia has of the 3D industry--that all of the industry's software institutions exist as nothing more than extensions of nVidia, and if not they serve no purpose (ducking and running...)
StealthHawk
04-Aug-2003, 03:39
WaltC, don't forget that according to Nvidia, FRAPS is evil too... :)
Really? What do they have to say about it?
There's a thread here somewhere. Basically NVIDIA said it questions the accuracy of FRAPS as well as the accuracy of the people using it.
Since 3DMark03 no longer uses a game engine it can have no relevance to any game at all.
So no game, now or ever, will use pixel shaders?
There is a terrifying trend where people think that if it isn't an engine, it has nothing to do with games. Yet what 03 does is give a general overview, which is GREAT! That's exactly what you SHOULD want! Telling you how a card performs in a highly optimised enviroment which is coded in such a way as to help the card is about the worst way to benchmark imaginable, yet is the way so many people would have us go.
A benchmark which suggests how a card will perform in all things is far more useful than something that tells us how it will in one (followed by the reviewer telling us this reflects all).
StealthHawk
06-Aug-2003, 06:22
Since 3DMark03 no longer uses a game engine it can have no relevance to any game at all.
So no game, now or ever, will use pixel shaders?
There is a terrifying trend where people think that if it isn't an engine, it has nothing to do with games. Yet what 03 does is give a general overview, which is GREAT! That's exactly what you SHOULD want! Telling you how a card performs in a highly optimised enviroment which is coded in such a way as to help the card is about the worst way to benchmark imaginable, yet is the way so many people would have us go.
A benchmark which suggests how a card will perform in all things is far more useful than something that tells us how it will in one (followed by the reviewer telling us this reflects all).
No, I think many people here think that devs will code optimized shaders for NVIDIA hardware. Or like Valve, they will give the green light for IHVs to optimize the game's shaders. In other words, many people think that NV3x's "real power" won't be an issue.
So no game, now or ever, will use pixel shaders?
There is a terrifying trend where people think that if it isn't an engine, it has nothing to do with games. Yet what 03 does is give a general overview, which is GREAT! That's exactly what you SHOULD want! Telling you how a card performs in a highly optimised enviroment which is coded in such a way as to help the card is about the worst way to benchmark imaginable, yet is the way so many people would have us go.
A benchmark which suggests how a card will perform in all things is far more useful than something that tells us how it will in one (followed by the reviewer telling us this reflects all).
Right, and the performance differential between 3D game engines can be so great, and so conditional, that there is simply no substantive difference between comparing two game engines against each other and comparing a synthetic benchmark with a game engine. If it is true that a synthetic benchmark does not equate to a game engine, it is also true that one game engine does not equate to another game engine. When a benchmark professes to be based on a game engine there's the real problem of just how loosely that basing is, just to name one problem with the concept. If the benchmark takes serious liberties with the engine in order to meet the needs of the benchmark then the claim has no foundation in the first place.
Also, benchmarks which claim to be based on game engines are of necessity backwards looking rather than forward. For instance, in the two years or so that people used 3dmk01, which was promoted as being "based on" the Max Payne D3d engine, the assumption had to be that game engines, API's, and 3D hardware did not progress during that period in order for the "game engine" theory to have any merit relative to the benchmark. But of course the industry sits still for no benchmark--heh--and hence 3dMk03 was created, which takes an entirely different approach.
Who wants to buy 2d games anymore, or run cpu-dependent software rendering? Everything is 3d these days. And 3d is most certainly the province of the vpu/gpu, and not the cpu. So the vpu/gpu dependency of 3dMk03 was inevitable, and we'll see the same sort of dependency manifest in games requiring DX9 and up to run. If you don't mind the fact that you can't describe how a given system will run HL2 by running Aquanox, or you can't describe your system's performance of Doom III by running NWN, etc., then running 3dMk03 should present no dilemma whatever.
So what's caused some people to forget that a benchmark is a benchmark and a game is a game, that it's OK to look at both, and that this is the way the way things have always been? IMO, nVidia's handling of its problems post GF4 is the root of the situation.
Colourless
06-Aug-2003, 16:42
I think that it must be said with regards to Max Payne and 3DMark2001, that 3DMark2001 came out before Max Payne and uses more advanced effects. Game Tests 1, 2 and 4 were nothing like Max Payne, and the high detailed version of GT3 did a number of things that Max Payne didn't.
Either Max Payne has one of the most flexible engines or all time, or Futuremark wrote significant amounts of custom code themselves.
Getting back towards the original point, I see two issues that have come up repeatedly.
1) [H]ardOCP's stance that Nvidia's pseudo-trilinear settings, and overriding of the user's preferences are acceptable.
2) Kyle Bennett's willingness to ban from his forums those who are willing to stand up to his interpretation.
digitalwanderer
09-Aug-2003, 02:59
Getting back towards the original point, I see two issues that have come up repeatedly.
1) [H]ardOCP's stance that Nvidia's pseudo-trilinear settings, and overriding of the user's preferences are acceptable.
2) Kyle Bennett's willingness to ban from his forums those who are willing to stand up to his interpretation.
One thing you forgot:
3) Most people seem to think this is bad/wrong. ;)
Sorry Dig, if I may call you that,
I thought that didn't need to be stated.
digitalwanderer
09-Aug-2003, 03:43
Sorry Dig, if I may call you that,
I thought that didn't need to be stated.
Feel free to call me what ya will, I know me moniker is a little lengthy. :)
I know it should be implied, but people from nVidia might read this thread and I'm afraid they all seem to be a bit rusty when it comes to the right/wrong, good/bad thing. ;)
Yeah I think you're right.
Looking back, I think it started getting obvious when the Gf4MX was announced-- a DX7 part with a name that made people think it was DX 8 or better.
Nowadays I don't trust anything they say about their cards, though I still think the Gf4 Ti cards were good products.
digitalwanderer
09-Aug-2003, 05:05
though I still think the Gf4 Ti cards were good products.
I totally agree with that one, I do love the GF4 ti series and still think they're great cards.
I have a busted pc to fix for a friend while he's on vacation next week, and I just pulled his GF4 ti4200 and swapped it for me V5 on me secondary rig and gotta admit it's a nice step up for that system. :)
vBulletin® v3.8.6, Copyright ©2000-2013, Jelsoft Enterprises Ltd.