PDA

View Full Version : What do you guys think of this Anand quote?


Dean
30-Sep-2003, 00:53
"Speaking of video card stuff, we're hoping to have the Half Life 2 benchmark soon to do an updated comparison with NVIDIA's latest drivers. I'm hearing that the latest Detonators NVIDIA is working on fix the issues that Gabe outlined in his list of complaints and improves performance significantly. It seems like the 5900 Ultra is now slightly slower than the Radeon 9800 Pro with AA/Aniso disabled, and slightly faster with it enabled; I'm hoping to see for myself shortly. I haven't heard anything about how the 5600/5200 perform with the latest drivers, but I'm guessing the performance isn't as competitive. "

Read the rest here: http://www.anandtech.com/weblog/index.html

If Nvidia can pull that huge rabbit out of a hat without butchering image quality, it would be a miracle.

Dave Baumann
30-Sep-2003, 01:14
Given the performance disparities in the HL2 rendering paths, and the types of performance gains shown under Shadermark 2.0 with the 52.xx'a my guess would be that's a talking about the standard path for ATI but the mixed mode for NV3x.

gkar1
30-Sep-2003, 01:33
Yup I was thinking the same. Just make HEAVY use of low precission calculations and throw in some "automatic shader optimizer" shaders throughout the scene and you can gain a lot more frames at the cost of image quality of course.

RussSchultz
30-Sep-2003, 02:19
Going to partial precision doesn't necessarily mean sacrificing image quality.

Adding 2+2 gets you 4 in 8 bits, 16, 32, 64, or 128 bits. If your shaders doesn't need the extra precision, you're not gaining anything by having it.

But, I'm suspicious of the numbers, also. None of the shader benchmarks seem to suggest that the 5900 should perform equal to or better than a 9800, at any precision.

WaltC
30-Sep-2003, 02:22
Agreed, he's talking about differing code paths and of course can't be bothered by such a "technical" distinction.

Also, this statement:

...I'm hearing that the latest Detonators NVIDIA is working on fix the issues that Gabe outlined in his list of complaints and improves performance significantly....

...simply means to me that he's "hearing" the same things nVidia initially stated about the 50.xx Dets that they later recalled and decided weren't so great for general use after all (of course, people are still using them anyway.) IE, he's "hearing" the same thing we all heard initially about the 50.xx's. So of course he has no first-hand knowledge of whether any of that might be true, and I guess he hasn't "heard" yet about all of the tests people across the Internet have been doing with the 50.xx's nVidia tried to convince Valve to use--tests which have not been favorable. So my guess is what he's "hearing" is coming not from people using the Dets, but direct from nVidia. Of course we all know that nVidia always tells the Gospel truth about its drivers, don't we? And so, Anand, of course, would have no earthly reason to doubt what he's "hearing".

I'm glad Anand's hearing is OK, but this kind of statement really indicates to me how far he's slipped. I mean, unless he just completely avoids the Internet entirely with the exception of posting to his own site, I find it really baffling how he thinks that what he's "hearing" is anything especially newsworthy, since we've all been hearing it for the last couple of weeks, and nobody's seen anything definitive yet--except Valve--who was so impressed they refused to use the 50.xx's. Including Anand--who admits to having seen nothing himself--apart from what he's "heard," of course. I'm beginning to think I liked Anand much better when he was 16...:)

I don't mean to sound abrasive, but this is the kind of gossipy, "Gee, listen to what I've heard," frivolous remark that is more at home in the Drudge Report than in a "major" hardware news and review site like AnandTech. Things are changing these days, and the fellows running the "old guard" better sit up and take note, else they find their assumed positions eroding before their eyes.

"I'm hearing"....oh, good grief. He doesn't even do us the dignity of telling us the identity of the little bird chirping in his ear...OK, sorry guys...I guess I'm so sick of reading nothing but empty fluff all year from nVidia and *some* of the sites which discuss its hardware that my irritation is bubbling over...(It's a good thing I don't read Anand or THG anymore on any kind of regular basis--I'd probably have a stroke...:))

Dave Baumann
30-Sep-2003, 02:22
There are reasons why it could be the case with FSAA enabled.

Joe DeFuria
30-Sep-2003, 02:25
But, I'm suspicious of the numbers, also. None of the shader benchmarks seem to suggest that the 5900 should perform equal to or better than a 9800, at any precision.

Unless, of course, there are lots of drop downs to integer precision....

MrBond
30-Sep-2003, 02:58
There are reasons why it could be the case with FSAA enabled.

Wouldn't the higher bandwidth and that big clock speed help out in this situation?

Of course, with perf increases such as these, and from nvidia, I have to wonder what was done to achieve such an increase. I hope valve sticks to their guns and doesn't buckle under any nvidia pressure...

Tagrineth
30-Sep-2003, 03:05
Unless, of course, there are lots of drop downs to integer precision....

...and considering how much HL2 relies on HDR...

nelg
30-Sep-2003, 03:06
Anandtech seems to put alot of effort in showing nV in the best light possible. I will give you and example that I commented on @ nVnews.....
What troubles me about that article is that Anandtech seems to use language that obfuscates the issue intentionally. Read the first two paragraphs and imagine that you have no knowledge of the current issues surrounding the GFX series of cards......

N.B. The bold parts are added my me and are my opinion as to how it should have been said..


quote:
--------------------------------------------------------------------------------
Well we are here at Computex, and the buzz surrounding NVIDIA and DX9 [particularly Half-Life 2] hasn’t let up. As a matter of fact, it is the hot topic of many under the table talks between video card manufacturers and their respective customers. Our latest mobile look into the latest and greatest for NVIDIA and ATI didn’t leave much room for argument on the DirectX 9 front (which is better ?). Meanwhile, the industry buzz is still debating about who has legitimate claim in this controversy. There are suggestions that ATI and Valve have been conspiring, and Valve specifically coded their image quality paths to ATI hardware (By whom other than nVidia?). This is really hard to validate since there are other DX9 games that show similar [though with less intense margins] results between NVIDIA and ATI hardware(isn't this the proof you are looking for?).

Due to all of this we turn back to the concept of our Forum articles: basically, inquiring about the thoughts and opinions of various manufacturers anonymously. While Half-Life 2 may be centralized toward the American market, DX9 is a concern for consumers, manufacturers, and programmers worldwide. Consumers are less likely to buy a certain graphic solution should he or she know that image quality and rendering abilities are inferior of its competitor. Those that have already chosen the “unfavorable” graphics solution will likely then be alienated. Manufacturers, therefore, are directly effected due to sales [or lack of] relating to the limitations of a graphic solution, in this case NDIVIA and ATI(why include ATI in this statement ?). Programmers, as we noted before, are frustrated on two different levels. First is the topic of resources, typically, developers have an optical frame per second range, which they try to hit on all graphic solutions. Regardless of the reasons behind it, ATI and NVIDIA graphic processing parts can’t hit the same frame per second range in an intensive image quality game, at least for now(should this not say that nVidia cards cannot reach the same performance as ATI ?). This means that to keep up to par with the competition, developers have resorted to coding special code paths for these frames per second ranges to be hit. Obviously, this means image quality settings need to be lowered and this directly points to our second conundrum: consumers not being able to enjoy the full DX9 experience the way the developer intended. Programmers are artists, and for this reason; they hate to see their effort and artistic talent go to waste. On the other hand, they also understand that not creating a special code path could possibly lead to low sales or even undercutting them.



--------------------------------------------------------------------------------

(which is better ?)

Why all this pussy footing around ?

T2k
30-Sep-2003, 03:39
Anand is almost the same crap like THG's... ok, just almost.
Nevertheless I stopped checking any of them long time ago...

3dilettante
30-Sep-2003, 03:48
Oh gee wiz, browsing those sites can still be fun! :)

WaltC
30-Sep-2003, 05:31
Nelg, that quote from AnandTech has got to be one of the worst written I've seen in recent memory...I can't recall the last time I've seen someone write so much and manage to say so little at the same time...:) (Oh, wait--I just read Kirk's interview on Firingsquad, so I should probably take that back.)

Besides the rather large inconsistencies in logic you point out, the general wording of the quote and phrasing indicate an author not really comfortable with English. I'm assuming Anand did not write this. Things like "effected" instead of "affected" and "optical" frame per second range instead of "optimal...," seem to bear this out as well. The phrasing is just stilted and pretty poor as a result. I'm really not sure that the author was intentionally being evasive--what comes out for me is that in English his thoughts are pretty muddy, basically, and he probably thinks he's said something much different from what he's written. Just a guess...Since the story was from Computex it sounds like a stringer piece somebody sent them which they just printed verbatim without bothering to read or edit it first (they could have addressed the spelling errors and corrected some of the punctuation and polished up some of the phrasing, at the very least, before running it.) But I guess the AnandTech "editor," whomever that is, was indisposed at the moment--but apparently not the person who posted it to the site--whom I have to guess didn't understand it any better than I did and so just reprinted it as it was received. (I *really* hope this was not an edited result...:))

Pete
30-Sep-2003, 06:11
nelg, Anand's first paragraph was pretty clear, as I think it was written right after Andrew Ku's review of ATi's and nV's mobile parts (9600 and 5600), in which ATi literally spanked nVidia. You're really picking nits there.

The second paragraph is somewhat evasive, though. Again, people like Anand can't really afford to malign a product without specific and multi-sourced proof. I don't think beta benchmarks constitute enough proof for him to firmly conclude that nVidia sucks all around and to recommend ATi to all his readers.

His review of the XT should be very interesting, though. I'm curious to see what lessons he's taken from his 5800 review, and how his new writer does. Not to mention we have a bunch of new benchmarks for him to try, and it appears he's updated his test suite significantly.

madshi
30-Sep-2003, 10:30
There are reasons why it could be the case with FSAA enabled.
Which reasons?

The ATI cards do that centroid mode of FSAA for free on HL2, while the NVidia cards need to do additional clamping work for FSAA in a DX9 shader. So the ATI cards should have a theoretical performance advantage in FSAA. What am I missing here?

THe_KELRaTH
30-Sep-2003, 11:00
There are reasons why it could be the case with FSAA enabled.
Which reasons?

The ATI cards do that centroid mode of FSAA for free on HL2, while the NVidia cards need to do additional clamping work for FSAA in a DX9 shader. So the ATI cards should have a theoretical performance advantage in FSAA. What am I missing here?

While I've read that there's a solution for Centroid AA for the FX range I've not seen any confirmation that Nvidia are implementing it. With Nvidia's reputation for cutting every corner they can just to get reasonable fps can you believe that they would then include a performance hit feature?

nelg
30-Sep-2003, 12:37
nelg, Anand's first paragraph was pretty clear, as I think it was written right after Andrew Ku's review of ATi's and nV's mobile parts (9600 and 5600), in which ATi literally spanked nVidia. You're really picking nits there.



Only because you know the realative performance between the two parts. Try reading it with the perspective of having no prior knowledge.

OpenGL guy
30-Sep-2003, 19:25
While I've read that there's a solution for Centroid AA for the FX range I've not seen any confirmation that Nvidia are implementing it. With Nvidia's reputation for cutting every corner they can just to get reasonable fps can you believe that they would then include a performance hit feature?
I believe the application should be sending down the appropriate shader (i.e. one that clamps texture samples) as the driver doesn't export this feature.

K.I.L.E.R
01-Oct-2003, 05:07
Only because you know the realative performance between the two parts. Try reading it with the perspective of having no prior knowledge.


True, I bet anyone without prior knowledge of what's happening would think:

"Hmmm, this FX costs $1100 and this Ati card costs $700, I think I read about this at *insert fanboi site*. The FX was kicking ass and considering it's more expensive than the Radeon how can I possibly go wrong?"

This is the type of attitude a majority of consumers have, believe it or not. When I ask why a friend or whoever why they bought an nVIDIA card (NV3x) that would be the usual response or this one sometimes:

"nVIDIA is the best".

Lame responses eh?

These guys have no knowledge of what has happened to nVIDIA nor what they have done. These guys think nVIDIA's drivers are perfect and have awesome IQ, performance and are bugless. :lol:

cthellis42
01-Oct-2003, 09:00
Lame responses eh?

Sadly, no. I could almost ACCEPT that as it's just typical "uninformed consumerism." (Of course if they kept you aware of upcoming purchases, you may have wanted to inform them more. ;) )

The worst I get are from people who have no interest in reading up or checking anything, yet somehow remain die-hard fans. "I had a problem with an ATi card, therefore ATi sucks." "ATi makes horrible drivers (referring to, like, Rage128-era stuff, even)." "I know a guy who had problems in a MMORPG with his ATi card, so ATi sucks." "nVidia has more marketshare so they're the best!"

<sigh>

Funny thing is I would debunk their comments and explain how one SHOULD make smart hardware-buying decisions, and all of this was easily done WITHOUT talking about anything the FutureMark issues on up.

I'm pained a lot more by people who do no research, don't care, yet can be amazingly stubborn regardless.

K.I.L.E.R
01-Oct-2003, 09:28
Those are the type of people that you should leave alone, so they can rip themselves off.
After all, it's a free country. ;) :lol:

T2k
01-Oct-2003, 16:31
Those are the type of people that you should leave alone, so they can rip themselves off.
After all, it's a free country. ;) :lol:

KHm... which one? ;)

hjs
02-Oct-2003, 19:59
http://www.anandtech.com/weblog/index.html


Here are some Half Life 2 numbers for you to look at; they were provided by a reliable source, but I could not verify anything myself so take them with a grain of salt. ATI was running in their DX9 codepath and the mixed mode codepath was used for NVIDIA. No AA/AF was enabled and we're looking at 1024x768 scores:

Joe DeFuria
02-Oct-2003, 20:13
Lol...

1) He has a "reliable source"....someone with HL2 access...
2) They have the "in" with nVidia...given the fact they were allowed to bench the NV38

I'll give 1 guess as to which company the "reliable source" works for...

:roll:

hjs
02-Oct-2003, 20:22
Lol...

1) He has a "reliable source"....someone with HL2 access...
2) They have the "in" with nVidia...given the fact they were allowed to bench the NV38

I'll give 1 guess as to which company the "reliable source" works for...

:roll:

You forget about the acces to the newest drivers, How else they get this score :roll:

Joe DeFuria
02-Oct-2003, 20:24
Looks like the [H] / nVidia divorce is now complete.

It just saddens me that nVidia found a new b*tch in such short a time...

CorwinB
02-Oct-2003, 20:24
No AA/AF was enabled and we're looking at 1024x768 scores:

Err... Wasn't the 9800 supposed to be CPU-limited at 1280 anyway ?

And I agree, before :
- everyone gets access to those miracle drivers
- people apart from Anand get access to NV38 cards
- people get a chance to look at the various butcheatoptimizations performed

people shouldn't get too excited... :P

WaltC
02-Oct-2003, 21:24
Lol...

1) He has a "reliable source"....someone with HL2 access...
2) They have the "in" with nVidia...given the fact they were allowed to bench the NV38

I'll give 1 guess as to which company the "reliable source" works for...

:roll:

Followed your response from the other thread, Joe--Thanks.

Yes, who on earth might the "reliable source" be, if not nVidia? Who but nVidia would attempt to benefit from benchmark scores in HL2 contrasting ATi's products against nVidia's, running two different code paths?

I might imagine that a third party not nVidia would want to see a bunch more scores contrasting a bunch of different code path performances other than a noAA/noAF glimpse between the 5900U and the 9800P--and at a single resolution of 1024x768, no less. In short, I can see nothing of benefit Shimpi is providing to his readers with the publication of such information except as it is intended to promote an IHV--obviously, nVidia, in this case.

Personally, if Valve's source code has indeed been stolen, or else the "source code" is found to be fraudulent with derogatory notes in it about ATi, I believe if I was Valve I'd see about hitting Shimpi with a subpoena to force him to divulge his source for this "benchmark." It's possible there is a direct link to the "source code" stuff right there. Obviously, he has no intention of voluntarily releasing that information--which really makes you wonder why someone with a "reliable source" for such info, with nothing to hide, would mind naming his source. That makes Shimpi as guilty as his source as well as complicit in anything illegal that might be taking place here.

You know--a little sparring back and forth between companies in a restrained and civilized fashion, is one thing--but this is getting really nasty...I hate to say it but this has gone beyond honest competition, and I'm afraid we might be witnessing industrial espionage in its purest form here.

Dean
03-Oct-2003, 16:43
Anand has a new weblog regarding part 2.

http://www.anandtech.com/weblog/index.html

PatrickL
03-Oct-2003, 17:30
I had fun reading the weblog and the doubts about benchmarking with tombraiders. I bet tons more people are playing Tomb raider (despite not being a successfull title) than the total players of AM 1 to AM 3 :)

I think working on including an EA game in the benchmark suite will be the next logical step to be fair to everyone :roll:

Dean
03-Oct-2003, 17:44
I had fun reading the weblog and the doubts about benchmarking with tombraiders. I bet tons more people are playing Tomb raider (despite not being a successfull title) than the total players of AM 1 to AM 3 :)

I think working on including an EA game in the benchmark suite will be the next logical step to be fair to everyone :roll:

Yeah thats exactly how I picked it up. How do they regard a game successful? By reviews or sales? Last I heard the game was a fairly good seller even though most critics panned it.

AJ
03-Oct-2003, 19:36
"...do an updated comparison with NVIDIA's latest drivers... "

This is a question I've been wanting to ask for quite a while: What is the deal with web sites and Beta Drivers anyway? During the last year it should have become obvious to everyone that beta drivers are leaked on purpose and may or may not have serious issues with regards to quality vs. performance. I don't think anyone in their right minds can really claim that the correlation between beta drivers and final drivers would not be subject to possible changes. Yet to me it seems that many websites that I personally would categorize as respectable still jump way too easily to the 'beta drivers' bandwagon?

With beta drivers it's impossible for the readership to verify in any manner whether there are any issues or not. It's also virtually impossible for any website to do the required QA for testing these drivers (unless you happen to have a few weeks at your disposal and have capable engineers working for your website).

At the same time there are little if any reasons that I can think of why one could not wait until the public release (or even preferably the WHQL'd public release).

As far as I can tell the only reason to use beta drivers is the 'exclusivity' that you can get with them.

Could someone please explain to me why even very respectable websites so easily use unavailable, non-public beta drivers?

Cheers,

AJ

Joe DeFuria
03-Oct-2003, 20:05
Could someone please explain to me why even very respectable websites so easily use unavailable, non-public beta drivers?

Because that what certain IHVs recommends they test with...and they don't want to "upset" the IHV for fear of retribution. Retribution being "you might not be on 'our list' in the future to get early access to hardware...so your web site is gonna lose some hits...."

I'm with you. I wish web sites would just not even entertain the thought of publishing benchmarks on drivers that are not yet available for public download from the vendor.

How about THIS for an ironic quote....I'll add emphasis:

http://www.anandtech.com/video/showdoc.html?i=1544&p=4

NVIDIA clearly spoiled the preliminary introduction of the Radeon 8500 back in August with the release of their Detonator 4/XP drivers. We were told by NVIDIA that these new drivers would not only improve performance, but that they would be made publicly available the very same week we tested with them. Obviously, that didn't happen, and it ended up taking another month before the drivers were released. The performance gains were tangible, but the drivers weren't fit for release when NVIDIA provided them to the press and honestly shouldn't have been used. Hindsight being 20/20, we made a promise to ourselves that we would not allow any further performance enhancing drivers to be used in our video card reviews unless we could make the drivers publicly available to our readers immediately.

Perhaps hindsight isn't 20/20 for everyone... :roll:

jimbob0i0
03-Oct-2003, 20:16
AJ on that note (and I agree with you) this will stun you.....

Evan Lieb has been defending Anand's "review" of the NV38 and a few parts stand out (note I can't be bothered finding the right link again so I will paraphrase slightly although they are within the nvnews threads):

"Why use public drivers on a non-shipping card?"
"The performance enhancements seen in beta drivers often are included in final drivers so we might as well test the betas"
"Getting an exclusive on the NV38 is not about hits"

Hmm... in the style of Dogbert - BAH!

hjs
03-Oct-2003, 20:29
AJ on that note (and I agree with you) this will stun you.....

Evan Lieb has been defending Anand's "review" of the NV38 and a few parts stand out (note I can't be bothered finding the right link again so I will paraphrase slightly although they are within the nvnews threads):

"Why use public drivers on a non-shipping card?"
"The performance enhancements seen in beta drivers often are included in final drivers so we might as well test the betas"
"Getting an exclusive on the NV38 is not about hits"

Hmm... in the style of Dogbert - BAH!

i believe that was in the nvnews forums, the Tom or the Anand thread in the nvidia cards part.

g__day
03-Oct-2003, 23:32
Okay - Aquamark 3 raised this same conumdrum - you have a game that can lower shader models from Dx9 all the way to Dx6 if your "DX9" class card can't cut it.

We are now told HL2 with NV35 use a mixed path - that sounds like a DX8 / DX9 mix rather than a pure DX9 path with Partial Precision hint spread liberally.

If this is the case what constitutes a fair analysis of a benchmark? If one card runs with much better effects and the other is faster with less effects what does a reviewer say? ATi gives you speed and effects, NVidia is slower with the same effects or faster without them. Trouble is then that you are left out trying to work out hard large the delats are in terms of both speed and effects.

digitalwanderer
03-Oct-2003, 23:43
AJ on that note (and I agree with you) this will stun you.....

Evan Lieb has been defending Anand's "review" of the NV38 and a few parts stand out (note I can't be bothered finding the right link again so I will paraphrase slightly although they are within the nvnews threads):

"Why use public drivers on a non-shipping card?"
"The performance enhancements seen in beta drivers often are included in final drivers so we might as well test the betas"
"Getting an exclusive on the NV38 is not about hits"

Hmm... in the style of Dogbert - BAH!

i believe that was in the nvnews forums, the Tom or the Anand thread in the nvidia cards part.
The bottom of this page (http://www.nvnews.net/vbulletin/showthread.php?s=&threadid=18863&perpage=25&pagenu mber=3) and the poor fellow gets shredded by the next one. :(

3dilettante
04-Oct-2003, 02:35
It looks unfortunate to me that in both that thread and one in the Anandtech forum that there isn't much discussion about the poor methodology behind using the deadly combination of unreleased video card, drivers, and prescott cpu.

In fact, in the case of the Anandtech thread, that is the one concern that is seemingly not given a proper excuse, though there are plenty of justifications with the layout of the article, the lack of information, etc.

I guess the lack of sound testing methodology is less flashy than supposed IHV bias and alleged selling out, but that's the one thing that I found disagreeable.

Should Anandtech have a bias, the next few articles will clearly show it, and I will give them the luxury of having a bias: I just won't listen to them.
However, I really don't like bad tests and the dubious conclusions drawn from them.

Putting together a list for the number of unknowns, we can see just how useful the review was.

Radeon 9800XT not being in general availability (fine if the article was only about that and it was the only unknown...)

NV38 not released. No specifications. Non-released drivers. No review of functionality. Could be faster than actuality, slower, equivalent, invalid.

Prescott not released. Few specifications. Unknown or limited performance data. No comparison to known processor, could be faster or slower or equivalent or possibly invalid.

System specs sparse. Unknown or limited performance data. No comparison to known setups, no reproducibility outside of Anandtech. Could be faster, slower, equivalent, invalid.

Test settings were not exhaustive or varied. Unknown or limited performance data. Due to lack of system data, unreproducible.
Could be faster, slower, equivalent, or invalid.

Test software of uncertain value. Granted there are few dx9 games out at the moment, it goes to the heart of the matter to test them. Software anomalies are quite possible on unreleased hardware and drivers.
Could be faster, slower, equivalent, or invalid.

There could be any combination of the above faster/slower/equivalent/invalid in respect to actuality(or what can be passed as a reasonable approximation), and there are five areas in this list alone. There could be additional areas and additional outcomes, but the numbers present are serious enough.

4^5 is 1024 unique combinations of valid and invalid points for this article, and vague hinting doesn't make it any more certain.

Unless they redo the entire thing and strip out a whole bunch of unknowns from each test run, the uncertainty is still there, and no excuse is going to change that. No amount of work or honorable intent is going to magically make up for a bad test basis, and this article had that to a great extent.

I don't give a rat's ass what IHV they like to cuddle with, but they sure better stop screwing with their tests.
Even biased tests tend to be more certain than what I've seen, since at least most are willing to show the systems they used.

At present, there is no restriction of unknowns, reproducibility, rigor in testing, nor transparency in the collection and representation of data. And no, saying "these numbers are great, but there may be image quality concerns which we might address later" does not count as being transparent.

edit: grammar/clarity

AJ
04-Oct-2003, 10:33
Because that what certain IHVs recommends they test with...and they don't want to "upset" the IHV for fear of retribution. Retribution being "you might not be on 'our list' in the future to get early access to hardware...so your web site is gonna lose some hits...."


I tend to agree. As far as I can tell the way things typically pan out is that a manager from a major IHV gives you a call and urges you to use the 'latest drivers' because they give the 'correct' performance profile.

And well, you don't want to mess that relationship.

[/rant mode on] But from a journalistic point of view this is simply unbearable. I mean afterall any decent journalist that carries a responsibility towards his readership should simply run the performance figures with publicly available material and then make a note along the lines of 'please observe that these figures may change in the future when new drivers became available'.

Otherwise it's possible if not even likely that you're simply misleading your readership. But if you really, really want to do that there's no reason why you could run the default numbers with publicly available stuff and then include grey out versions of non-publics with the message and that in the future this might be a possible performance scenario (Lars Weinard has been doing this lately and I kind of like it too [keep it up Lars, you're doing the right thing! :)]).

BTW: The same in my opinion goes for the benchmark tools also. There's now way that one can do proper technical due diligence on the benchmark tools and I don't think that it's very wise to jump into conclusion based on something that's not available to generic public (Doom III comes to mind, but this goes for HL2 also).

The part that I personally find quite frustrating is the fact that none of this is really rocked science. I find it constanty amazing that even major websites do not really seem to care much about their journalistic integrity and let the IHV's influence their practices.[/rant mode off]

Cheers,

AJ

AJ
04-Oct-2003, 15:23
AJ on that note (and I agree with you) this will stun you.....

Evan Lieb has been defending Anand's "review" of the NV38 and a few parts stand out (note I can't be bothered finding the right link again so I will paraphrase slightly although they are within the nvnews threads):

"Why use public drivers on a non-shipping card?"
"The performance enhancements seen in beta drivers often are included in final drivers so we might as well test the betas"
"Getting an exclusive on the NV38 is not about hits"

Hmm... in the style of Dogbert - BAH!

*grin* Yeah, while you're at it, why not just print out any numbers that 'seem right' without any verification what so ever? Basically that's what it all boils down to. If you use testing software or drivers without being able to back up your claims then you could just as well print out numbers based on your own guess work and tell your readers that 'you have a hunch' that the performance might in the end look a bit like this.

*dry grin* Maybe I'm being a bit harsh here, but for heavens sake, there's so many very good alternatives that I'm just a tad annoyed with the 'we don't give a damn and why worry about integrity' approach to news that so many sites have.

The testing tools and the drivers being tested _have_ to be able to deal with public scutinity. Otherwise it's not really journalism, just fanboys flattering their idols...

Having said that I do think that there are constructive ways of dealing with issues. Lars Weindard's current "grey-out approach" is a very decent way of dealing with the issue and actually delivering more information to readers without misleading anyone. And yes, I do agree that HL2 performance as well as DoomIII performance are very important for the readership to know about and preferably before they've upgraded their machines. But this can also be executed properly with correct disclaimers and explanations so that the readers are not misled to any direction but rather can use that information as they see fit.

Cheers,

AJ

jimbob0i0
04-Oct-2003, 17:02
*grin* Yeah, while you're at it, why not just print out any numbers that 'seem right' without any verification what so ever? Basically that's what it all boils down to. If you use testing software or drivers without being able to back up your claims then you could just as well print out numbers based on your own guess work and tell your readers that 'you have a hunch' that the performance might in the end look a bit like this.


shhh AJ quiet... you'll give them new ideas.....

Actually as far as Lars is concerened.... I stopped reading THG long ago due to uh 'disagreements' with certain things published there.... but recently Lars has been drawing me back for his reviews alone.... I definately agree with his approach and fins it very informative.....

It's just Anand's "don't wworry about IQ... I think it's okay otherwise I wouldn't print these... oh and lookout for part 2 of this where I show you how much of a discrepancy in IQ there really is"....

Based on past experience of *that* site TBH I really cannot forsee AT redoing the FPS scores in 'part 1' if they publish NV as having worse IQ.... and then NV/fanboys point at teh 'part 1' graphs and say - look we get AA/AF for less than free and beat ATI in all *current* games!! Buy that $500 card now!!11!!!1!

CorwinB
04-Oct-2003, 19:07
Actually as far as Lars is concerened.... I stopped reading THG long ago due to uh 'disagreements' with certain things published there.... but recently Lars has been drawing me back for his reviews alone.... I definately agree with his approach and fins it very informative.....


Agreed, I just wish the Good Dr. would stop using Lars' reviews as an editorial platform for sharing his "enlightened" views with us.

jimbob0i0
04-Oct-2003, 19:24
Actually as far as Lars is concerened.... I stopped reading THG long ago due to uh 'disagreements' with certain things published there.... but recently Lars has been drawing me back for his reviews alone.... I definately agree with his approach and fins it very informative.....


Agreed, I just wish the Good Dr. would stop using Lars' reviews as an editorial platform for sharing his "enlightened" views with us.

Yes I can't *believe* he did that... uh yes sorry I can believe that... just find it unfortunate....

Hey who wants to start a pool on when Lars will leave THG for a site of his own or to join another site? :twisted:

Quitch
06-Oct-2003, 11:06
Don't knock it. I wish more reviews would have some comic relief in the middle of all the graphs.

AzBat
06-Oct-2003, 20:37
Hey who wants to start a pool on when Lars will leave THG for a site of his own or to join another site? :twisted:

How about Lars and Brent join up with Beyond3D or start their own site? Would help me forget there is a THG or [H].

BTW, how about we use the money from the pool as an incentive to Lars and Brent to leave their sites? Hehe ;)

Tommy McClain

digitalwanderer
06-Oct-2003, 21:11
I was thinking Lars & Pelly for some reason... ;)