Anand's Mobility Radeon 9600 vs GeforceFX go5650 review. . .

[Ostsol]

http://www.anandtech.com/mobile/showdoc.html?i=1866&p=1

I don't know about you, but his use of the HL2 benchmark seemed kinda silly, to me. . . I know that many sites are striving for apples-to-apples comparisons, but no one is gonna play HL2 in DX9 mode on an FX5600. . . The least Anand could have done is also included the mixed mode or DX8 benches as well. . .

 

[Doomtrooper]

Why its touted as a DX9 chip ?? People will buy the card just for that reason...and should KNOW.

 

[WaltC]

Re: Anand's Mobility Radeon 9600 vs GeforceFX go5650 review.

http://www.anandtech.com/mobile/showdoc.html?i=1866&p=1

I don't know about you, but his use of the HL2 benchmark seemed kinda silly, to me. . . I know that many sites are striving for apples-to-apples comparisons, but no one is gonna play HL2 in DX9 mode on an FX5600. . . The least Anand could have done is also included the mixed mode or DX8 benches as well. . .

Well, I don't agree with the "5600-is-not-expected-to-play-DX9-games" view, but I do agree with the idea that looking at laptops like this is kind of silly. But that's just me...I've got an inborn aversion to laptops in general...like integrated sound...and a few other well-established prejudices...

But the thing is that nVidia is publicizing and pushing the 5200 and the 5600's as "DX9" hardware. I mean some of the recent publicity I've seen nVidia launch about "Driving DX9 down to the low end of the market" with the 5200 is just plain laughable. Or at least it would be, if it wasn't such a flagrant attempt to con whomsoever may be conned. So I think under those circumstances, this kind of thing is probably OK...and may appeal to someone who might be tempted by the advertising to think that a "laptop with a 5600 that will also run DX9 games" is just the ticket...

 

[nelg]

http://www.beyond3d.com/forum/viewtopic.php?t=7902

 

[Ostsol]

Well, I never said that it wasn't expected to play DX9 games. I simply said that no one will ever play HL2 with the DX9 path on that card. It's simply a matter of what provides decent performance for the card -- and the DX9 path ain't it. . . I wouldn't recommend the FX5650Go to anyone (based not only on the HL2 numbers), but those who already bought into it (or for whatever reason are stuck on getting it) might want to know if there is any hope at all for them in HL2.

 

[Natoma]

Ostsol,

The title of the article was "Taking on DX9". For an end user looking to purchase a laptop that can play DX9 titles, this is most certainly a pertinent article. Not every DX9 game is going to have mixed mode for NV3x performance.

If I were in the market for a laptop and I wanted DX9 performance, I'd want to know the true look of the market, not what I should be expected to run with my system. Whether or not the 5650s will ultimately be run in DX8, if I'm looking for a DX9 system, this review makes sense.

 

[Ostsol]

I still think it is wrong, at least in the case of HL2. HL2 offers multiple render paths, so why is it so wrong to use them in a review? If you want to show the card's performance in DX9 render paths, then use the HL2 one, but also provide the results for the other paths. It doesn't matter when a synthetic review shows crappy DX9 performance, because you have no choice in that matter.

Synthetic benchmarks are meant to show raw performance in specific features. However when people look at a game benchmark they see it as how the video card will perform in that game, regardless of whether it is representative or not about the practical settings for playing the game. Heck, even in Beyond3D's TR:AoD comparision the game's default settings for each card were benched.

Like I said, apples-to-apples has its place, but ultimately what is practical is much more relevant.

 

[Natoma]

Again, if I'm Joe Schmoe user and I see the latest computer buzzword, i.e. DX9, I want to know how my potential new toy will perform with DX9. I don't want to purchase a laptop with the DX9 graphics card 5650 and find out that playing DX9 games is unplayable on my system, then be forced to move down to DX8.

If you want to debate "practical" settings then that's fine. Unfortunately, the great swath of computer users are not that savvy and only see buzz words. I know if I were a regular end user, I'd be pissed if my newly minted DX9 $3000 laptop couldn't play the DX9 games in DX9 settings. I'd assume it was defective and send it back, or I'd see reviews and wonder why they didn't test my DX9 laptop with DX9 games.

It is the responsibility of the reviewer to tell users how their system will perform in DX9 if that is the stated intent of the article. Again, not every game will have mixed-mode for NV3x chips. Not every game will automatically default NV3x chips to DX8. Because the stated intent was "Taking on DX9" then I think it was highly responsible of them to use DX9 only render paths and DX9 only games and benchmarks. If I were in the market for a DX9 laptop this article would be perfect as it would show me performance in the DX9 lens and nothing more, nothing less.

 

[demalion]

Hl2 in DX 9 mode wasn't the only benchmark used, it was one of many. I really can't make sense of your objection, Ostsol. It was used to represent direct DX 9 feature delivery performance. A role 3dmark 03 has been excluded from due to propaganda and lack of technical savvy by most "recognized" sites, but that is still important to inform consumers about (in a case of "better late than never").

Your commentary only makes some sense to me if it was the only benchmark and benchmark mode used, but doesn't make any sense to me for the actual situation in the article. Your words end up serving as a demand to keep consumers ignorant of DX 9 performance characteristics of the chips, AFAICS.

Myself, I think a dedicated DX 9 benchmark could have served, and then HL 2 could have been tested exactly as you say. But that's not the situation we have, and HL 2 is required to fulfill that role specificaly as the result of a long campaign of disinformation and pressure discrediting such standardized and dedicated benchmarks with the mantra "only games matter". This leaves the only alternative to what was done here, for the stance these sites have taken (i.e., without accepting accountability for independent evaluation of technical merit of their benchmarking), as keeping consumers in the dark. I don't currently see how that is a valid proposition given the situation.

 

[CorwinB]

I haven't had time to check the review, but :
- if the intent is to benchmark DX9 performance (using HL2 only as another benchmarking tool), then there is no point to benchmark with the DX8 path
- if the intent is to benchmark HL2 performance on a specific platform, then benchmarking the various paths makes a lot of sense, provided you explain the quality differences between them

 

[Ostsol]

CorwinB, regardless of the title of the review, not all of the benchmarks used were DX9 benchmarks. If they all were, I'd understand the review as measuring the video cards' performance in DX9 applications. Since this is not the case, there is no point to exclusively using a render path that will not be standard for that video card.

Your words end up serving as a demand to keep consumers ignorant of DX 9 performance characteristics of the chips, AFAICS.

Nowhere did I suggest that the DX9 path not be investigated as well. I only suggested that in addition to it, the reviewer add scores for the render path that people will actually be using for -playing- the game on that video card. Since this review is not exclusively a review of DX9 performance, I see no point in keeping this information from the reader.

 

[CorwinB]

CorwinB, regardless of the title of the review, not all of the benchmarks used were DX9 benchmarks. If they all were, I'd understand the review as measuring the video cards' performance in DX9 applications. Since this is not the case, there is no point to exclusively using a render path that will not be standard for that video card.

Again, if HL2 was used to measure HL2 performance (as a part of various tests aimed at showing performance under different loads and different game types), then I absolutely agree that using different render paths makes sense in a review (just like using different resolutions and/or AA/AF levels makes sense). But if HL2 is only used as a tool to show DX9 performance (like a synthetic tool could be used), then it's a bit silly to use the DX8 path, because what you want is an apple-to-apple comparison...

Of course, you could also decide to use DX8 path HL2 as a DX8 measurement, and DX9 HL2 as a DX9 measurement. Was that what you suggested ?

 

[demalion]

...

Your words end up serving as a demand to keep consumers ignorant of DX 9 performance characteristics of the chips, AFAICS.

Nowhere did I suggest that the DX9 path not be investigated as well.

I understand the principle, but I'm talking about the specific review and website.

I only suggested that in addition to it, the reviewer add scores for the render path that people will actually be using for -playing- the game on that video card.

I understand the principle, but the source of the problem is the site's stand on synthetic benchmarks. What I take issue with as far as what you choose to attack about the article is that using HL 2 in the way they did is perfectly valid if the purpose of the usage is accurately represented, and this is what they seem to have done. I don't think attacking that usage of HL2 is valid at all currently.

This is further strengthened by the issue that readers cannot test this particular benchmark themselves and investigate screenshots at the moment, and the article does not have the focus and effort allocated towards successfully investigating and representing those other paths in this regard (which, at least for HL 2, makes sense wrt to the NDAs I understand to be in effect).

This is indeed a valid usage for HL 2, and due to the circumstance of web "journalism" in general, it is the only tool that can currently successfully serve in this capacity to the perception of many raders. Your saying this usage is not valid ("wrong", etc.) is what continues to not make sense to me, not that the situation couldn't be better all around. Also, the HL2 performance comparison article has already been done, and it was titled accordingly.

Since this review is not exclusively a review of DX9 performance, I see no point in keeping this information from the reader.

But they didn't keep information from the reader, they used many other benchmarks and added one specific set of information from HL 2 on top of that. Again, if they had not explicitly stated it, I would understand, but they did; if they had only used the one representation for chip comparison, I'd understand, but they didn't.

 

[Ostsol]

You appear to be trying to make the issue more complex than it is. . .

HL2's use in that review is not much different than that of a synthetic benchmark. Synthetic benchmarks have a single render path since they are meant to benchmark all hardware tested on an equal basis, utilizing the same featureset. In games that is not always the case. I do agree that there would most likely be a quality difference and most likely even a difference in the effects presented between the DX9 path and the default path for the FX5600; it would certainly not be an equal comparison. However, it establishes the review as benchmarking HL2 as a game.

 

[demalion]

You appear to be trying to make the issue more complex than it is. . .

I don't think so. I'm going to go point by point, to specify exactly where I view the disconnect.

HL2's use in that review is not much different than that of a synthetic benchmark. Synthetic benchmarks have a single render path since they are meant to benchmark all hardware tested on an equal basis, utilizing the same featureset.

That is exactly how you measure feature support. It is indeed fairly simple so far, though I think I'll point out that "have a single render path" should be more like "use a single render path" for the one specific type of benchmarking I take you to mean, though I think this is what you meant.

In games that is not always the case.

So far I'm not seeing what complexity I'm adding, because I don't see disagreement with these comments.

I do agree that there would most likely be a quality difference and most likely even a difference in the effects presented between the DX9 path and the default path for the FX5600; it would certainly not be an equal comparison.

Another point that doesn't seem to contradict or be contradicted by my statements, and does seem relatively simple.

However, it establishes the review as benchmarking HL2 as a game.

Here is the issue...this characterization makes no sense to me for this article.

What they seem to establish to me is that they are benchmarking a game using DX 9 effects, for the purpose of examining DX 9 performance comparison. In trying to make sense of your comment in the context of the article's statements, I end up with concluding that you are proposing a divide between "synthetic benchmarking" and "gaming" that relates to the fallacious separation without examanition of technical merit that I was referring to earlier. Being a "synthetic benchmark" doesn't mean "unrelated to gaming", it means that it successfully isolates a component for examination...that component can quite certainly be specifically related to running a game, but its isolation allows information gained to be more universally applicable to evaluating other games accurately. What you seem to end up proposing is that usage of "synthetics" for game "benchmarking" is a bad thing, as if that makes indisputable sense, though it seems directly contradicted by the observation that this game is using this feature support for gaming and the isolation of DX 9 featureset as effectively as a "synthetic" benchmark is the result of that.

Perhaps this is due to a different interpretation of what the article said, and this differing interpretation is leading to misunderstanding. I'm going to quote what comes to my mind of what they said in relation to this, and perhaps you can explain to me how your characterization makes sense in view of it, and outside of the understanding I'm taking away from it above:

HL 2 page[/url]]From the code standpoint, NVIDIA’s NV3x has had to take a lower code path in HL2, which Valve had to incorporate specially into their design, so that reasonable game play would be achieved on NV3x based cards. You can read more details relating to Half-Life 2 in Anand’s coverage.

In our mobile coverage, we forced on the DX9 code path, 32-bit depth, tri-linear filtering, and other high settings for which NVIDIA hardware would not automatically allow. For this review, we ended up deciding to keep AA and AF turned off because Half-Life 2 is a very intensive game with the use of pixel shaders to match. The scores that we are reporting, however they may raise a brow, are reflective of game play. The version of Half-Life 2 we used was source v0.4.

Please take note of the article link and the explanation of DX 9 and performance characteristics they offer. I don't think they failed in communicating exactly what the benchmarks represent, and while I'll condemn them for their stance on synthetics, I can't see how it is valid to condemn them on their use of HL2 to achieve what they stated here they were trying to use it for.

AFAICS, your "benchmarking as a game" commentary seems to propose that a game can't be used to validly represent DX 9 performance for gaming, and that usage "as a synthetic benchmark" is the reason. If there is something I misunderstood, or some failing in the information they provided here that I missed, please point it out, but as it stands your last comment seems to be flawed as I've outlined AFAICS.

 

[Ostsol]

Please take note of the article link and the explanation of DX 9 and performance characteristics they offer. I don't think they failed in communicating exactly what the benchmarks represent, and while I'll condemn them for their stance on synthetics, I can't see how it is valid to condemn them on their use of HL2 to achieve what they stated here they were trying to use it for.

Regardless, it is still synthetic and is not representative of the way the game will be played on that video card.

AFAICS, your "benchmarking as a game" commentary seems to propose that a game can't be used to validly represent DX 9 performance for gaming, and that usage "as a synthetic benchmark" is the reason. If there is something I misunderstood, or some failing in the information they provided here that I missed, please point it out, but as it stands your last comment seems to be flawed as I've outlined AFAICS.

A game can be used to validly represent DX9 performance, but if there is another, higher performing rendering path for which the video card defaults to, that other rendering path is far more useful to the user as an indication of performance in that game.

Whatever, though. I did not make my original post to try and convince anyone. I merely stated my opinion and continued posting to defend it.

 

[demalion]

Please take note of the article link and the explanation of DX 9 and performance characteristics they offer. I don't think they failed in communicating exactly what the benchmarks represent, and while I'll condemn them for their stance on synthetics, I can't see how it is valid to condemn them on their use of HL2 to achieve what they stated here they were trying to use it for.

Regardless, it is still synthetic and is not representative of the way the game will be played on that video card.

It's not representative of the way the game will be played on that video card, because of the performance the card exhibits when delivering such DX 9 effects. That, quite "simply", is what is being shown!

Why is this use of HL 2 "wrong", especially with an accurate explanation of this, and a directly linked exploration of other characteristics of HL 2 in the article? The purpose of the article was hardware information, was it not?

You just demanded that it is wrong to show this information because no one would actually play with it! I'm left a bit speechless here...aren't hardware reviews and comparisons supposed to show consumers things like this, maybe even before they find out for themselves what they won't be playing with the hardware?!

The information is accurate, pertinent, not misrepresented, and offered with what seems to be a fairly selected set of other information. You do not take issue with any of this, merely with the information being present at all. Prior commentary covers my response, now that it seems clear there is no misunderstanding.

"Regardless" of the actual article explanation of what the performance figures mean to go along with the performance figures, what they did was wrong? Ack!

AFAICS, your "benchmarking as a game" commentary seems to propose that a game can't be used to validly represent DX 9 performance for gaming, and that usage "as a synthetic benchmark" is the reason. If there is something I misunderstood, or some failing in the information they provided here that I missed, please point it out, but as it stands your last comment seems to be flawed as I've outlined AFAICS.

A game can be used to validly represent DX9 performance, but if there is another, higher performing rendering path for which the video card defaults to, that other rendering path is far more useful to the user as an indication of performance in that game.

Yes, which is why it is a good thing they explicitly said they were using the game to compare DX 9 performance. But, by all means, let us discuss things "regardless" of that, the role HL 2 is specifically being called upon to play and why, and that this was a hardware comparison article.

Whatever, though. I did not make my original post to try and convince anyone. I merely stated my opinion and continued posting to defend it.

Sure, you stated it, and I'm telling you what I'm thinking about your statement and why. Why are we covering such redundant statements as if they give us license or excuse to say something independent of the merit of what was stated?

 

[Pete]

Regardless, it is still synthetic and is not representative of the way the game will be played on that video card.

People upgraded their GF3's and GF4's to 5200's and 5600's precisely because they were advertised as DX9. Cine FX, no more, no less. You're putting the cart before the horse in anticipating the FX's bad performance and trying to compare it using a lower-quality path.

Anandtech (and everyone else) also really couldn't have done so properly as they apparently weren't able or allowed to publish screenshots of the visual difference. So they're left with benchmarking the game with equivalent IQ, rather than equivalent framerate.

 

[Ostsol]

Regardless, it is still synthetic and is not representative of the way the game will be played on that video card.

People upgraded their GF3's and GF4's to 5200's and 5600's precisely because they were advertised as DX9. Cine FX, no more, no less. You're putting the cart before the horse in anticipating the FX's bad performance and trying to compare it using a lower-quality path.

I agree and that is why I have stated the following somewhere above:

1) I would never recommend the GeforceFX to anyone (well, perhaps the FX5200 non-Ultra for programmers wanting a platform for compatability testing -- though even then I'm not so sure).
2) I never said that the DX9 benches shouldn't be there. I merely said that there should also be benches for render paths useful to the video card.

 

[demalion]

Regardless, it is still synthetic and is not representative of the way the game will be played on that video card.

People upgraded their GF3's and GF4's to 5200's and 5600's precisely because they were advertised as DX9. Cine FX, no more, no less. You're putting the cart before the horse in anticipating the FX's bad performance and trying to compare it using a lower-quality path.

I agree and that is why I have stated the following somewhere above:

1) I would never recommend the GeforceFX to anyone (well, perhaps the FX5200 non-Ultra for programmers wanting a platform for compatability testing -- though even then I'm not so sure).

Yes, and HL 2 was used to show people who don't know as much as we do on the subject some specific information, in addition to the other information in the comparison. You have knowledge about how PS 2.0 shading compares. I have knowledge about how PS 2.0 shading compares. Many people do not, and this was the role HL 2 served, and served validly. Your continuing to state that usage is "wrong" is the issue with your statements. Your commentary on synthetic benchmarks further target the practice of presenting clear and specific performance information in general.

2) I never said that the DX9 benches shouldn't be there.

You said the benchmarks are "wrong" to be there as they were for HL 2, and that the DX 8 path for the FX chip should have been compared. But as they were, the benchmarks were specifically explained to be for DX 9 comparison, and presented along with information on other performance characteristics of the chip, along with, as far as HL 2, including a link to information on the other paths the FX chips would be required to run, their relative performance, and at least an attempt at a thorough explanation of HL 2's rendering paths and their rendering qualities at length, as is required without the option of screenshots.

You can't just ignore these specifics of the information that was presented, say the presentation was "wrong", and then say you weren't saying the specifics of the way they were presented were wrong as you continue to to condemn them when the specifics are brought up! Not and expect people to ignore the specifics just because you do.

If the performance results were not specifically explained to be used for DX 9, you might be saying something besides that "DX 9 benches shouldn't be there" by condemning them as you have. But they were.

If they were not presented with information on other performance aspects of the chips, or were presented with statements that mislead about the presence of other paths and options in HL 2, you might be saying something besides that "DX 9 benches shouldn't be there" by condemning them as you have. But they were.

I merely said that there should also be benches for render paths useful to the video card.

No, you didn't "merely" state a preference for extra information.
What you said was that the clearly explained as DX 9 benches they did do were "wrong", including commentary on the unsuitability of "synthetic benchmarks" as being applicable to evaluating hardware gaming performance in a hardware comparison article.