Brent's latest review........ the way it should be done?

[martrox]

http://www.hardocp.com/article.html?art=NTk5

I don't think anyone can say this isn't the way videocard comparisons SHOULD be done....... An absolutely fantastic review Brent. At this point, I can understand that there's minimal reasons for using synthetic's, as there is enough DX9 demos/games in this review to give us a view of the performance issues....

Brent is the only reviewer on the net that seems to be truly interested in the gaming experience .... rather than just posting numbers......

 

[Bouncing Zabaglione Bros.]

Why is the 5900U being reviewed against the 9600XT? Isn't this a high end card being compared to a much cheaper mid-range card? The conclusion talks about how the 9600 is better in a lot of things, and the 5900U better in a few, but surely that's misleading because compared against the high end ATi card that the 5900U is actually being pitched at, wouldn't the 5900U be slaughtered?

It's kind of like comparing a rubbish sports car against a family saloon to make a poor sports car look better than it is.

 

[Hanners]

Why is the 5900U being reviewed against the 9600XT?

It's in there to compare against the 5900XT. Seems fair enough to me.

 

[madshi]

Why is the 5900U being reviewed against the 9600XT?

It's in there to compare against the 5900XT. Seems fair enough to me.

Guys, guys, both of you should check the review again. There's no 5900U there.

 

[Blastman]

Good review with interesting analysis. But while I found the first review (last month) running different resolutions to achieve comparable playability a novelty, after reading this review I must say I much prefer running the cards at the same resolution. Running at the same resolution gives a more direct comparison of the cards speed differences. It will be obvious to the user that the faster card can run with more eye candy.

Also, some readers who just skim the article and look at the numbers may miss that some of the benches were run at different settings. This could lead to erroneous conclusions like -- the 5700U was faster than the 9600XT in many of the games when this was not really the case.

 

[mczak]

Brent is the only reviewer on the net that seems to be truly interested in the gaming experience .... rather than just posting numbers......

I also like it. That said, I think there are some dangers using this style of comparison: it is very easy to manipulate the data to favor one or another card/manufacturer. For instance, in a particular game, enabling 2xAA might not have a significant performance hit on some card. But, a reviewer could just say it was too slow, disable it and leave it enabled on another card - so the absolute performance number might be a bit higher on the card without AA, but a reader would think surely the card with AA enabled is faster/better if it achieves almost the same speed but with AA enabled, even though it might actually not really be faster if the other card would also use AA (since the comparative numbers with the same settings are not there it's impossible to figure out).
So I don't think I want to see every monkey without a clue but a lot of bias write reviews that way. Brent doing it is fine but for instance if Kyle would do it - no comment.

mczak

 

[Tim Murray]

When you're reviewing a card without any reference point for comparison or you're reviewing two incomparable cards (like when Brent did the Sapphire 9800XT and 9600XT), finding maximum playable IQ settings is definitely the best way to review. However, when you're reviewing cards from different IHVs, I think that you need to use the same settings because of bottlenecks and such.

 

[martrox]

When you're reviewing a card without any reference point for comparison or you're reviewing two incomparable cards (like when Brent did the Sapphire 9800XT and 9600XT), finding maximum playable IQ settings is definitely the best way to review. However, when you're reviewing cards from different IHVs, I think that you need to use the same settings because of bottlenecks and such.

Sorry, Baron, but you can find that at any 2 bit review site. And you end up with numbers, that's all. Nowhere else do you get an idea of exactly how these cards compete in the real world at the "best" and "usuable" IQ settings. The idea that comparing differant IHV's this way is wrong due to" bottlenecks and such" is deplorable.......

 

[Tim Murray]

Uhm, provide IQ comparisons, of course--IQ is fundamentally subjective. You can skew the bejeezus out of any benchmark based on the settings you used, so IQ comparisons are a must. But even if you're going to do what Brent does, I think you need to have a very specific and public set of criteria to determine "playable." Min 30 FPS or above? Brent seems to do that in some situations but not in others. If you want to be declared free from bias, you must have actual criteria instead of "it feels playable." For example, look at Call of Duty in Brent's review. Seems a bit odd that he doesn't provide 2x/8x numbers for the 9600XT or 4x/8x numbers for the NV cards (although I suspect that it's because NVIDIA cards hate 4x antialiasing with Call of Duty for some reason--they basically crap out).

This isn't really directed at you, Brent, although IQ comparisons between the different settings from each card would be nice (then again, it'd be what, 30 megs or so worth of PNGs for that single review?).

 

[Bouncing Zabaglione Bros.]

Guys, guys, both of you should check the review again. There's no 5900U there.

Aha! I have been caught out by the misleading model numbers and opposite meanings of the same suffixes! Makes much more sense now in a "deliberately confusing corporate policy" kind of way.

 

[jb]

At this point, I can understand that there's minimal reasons for using synthetic's, as there is enough DX9 demos/games in this review to give us a view of the performance issues....

Granted while we are in a better position now, we were not that postion 4/5 months ago so that data would have been usefull then....

 

[martrox]

At this point, I can understand that there's minimal reasons for using synthetic's, as there is enough DX9 demos/games in this review to give us a view of the performance issues....

Granted while we are in a better position now, we were not that postion 4/5 months ago so that data would have been usefull then....

I totally agree with you........

 

[Hanners]

Guys, guys, both of you should check the review again. There's no 5900U there.

It was mentioned in the opening comparison of card specs. You're right though, it's a 5700 Ultra in the rest of the review.

 

[Hanners]

When you're reviewing a card without any reference point for comparison or you're reviewing two incomparable cards (like when Brent did the Sapphire 9800XT and 9600XT), finding maximum playable IQ settings is definitely the best way to review. However, when you're reviewing cards from different IHVs, I think that you need to use the same settings because of bottlenecks and such.

Personally (Although I know it's a lot of extra work) I'd do both - Some tests at identical settings to set the baseline for performance, then move on to pushing both cards to their maximum playable IQ and seeing how they fare.

 

[volt]

I still don't get it why in the blue moon does Brent insits Prince of Persia is a DX9 title. Again, AFAIK it's using DX8.1 vertex shader effects, no more no less.

 

[John Reynolds]

You also have to be careful with your settings choices. For one of the tests Brent declared the 5900XT the clear winner because it ran the game at 4x AA compared to the 9600's 2x AA.

 

[nelg]

When you're reviewing a card without any reference point for comparison or you're reviewing two incomparable cards (like when Brent did the Sapphire 9800XT and 9600XT), finding maximum playable IQ settings is definitely the best way to review. However, when you're reviewing cards from different IHVs, I think that you need to use the same settings because of bottlenecks and such.

Personally (Although I know it's a lot of extra work) I'd do both - Some tests at identical settings to set the baseline for performance, then move on to pushing both cards to their maximum playable IQ and seeing how they fare.

What I would like to see is that once the optimum settings for a card/game have been established use those same settings on the other cards. Of course with the addition of comments WRT any differences in IQ.

 

[Joe DeFuria]

I like the concept of running at "playable settings", but it takes a bit more diligence.

What I would do:

1) Find the "playable sweet spot" for each card. (Highest image quality where you deem it "playable.")
2) Post benchmarks for BOTH cards at the settings for its own sweet spot AND the other cards sweet spot. Do not mix benchmark results from two different settings in one graph. Too confusing.
3) Post image quality comparisons / shots for each card at its sweet spot.

Now, the number of tests you need to can rise exponentially with every additional card you test, if each card has a different sweet spot...so things can get unruly fast. Should work reasonably well for a 2-3 card shoot-out though.

 

[Joe DeFuria]

What I would like to see is that once the optimum settings for a card/game have been established use those same settings on the other cards. Of course with the addition of comments WRT any differences in IQ.

Damn...beat me to it.

 

[nelg]

Hey Joe, that's plagiarism
I will grant you this though, you said it better.