Competetive Evaluation in Reference / First Look (P)reviews

[Dave Baumann]

OK, the ever present question - se above and vote.

Note: The point about not using it is to look at the product at hand and review it and its merits. For vendor product reviews (where we have alredy seen a reference sample) this would probably stay the same.

 

[jvd]

Dave I would say you should still ahve your first looks where you go in depth on only the new gpu . Then after that you can compare and contrast the card in a second review that just dives right into bencmarks

 

[ninelven]

erased

 

[Geo]

Dave I would say you should still ahve your first looks where you go in depth on only the new gpu . Then after that you can compare and contrast the card in a second review that just dives right into bencmarks

Quoted for truth.

I came in here very ambivalent, and maybe was going to dither in a note and not actually vote one way or the other.

But then I read the above and realized an important truth --there is only so much "Wavey Time" to go around, and inevitably bringing the competitor's products in too will have to bite into either your sleep or the depth you handle the card you're really reviewing. And B3D is all about the depth.

The horserace angle, for those who are interested (and that includes me), can be derived by either/and looking at the B3D review for whatever card you want to horserace against or other sites' reviews of same. Not a perfect solution, perhaps, for race enthusiasts, but that is definitely a secondary concern compared to even a hint of threat to the thoroughness of the typical B3D review.

So if you feel like doing horseraces, and have the time to devote to it after the primary mission, let it be in its own article (with links back to the indepth originals), not an initial on a new release.

 

[Reverend]

I'd say definately. However, it seems like it would be rather difficult to do as cards rarely get launched at exactly the same time, so something like jvd suggests looks good to me (though I think a small feature review/comparison/analysis would be nice as well).

To make things easier to understand, let's assume that ATI released their R520 now and NVIDIA's latest is still the currently-available NV40.

What I think Dave has in mind is to include NV40 performance head-to-head (theoreticals, synthetic and games) against the R520 in his R520 reference writeup. Is that right Dave?

That's the only scenario I see that is possible. Additionally, it would be good to include some comments from games developers about the new product in the reference preview. Non-marketing comments of course (i.e. don't ask Tim Sweeney ask, say, Deano ).

Oh, and that means goodbye to sex during the course of writing the reference preview

 

[CMAN]

Dave I would say you should still ahve your first looks where you go in depth on only the new gpu . Then after that you can compare and contrast the card in a second review that just dives right into bencmarks

I second that view. If you do head to head against different vendors, then create a second set of articles. In other words, only if they are in addition to the format currently set up.

 

[RussSchultz]

Make sure you take one apart, de-cap the chip, and get pictures of the die when you do your competitive analysis.

Oh wait, different competitive analysis...

 

[Hanners]

In the past I would probably have voted for starting to use head-to-head comparisons, but these days I'm far more comfortable with the way Beyond3D does things and think it offers a more valuable evaluation of a product at the end of the day considering the sites outlook and values.

So, in short, I vote to keep things as they are.

 

[Dave Baumann]

In terms of what it would ential would merely be the performance figures from competing hardware (with some analysis alongside), which will probably come at the expensive of one of the other boards of the previous generation from the same manufacturer.

 

[Druga Runda]

Dave I would say you should still ahve your first looks where you go in depth on only the new gpu . Then after that you can compare and contrast the card in a second review that just dives right into bencmarks

I second that opinion, as the "first look" is the best as it is, it would only be interesting to add some variety after in some kind of separate "comparison" review where you pit the competing products against each other as most other websites, givent that you have enough time.

For the "unique" value, please keep the current format first and foremost, but it would be good if you could add competitive reviews to the mix if you have time/resources to do so.

 

[Druga Runda]

In terms of what it would ential would merely be the performance figures from competing hardware (with some analysis alongside), which will probably come at the expensive of one of the other boards of the previous generation from the same manufacturer.

in that case - no

keep it as it is, we will see this elsewhere.

 

[Sonic]

If it could be done in a way as to not impact the review of the main part that is being written about itself then I guess it would be ok. The reviews here go very in depth and the only interest to me of having say an R520 review with results of a previous generation NV40 would be to see how much performance the new generation of ATI cards allows. This should also have the previous generation ATI card also, X800 etc., for reference sake.

One thing I'm slightly concerned about is the features of said card. Having the competitors' cards in the review may detract from this. Ity probably wouldn't though.

I am fine wiht the way thigns are run now, maybe as a subsection in a review it would be ok to see the performance from competitor parts.

 

[Druga Runda]

OK that is an idea, in the same way you have one page specifically for AA/AF you could have one extra page for comparison with competitors.

 

[Geo]

OK that is an idea, in the same way you have one page specifically for AA/AF you could have one extra page for comparison with competitors.

One page doesn't cut it tho, as by that requirement it will have to be fairly selective, and then you open the "bias" can of worms on what you picked and didn't pick to show on that one page. No, if you're going to do it at all and don't have more time to give, then you have to do what Dave said and take out one of the prev gen cards from the same manufacturer.

For instance, here: http://www.beyond3d.com/reviews/ati/r420_x800/index.php?p=14

There are five cards, two be reviewed (X800s) and three being compared against. One can probably assume it will be common on new arch to have two cards to review (maybe even three, maybe only one). On this review how many cards would you drop? Just the 9700pro? Or would you bring in *two* cards from the other guys lineup to match with (which seems reasonable) and now you've dropped two cards 9700pro and 9800 pro from this one?

Plus these things tend to be more time-sensitive than most in getting them up in a timely way. Certainly I fidget like hell waiting for them.

So I still like do the usual for new release, follow in a few days, or a week, or whatever is comfortable, if you like, with your horserace piece. I understand horserace will bring more folks in. And if you link the horserace pieces prominently to your usual thorough reviews, maybe you can "grow" your core audience of folks who appreciate the primary mission of this place.

 

[digitalwanderer]

I'd like to see seperate competitive reviews here.

Keep doing the reviews as you usually do, but also add a comparison between some cards too occasionally that you've previously reviewed. That way you can keep doing what you're doing and when two competitive cards are both out you can do a comparison.

The best of both worlds. 8)

 

[Himself]

I think there is a way to do comparisons without it having be a boring shootout of gaming benchmarks. So long as it is analytical with some discussion of why there is a difference between products, rather than hystrionics about crowning new speed kings, it would be cool.

 

[Geo]

I think there is a way to do comparisons without it having be a boring shootout of gaming benchmarks. So long as it is analytical with some discussion of why there is a difference between products, rather than hystrionics about crowning new speed kings, it would be cool.

Yeah, but at least when you short-shrift the prev gen of the same manufacturer in a review of the new gen you don't get people calling you an X800 f*nboi with an irrational bias against the R300 that served the entire industry so well in the past, yada yada. Inevitably bringing in the other guy will force you to use more words, and more carefully, about their product, and just as inevitably shift some of the effort and focus away from the card really being reviewed.

 

[MY80S]

I think it would be better if the reviews stay as they are (no comparison with competitors' products). I would however enjoy performance roundups (seperate from the reviews) including hardware from many IHVs for a given (or all) market segment from time to time.

That way, you'd have both reviews focusing on the reviewed product itself and comparable performance numbers (which isn't exactly the case right now as the platform/drivers sometimes differ between reviews) / analysis on them.

 

[ellingsen1]

I would like it if you included the competitors top card regardless of the generation. Some people may not consider it fair but over time the IHV's would see-saw back and forth as cards are released and would end up even in the long run. As far as I am concerned, you could skip the head to head game benchmarks. They are available anywhere. I would really just like to see a head to head analysis of the synthetic benchmarks like vertex shaders, pixel shaders, tex-bench, etc. It would just save time flipping back and fourth between previews/reviews.

 

[ANova]

In terms of what it would ential would merely be the performance figures from competing hardware (with some analysis alongside), which will probably come at the expensive of one of the other boards of the previous generation from the same manufacturer.

in that case - no

keep it as it is, we will see this elsewhere.

I'm inclined to believe results from B3D over any other site. I don't think including some simple numbers from a competitor's platform in addition to the current in depth analysis would hurt.