Dave Baumann said:
Although, in general terms, I find the write up of the game benchmarks the most tedious of elements within the reviews (less so in an architectural review, more so in a product review) game benchmarks have to be used because they do serve a distinct purpose.
That purpose (as you explained, which I didn't feel the need to quote) generally do not, IMO again since this is how I perceive your interest, matter much. You talk about various elements about a piece of (new) 3D hardware being used simultaneously in games. I'm sorry but with the never ending chicken-and-egg situation (HW, SW) these elements generally come down to filtering and AA performance at various settings that are no more than fill-rate-sapping higher resolutions. That's what every single games benchmarks offered in your (B3D) reviews come down to, AA and filtering performance with more than a touch of bandwidth considerations understandably coming into the picture. There are no explanations offered about the kind(s) of shaders used in a game (and in particular, in the demo used for a game benchmark) -- we all know increased filtering and AA levels WILL result in lower benchmarks... why repeatedly give us such benchmarks (in every single review)? It wastes your time, it wastes my time. If a game suffers little-to-no performance dips in performance due to increasingly higher filtering and AA levels, is it worth the time doing the benchmarks only to surmise what most really ought to know, that those games are bound by the performance of the host CPU?
How much do you know of demos used in benchmarks in your review? You attempt to lay it out in your tech sections in a review when it comes to explaining shader work. There is a big emphasis -- by you -- on the ROPs and intricacies of a G/VPU when it comes to shader work. Yet all we see in games benchmarks are obvious things -- fillrate expensive stuff like AA and filtering with nothing offerred by way of explanation regarding all those IHV-
and-media-outlet-touted shader work put into hardware by the IHVs. Why? Coz you don't know the games (demos used), coz of the chicken-and-egg situation (how many SM3 games now, after the first SM3 hw came out?), coz you want to just churn out the AA-and-filtering games benchmarks coz those are what "regular" foilks can get OOTB, coz devs won't respond to you when you ask them about a demo you recorded, coz... coz...
I'm not saying AA and filtering performance aren't important. I'm only saying these are "Duh!" stuff -- the higher these two are, the more expensive they will be, coupled with higher rezs. We're talking about fillrate here, surely and most definitely something that Kristof was thinking about when he coined "Nothing in 3D is free". There CAN be a big reason of course -- look at the "free" 2xAA of the Xenos. That's worth studying (tho we all probably won't be able to benchmark the XB360!).
I've seen you (Dave) basically "going to the metal" in increasing frequency the past 3 or 4 months, either in your official B3D articles or in the forums. Innovative new memory bandwidth structures or new (which hasn't been the case) AAs may warrant many games benchmarks in your articles but other than these two I'm not sure if you realize what you actually posted above.
I see you analyzing hardware more than you do the software that you use to analyze the hardware. In one single scene in one of the game demos you use in your reviews, what do you really know about that scene Dave? Not to mention the other 1000+ frames in that demo!
Games are useful for you Dave because it serves to lend relevance to the reason
the majority would read your video card reviews.
10 pages of your video card reviews consist of stuff that has little to do with what programmers seek in PC3D that are beyond their control. What do they seek? What percentage of silicon and resources are spent by an IHV on a new 3D chip? Why do new iterations of DX ignore filtering and AA that are so prominent in reviews?
Why waste time on something you aren't that interested in and/but concentrate on other things that I know you'll do well in?
Oh yes, it is that much easier for me to comment on something I'm not a part of, especially when it comes to something that involves costs and time. But there you go; this is B3D, this is Dave Baumann, this is where I get 3D hardware info I almost never get elsewhere. Games benchmark? There's Kyle, there's Anand, there's a bunch of others...
