Ratchet said:
How is this the greater good though? I can review any game I want now (not like there are tons of benchmarkable games to choose from, maybe a dozen if you look hard) and I don't have any problems getting other sites to link to my reviews. Why would I want to restrict myself to only using games that are approved by this group. And who decides who's allowed to be in this group and who isn't?
I don't get it.
No, the Consortium never limits what games/benchmarks you put in your review. Put anything you like in there, as many as you want. All you are guaranteeing is that one specific title of your choice, picked by you pursuant to the consortium rules, will be included in your graphics reviews and from that section/page of your review that includes the title you chose that you will link to the Consortiums webpage.
What this does for the community is give Joe Surfer a navigator to get around to reviews that include lesser-known titles. This will be taken advantage of by two groups of people --those looking for a benchmark of a specific lesser-known game that interests them, and those (a pretty nice chunk of the community, I believe) who are interested in checking out a broad range of results from outside the group of "usual suspect" games/benchmarks that get benchmarked to extremis.
This, btw, is not an "anti-usual suspects" proposition. They are still going to get their due (and more). It just ensures that a broader range of games/benchmarks get benchmarked on new cards, and possibly more importantly makes it easier for Joe Surfer to find those benchmark results on new cards for lesser-known titles.
If (big if) this catches on, it even has the potential for impacting the behavior of the IHVs in broadening their concerns for making sure a wide range of games/benchmarks work properly and at a high performance level on their hardware.
For as long as I have followed graphics reviews and browsed the community's reaction to them (uhh. . .8 years or so?) there has been a consistent (and justifiable) complaint that the "usual suspects" of the day get too much benchmarking attention and the lesser-known titles are largely ignored. This is a disservice to the greater community that plays more than just "the ususal suspects" on their cards and all too often receive a nasty surprise after reading the reviews, buying a new card, and then trying it out on a game outside "the usual suspects".
This proposal addresses that problem in a practical, implementable way to at least a degree. Again, "the usual suspects" will still get theirs --but anyone interested in more will be able to find it.