I agree with there being need for tweaking that increases a card's performance since every card needs specific code. There's nothing wrong with that at all. You have to draw the line where standards (I don't even know if you can call them that) are breached.The Baron said:This is where I'm on the fence.You need a set, but you can't get accurate results about what the entire card's deal is without results that aren't tampered with.
What IS an accurate result? If Card X benches 25% faster than Card Y in unoptimized benches, but Card Y's driver team creates drivers that allow Card Y to run 50% faster than Card X in 95% of situations, which would you buy? In a clean benchmark, Card X would demolish it. But, Card Y would likely be the "better" package.
Note, I did not say optimizations that cause IQ loss or optimizations that only affect benchmarks. I just mean optimizations in general.
I think we're reaching a point where you have to differentiate between the speed of the card and whatever the benchmark results tell you. How will you do this? Hell if I know. But, I have to think that application-specific optimizations are here to stay, and no one will ever remove them. They're just too beneficial for the majority of people for anyone to take them out. What we need is a way to toggle specific ones in order to judge the speed of the card and then a way to judge the effectiveness of that optimization.
This way, you can judge the speed of the card without any optimizations (remembering that only the more popular pieces of software will receive the time it takes to really increase its performance) but also see the potential of a card when code is designed specifically for it.
So, basically, having multiple codepaths for a card isn't a bad idea. It's just not the only thing we should do yet.
1: The user has AF enabled.
2: The user wishes the receive the results of this setting.
3: The user doesn't get it because the chip maker who makes the drivers wishes to make them think the card is capable of more than it would be if they didn't force a type or level of texture filtering.
How to solve the problem of benchmark standards? The answer is simple and we have seen it already. You just make sure you read good reviews. I've said it before and I still think it's a good idea... we need a group of people in the media and industry that can decide what is best for the readers and buyers to see. This would uproot cheating and keep bad reviews from happening or being taken serious... because they lack "the certified seal" or something.
I guess looking at it all, there is no such thing as a true gaming benchmark right now. Not one that we can trust without looking at the past freaking history of the site, author, driver team, chip maker and alignment of the planets.
Getting to the meat of the discussion, I don't think ATI needs "special tweaking" right now. They have enough power over nVidia to leverage the fact that they don't need those tweaks. Yes, they might have application specific tweaks that use detection methods, but those are for legit reasons.
Just because ATI isn't tweaking in an unjust way to the user (or we don't know about it) doesn't mean they aren't capable. If ATI had a card that performed sadly, I wouldn't discount on them cheating. However, I believe ATI has been and will run their business more respectively than nVidia has been for the last bit.