Quote from Brents missive...
" Lastly, the timedemo stuff. Yeah, it’s not perfect. The evidence points to some foul play in timedemos, can’t deny that. And we know it’s not perfect either. In fact we have already been discussing the fact of using private timedemo’s made by us and only used by us. We have some thought going that we will probably make 2 timedemo’s per game. One that is private, and one that is public, therefore the private one acts as our control. We will be able to see if any foul play happens. There are many things we are considering, and believe me we are working to get better honest and fair benchmarking. "
Found your post as a whole enlightening and interesting. I do have a couple of observations about the above.
On the face of this, It sounds like a good idea, and that it might be made to work with one caveat. I would have to be able to trust the Website and it's editors to not leak the Private Demo to any IHV's and my "A" list of those trusted Web sites is quite short at this time. I am not particularly trusting when it comes to People who may have a vested interest in how things transpire. This is not directed at you personally. What may happen without your knowledge is something else though, that you have no control over.
Trust must be earned, and once having been earned , if broken, becomes even harder to garner. I am quite saddened by this whole affair as the trust I had for a certain Hardhitting, raw, (and believed to hold unbiased content) Web site has been broken. There have been some very good questions asked here by the serious posters as to the motives of a certain web editor, and an explanation and some substantiation for his stance in a reasoned manner has not been forthcoming. Instead he chooses to single out the one liners that suit his purpose as he slams and catergorizes this web site based on those few chosen remarks that are ignored here by the serious minded just as he supposedly does. He still avoids the valid issues raised here by a few, with rebuttals like that instead.
Now as I read it, the [H] stance is to change the way benchmarking will be done, which on the surface seems good, but I have to wonder who is directing this endeavor. It sounds like Nvidia PR speak almost from word one. [K] prognosticates about the future of 3D programming, (Divergent Code Paths and all game developers aligning themselves with a specific IHV) as if it is a given, but this sounds exactly like Nvidia speaking. This is how Nvidia would like to see things go. It just seems like more of the Nv party line.
Some editorial excerpts:
" Quite simply, NVIDIA has changed the rules of synthetic benchmarking. Synthetics were understood to be free standing utilities that were not optimized for. We think that is how the hardware community saw these tools. But let’s look at the facts and legalities. As in the case of 3DMark03, we don't think there was anywhere in their EULA that specified you could not “optimize†for the benchmark. We don’t think “cheating†was defined. Like many other synthetic benchmarks, everyone understood the rules of the game, but no one defined them, as they possibly should have been."
This sounds like pure justification for Nvidia cheating... And now he's a Legal Expert too! but He's right! "everyone understood the rules of the game" It's a matter of having the integrity to play by those rules Fairly that have come into question and the integrity of Web sites who ignore the real issues.
"NVIDIA is optimizing for benchmarks quite clearly and that is something we will have to deal with"
Why do we HAVE to deal with this? Why do we HAVE TO put up with cheating? Why are they above the law. Has Nvidia ever heard of the word Ethics? How about [K]?
" Apples to apples benchmarking is becoming more difficult. "
Only more difficult because Nvidia is making it so.
"The fact of the matter is that benchmarking the same code path on competitive video cards will not give us an accurate representation of gameplay on one of the two major cards out there.""
Because Nvidia chose not to follow DX9 standards? So where Nvidia leads the sheep should follow? Sorry, but they screwed themselves by believing that they held more power in the industry than they thought. To hear them talk, 9 out of 10 Games are being developed on Nvidia platforms with it soon to be 10 out of 10, implying that the Developers are Only using Nvidia Hardware. and if you don't own an Nvidia card, no future games will run for you?
" What we mention above with DOOM3 is only going to become more widespread and not necessarily in such a cut and dried fashion. We are going to see D3D games that behave differently within the same API."
Silly me, I thought the Game/API told the hardware what to do, not the other way around? I guess not in Nvidia's world.
"As the video card companies move forward with developing tighter bonds with game developers and publishing houses, there is no doubt we are going to see more hardware specific effects in games. Depending on your hardware, your game experience may differ. To what extent is yet to be seen, but we think that this alone will show that the days of apples to apples is coming to a close as the proprietary hardware technologies diverge."
The world according to Nvidia... But I thought that DX9 was the standard, Not Nvidias interpretation of it via non-standard code paths using sub-standard precision?
"While this is just speculation, we think that NVIDIA moving some of their high-end business to IBM will facilitate this happening more rapidly as each faction of the video wars becomes a bit more shielded from each other’s technology.
"
Most definately speculation, and why would hiring IBM to Fab promote divergence of Video technology? More thinly veiled Nv PR to justify divergence?
Kyle continues to aide and abet Nvidia in it's long term goal to dominate the Video card market through Proprietary means. His stance that divergent engines is a given, and that that is Ok is scary. Splitting the Game developers into ATI or Nvidia camps hurts the industry and especially the Gamer. I want the Game developers to code to a standard that will run well with good IQ on any capable piece of harware I see fit to use.
Proprietary API's= bad
Standard API's =Good!
All I have to to do to remember this is recall getting a Diamond Stealth S220 with the Rendition 2100 chip some years back. Worked pretty good, But it didn't look very good on any number of games because They either ran in software mode, or used Glide. The one I remember best was Janes Longbow Gold. Glide only or 640X480 software mode. yuck! Luckily I had a few Rendition "Optimized" games too, like Nascar Racing. And when coded for, The Rendition card was no slouch, and looked pretty good too.
anyway... The point being that I didn't like seeing cool games out there that didn't take advantage of my Hardware. Playing a game at reduced settings and or with poorer IQ because My hardware wasn't supported generally made me pissed. Why not code for both? Ding! the light goes on! duh! because it is more work!
I believe in the K.I.S.S. principle. Keep it Simple Stupid! Lets keep it simple and straightforward with one or 2 Standards, say DX9/OGL, and build our Hardware to run those fast without resorting to special Codepaths etc. Seems simple enough. That way all the games run on all the Hardware. Why regress back to Glide days... wait, didn't the 3Dfx crowd migrate to Nvidia. Has 3Dfx conquered from within and are now running Nvidia "The Way it's meant to be Run"?
These are my opinions and mine alone. They do not represent in any way the Opinion of any B3D staff.
, but, believe it or not, I covered that in the missing thread as well....basically, I'm hopeful it will be food for thought for some, and help in further support achieving what (I think) Wavey is asking for.
