Joe DeFuria said:
Chris123234 said:
A game is a game whether anybody plays it or not. Do games that nobody play anymore become synthetic since they are no longer played?
No, they do not become synthetic. However, they become just as
relevant as synthetic benchmarks.
Why is something being synthetic related to the relevance of a benchmark? If a dedicated benchmark application used the Quake III engine or the Unreal engline, it wouldn't be lent automatic relevance. Your discussion seems to be continuing the Aquanox example in the context of the upcoming Aquamark, and that is the detail I'm going to reply to...
The general relevance of a benchmark is completely due to the implementation decisions and stresses of the benchmark...nothing to do with popularity. Whether it is a game, a popular game, a good game, or whatever, are unrelated aside from how these factors might or might not cause other games to make the same implementation decisions and stress the same factors.
Of course, a game engine might be more popular than another, so the performance of a more popular game (the game itself that is using the popular engine) might be more relevant to gamers, but what seems to be ignored here is that that relevance of a benchmark to gamers depends on more than
just using the same game engine.
Note that the example of the Quake III engine, which, due to its popularity and frequent usage as a game engine, would seem to encourage the idea that popularity might be related to a benchmark's relevance. It instead seems, AFAICS, to show that not even with the same engine being used due to popularity, does the significance of specific implementation decisions matter any less...note the varying results and characteristics exhibited by the varying implementations of the engine. Splinter Cell seems to indicate a similar tendency with the Unreal engine. ("Warfare engine"? Not sure what id calls theirs)..
What matters for "Aquamark" as a general benchmark is not whether "Aquanox" or whatever is popular, it is the information we can gain from the Aquamark benchmark, and its accuracy and clarity. Where popularity comes in is when deciding which games a reviewer might use to serve their readership, and that's only as far as the direct measurement of in game performance for the specific game is presented for user info.
As an aside, I think this idea is what nVidia finds inconvenient. If, instead, only popularity is a factor, that fits well with the goal of targetting specific timedemos and games while hiding actual hardware characteristics, atleast when image quality issues can be hidden and when the timedemos can be properly targetted (assuming the clipping plane methodology or similar is used). And, of course, when these targetting methods acceptably alter representation in benchmarks to nVidia's liking.
That is, they do serve a particular purpose, just not of the "this tells me how cards x-y and z perform on the popular games a-b-c at this moment in time."
When it comes to evaluating the results as something general about the video card, and not specific to the application used, popularity is a red herring, and the same with whether a benchmark engine is even used in a game or not. What I'm having a problem with is seeming to use the idea of benchmark relevance for evaluating video cards as dependent on the popularity of the associated game, when we're talking about a benchmark that will only be indicative of that
game's performance
if it uses the same scenes in the same way, as well as the same engine.
I.e., the game's popularity is by nature irrelevant to discussing any separate benchmark application, even when it uses the same engine, unless we're talking specifically talking about in game timedemos (and then other system performance usage issues come into play), or if the benchmark ends up being nothing more than a "timedemoed" version of the game. I think the Comanche benchmark program, for example, might fit that description (though I think there might be concerns with game performance issues that might differ due to patching), but it isn't even being evaluated when popularity is mentioned as a factor in this way. In the case of Aquamark, it might be a "timedemoed Aquanox" as far as performance characteristics, but that's irrelevant to the information gained from it about video cards, and your commentary, for example, doesn't recognize the significance of that distinction in what you state.
That's my opinion...do you really disagree, or did you temporarily get caught up in the hard sell of this "game"/"synthetic"/"popular" idea as being "common sense" that is currently occuring? Or were you just not intending the comments in the context of Aquamark/Aquanox?
If you do disagree, do you have some reasoning to share? Basically, I'm posing the question: Are trying to evaluate the
engine and the relevance of finding out how the game using that engine in the same way (if it is) will perform, or to evaluate the
benchmark and the relevance of what it tells us about the video card? I think when analyzing something as the latter, it is introducing confusion to think popularity of the game engine matters at all.
Please note that nothing stops people from completely changing the factors stressed when they take an engine used in a game and use it for a dedicated benchmark that might or might not resemble the game in question. I'm saying that there is no substitute for examining and evaluating the implementation details when evaluating a video card is the question, and that evaluating how a particular game runs on a video card is a separate question that it just so happens can be somewhat correlated by enough accurate answers to the first, no matter for the popularity of the applications for which it was answered.
