Crusher said:
First, you can say that my computer getting 30 3DMarks shows how useless it's going to be on DX9 games. The fact that it's going to be useless on DX9 games is probably true. However, that score is based only on Game Test 1, since that's the only test my computer can run. Game Test 1 is supposed to be a DX7 benchmark. Well, last time I checked, the GeForce 2 was one of the best DX7 cards around. I think the peak framerate I saw displayed was 8 fps, and it usually was below 3 fps. That's not indicitive of any DX7 game I know. Even UT2003 and NOLF 2 run much better than that on my computer, and they're about the most demanding DX7 games I've seen.
That tells me that Game Test 1 is clearly not indicitive of the type of situation it's meant to portrait. If GT1 can't properly judge the abilities of a DX7 card running a DX7 game, how is it going to properly judge the abilities of a DX8 or DX9 card running a DX7 game? Now you might say, "who cares? there are DX7 games out to test with if you want to know how well a card works with DX7 games. 3DMark03 is supposed to compare cards with DX8 & DX9 level games." That's all well and good, except that Game Test 1 still counts towards the final score on DX8 and DX9 cards. So here we have a clear example of how the final 3DMark03 score is being based in part off an irrelavent and inaccurate test.
This is a common misconception about what the target of a future-looking benchmark like 3DMark03 is or should be. That is, a lot of people look at 3DMark03 and think that the four tests are supposed to represent:
- a game that might be released today
- a game that might be released ~6 months from now
- a game that might be released ~12 months from now
- a game that might be released ~18 months from now
I can't claim to speak for Futuremark, but IMO that's not their aim at all. Instead, these four tests are supposed to represent
4 games that might be released around 18 months from now. That's why it's a great choice to make GT1 a flight simulator, because it's quite likely that this is one of the game genres most likely to still not be using pixel shaders in games released in late 2004. Note further that while Nvidia's market research may give "flight simulators" only 1% of the market, there are a number of other genres (e.g. outer space RTS) that are also characterized by most of the screen being covered with a single-textured skybox.
The next thing you need to realize about GT1 is that it's
not a DX7 benchmark.
It is a DX8 benchmark with a fallback to DX7: that is, it uses VS 1.1, but if the GPU does not do hardware vertex shading it can emulate the functionality on the CPU. Moreover, this is almost entirely to blame for you score of 30 with a GF2 and (IIRC) a Celeron 5xx@8xx; your Celery is based on the Mendocino core and does not support SSE1, which is very helpful for running vertex shading calculations. For example, my phat gaming rig (1.2 GHz Thunderbird, GF2MX 400) scores a towering 130, despite a GPU with half the fillrate and bandwidth of a GF2 GTS. (Yeah 130! Suck it down!! 8) )
The use of VS 1.1 instead of fixed-function T&L was quite deliberate, and is discussed in the White Paper. GT1 is as much a DX7 benchmark as DoomIII is a DX7 game, which is to say, not much.
In any case but your poor SIMD-lacking Celeron (well, on the bright side it has MMX), the real beef Nvidia has with GT1 is that it's mostly single-textured (again, extremely appropriate both for the flight simulator genre and for the "type of game likely to use a simple, pixel shader-less engine in late 2004" category), which puts their current lineup at a huge disadvantage. Every Nvidia GPU since the GF1 has had either a 2x2 or 4x2 organization. Meanwhile, every ATI GPU in their current lineup (with the exception of the 8500/9100, which is being phased out) is either 4x1 or 8x1. So obviously a mostly single textured benchmark is going to make ATI's cards look better.
The question is, who's "right"? Well, considering every future GPU core with known specs is nx1 (including NV3x, of course), and that loopback has removed nearly all the reasons for organizations other than nx1, it's safe to say ATI's lineup is the more forward looking in this regard. It's also interesting to note that all of these nx1 GPUs from ATI have been released in the time since Nvidia last released a GPU.
What good does it tell you to know that one card is faster than another at rendering stencil shadows when calculating the geometry redundantly at intermediate steps in the rendering process, if no game is ever going to do that? Does it tell you how fast their vertex shaders are relative to one another? Or does it tell you how much each card stalls when throwing in redundant processing at different points? Or does it say something completely different? The one thing that you know it doesn't say is how fast Doom 3's vertex processing will be.
First off, as I understand it, DoomIII will "calculate the geometry" just as redundantly (i.e. on every pass: 1 + 2-per-light), in the sense of doing T&L on every pass. It's just that it does the skinning once per frame (on CPU) and sends the info over the AGP bus (on every pass?). (Or does DoomIII do all geometry calculations, including transform (I guess there's no per-vertex lighting), on the CPU, i.e. no use of hardware vertex shaders at all?)
Multipass inherently means redoing the geometry for every pass. That's why DoomIII has such a low poly count (per scene) relative to other high-end games.
As for reskinning with every pass:
As OpenGL Guy has pointed out and as Ichneumon's with/without PS1.4 tests have proven, the Radeon 9700 Pro isn't vertex shader limited running GT2 or GT3. (PS 1.4 only gives a ~23% boost over PS 1.1 on 9700P, despite almost 100% higher geometry.) And indeed, there's no good reason I can see why
any modern card (or even a year-old one like GF4) should be vertex shader limited from having to skin what is likely not more than a few hundred thousand polygons a second. After all, only dynamic joints (as in skeletal animation) need to be skinned, and GF4 Ti4600 only gets 12.6 and 10.5 fps in GT2 and 3 respectively. Skinning, as I understand it (and as Futuremark claims) is a very light workload, and with such low fps, the number of skinned vertices per second should not put a serious damper on any GPU that is honestly designed for more than a trivial application of vertex shaders.
With the ammount of programability in DX9, there's a vastly greater number of ways of performing the same type of task. A hardware light is a hardware light in any program, and the T&L functionality is the same across all cards, the only thing that differs is the performance. Pixel shading version 1.1 was also fairly limited in what it allowed. Pixel shading 2.0, Vertex Shading 3.0, and the various ways of breaking them down to allow support for cards with lesser versions are completely different situations. To take the specific way 3DMark03 implements these features and the way the perform fallbacks for older cards, and say it's indicitive of the overall ability of one card compared to another is rediculous.
I strongly suspect that this is incorrect; that because shaders are run on general-purpose instead of fixed-function hardware, a GPUs relative performance on a certain set of PS 2.0 shaders will be a very good proxy for all PS 2.0 shaders with a similar length and instruction mix. (And it is a good bet that the shaders in 3DMark03 are at least similar to what real shaders will be like w.r.t. length and instruction mix.) However, I'm not an expert on this so I'll defer to someone who can shed a bit more light on this.
And that brings up the other twist--the fact that the majority of game developers inherintly try to do their best to make sure games perform acceptably on all makes of video cards. If a developer does things the way 3DMark03 does them and finds tremendous descrepancies between how different cards from different vendors and/or different generations perform those actions, they will probably change the way they're doing things. You can argue the extent to which things will change, or the number of developers who will ultimately make such decisions, but that doesn't change the fact that it's a relevant variable.
This is an interesting point, and quite rightly made. It brings up an interesting dilemma in any purpose-made benchmark software. Benchmarks are supposed to be impartial among IHVs. And benchmarks are supposed to reflect real software performance--in this case, "real" (projected) games circa 2004 or so. The problem is that real games are not impartial among IHVs; they will tailor their engines to run well on whatever cards have the largest installed base, even if this is "unfair" in the sense that it disadvantages GPUs with dissimilar designs.
An interesting dilemma but, in this case, not a relevant one. As I've said, these tests are all supposed to represent various games released in the late 2004 time frame. And, as Nvidia has so
helpfully pointed out, the decisions taken in 3DMark03 which most directly disadvantage Nvidia hardware are to make GT1 mostly single-textured and to use PS 1.4 (with PS 1.1 fallback) heavily in GT2 and GT3. Happily, starting with NV30 these decisions will no longer disadvantage Nvidia hardware in any way. All future Nvidia cores (certainly all cores released between now and late 2004) are likely to be nx1 designs, and certain to support PS 1.4. Come to think of it,
all near-future discrete GPUs are likely to be nx1 and support PS 1.4.
Yes, games released in late 2004 will still be written to advantage Nvidia cards. (Assuming, as is likely, they maintain their high marketshare.)
But this will no longer mean avoiding PS 1.4 or single texturing. By that time, the GF4 Ti will be over 2.5 years old. Would you expect a game released next month to significantly modify its engine in order to cater to GF2 GTS owners? No? Then you shouldn't expect a late 2004 game to make the changes Nvidia suggests, either.
