These gains are suspect because they look like a case of hand tuned optimization and instruction scheduling by the NVidia driver guys, as we haven't seen any other shaders improving in performance. A case in point is the digit-life review, where all their shaders run slow on NV30 even with the new drivers.
This is a very good point. Although, just to play devil's advocate for a moment, it's worth pointing out that while the Digit Life review--with new drivers--showed GFfx losing to 9700 by a factor of ~2 on the Rightmark shader tests, back a few weeks ago Brent posted on this forum some Shadermark results--with the old drivers--that showed GFfx losing by a factor of ~3. (edited for Tagrineth) Is it the new drivers, or the difference in tests? I dunno.
Still, there are explicit comments from Nvidia engineers to the effect that the new drivers hand-optimize for the 3DMark03 shaders, if not by actually optimally scheduling the shaders in the drivers themselves, at the least by deciding case-by-case whether to run a PS 1.4 shader in NV30's general-purpose PS 1.4-2.0+ shader pipeline or to break it up into several passes through the legacy PS 1.0-1.3 path.
Now, the first thing to be said is that these statements are obviously part of Nvidia's campaign to discredit 3DMark03. They're probably true, but just as clearly Nvidia (and ATI) do the same sort of thing for previous 3DMark benchmarks, and for any games heavily used by benchmarking sites; the only difference is that they don't proclaim it so publicly and in a way that suggests the tests themselves are invalid.
The second thing to note is, golly, it sure doesn't say good things for NV30's real shader pipeline if emulating a PS 1.4 shader with 2 or 3 PS 1.1 passes is faster in many cases.
Now, with those two things out of the way I think it's obvious that normal games (i.e. outside of the 4 or 5 somehow annointed as benchmarks) are not going to get special case treatment in the drivers. On the other hand, it's also obvious that developers are going to optimize their games for Nvidia GPUs. If it's "just" a matter of trying the same effect in both PS 1.4 and multipass PS 1.1 configurations, and putting in a special path for Nvidia cards to do whichever is faster, most developers are going to do it (especially as they will have help from Nvidia devloper relations). So in some sense, it's not
really that unrepresentative if the special optimizations made in the drivers are likely to be done by developers for most games.
If it's a matter of explicitly scheduling the shader in a way different from what the general-purpose compiler in the driver would do, then of course developers aren't going to have the ability to do that on their own. I find it difficult to believe that NV30's shader pipeline is so difficult a compiler target that future drivers won't be able to at least approach the performance of hand-optimized code. Still, I'd much rather wait for drivers that optimize in the general case than see hand-tuned results right now.
In case I sound inconsistent: the difference between the two cases is that in the first the developer has the ability to do the optimization themselves and likely will, but in the second they can't (unless perhaps if they use the proprietary NV OpenGL extensions).