Originally Posted by Andrew Lauritzen
7) This is precisely why it is important to measure frame latencies instead of frame throughput. We need to keep GPU vendors, driver writers and game developers honest and focused on the task of delivering smooth, consistent frames. For example (unrelated to the above case), allowing things like a driver to cause a spike one frame so that it can optimize the shaders and get higher FPS number for the benchmarks is not okay!
8) If what we ultimately care about is smooth gameplay, gamers should be demanding frame latency measurements instead of throughput from all benchmarking sites.
I don't know what is going on with Borderlands 2, I work on OpenCL. However, I think you are being a bit too judgmental here. First, when a new effect is used, the driver must compile the shaders involved. Yes, this means optimizing the shaders too. You can experience that as a "hitch" during gameplay, but once the effects are compiled, you won't experience that hitch again, unless the shaders are recreated (i.e. next level :P). Second, some features are emulated now (think fog). So if an API feature were enabled that required recompilation of shaders, then you might experience a "hitch". Third, consoles can avoid all of this easily as applications can ship precompiled shaders for every effect used.
OpenCL also allows for precompiled kernels to be saved/loaded. This is an important feature as some kernels are huge and can take several minutes to compile. This also allows for IP protection as you can just ship a stripped binary and not your source code. This would be a nice feature for Direct3D, but I don't imagine game developers would want to ship precompiled shaders for every possible GPU out there. So it would be better if the game could compile the shaders as they are used and save them to the hard drive. Then the game would just reload the binaries as they are used/needed. If you happened to change GPUs in your machine, then the game would have to recompile the shaders again, but that wouldn't be a huge deal unless you were changing GPUs constantly. Of course, this wouldn't necessarily catch cases where state changes caused recompilation. Perhaps you could create a query to check if there was recompilation.
PCs and consoles give different experiences, but some of this is not really under the control of the driver/GPU.