Deano Calver
Newcomer
Yesterday I went to Mojo Reloaded (ATI, MS and Intel game developer day) at Guildford. These events are always a combination of interesting presentations and catching up with the industry gossip. This one for me was far more gossip then prensentation, not that the presentations weren't interesting, My favs were the PRT/SH and non linear post-processing streaks (I'll give a better description in a minute). As this will be up on the web for some time its worth stating what day yesterday was, a day after the HL2 benchmarks were released.
The gossip on this was fairly dominant throughout the day about the benchmarks and comments putting NVIDIA in a bad light at Dx9 shaders.
Obviously ATI weren't that upset to see there hardware coming ahead so well, but what wasn't prehaps so expected was how glad everybody else was that HL2 results matched the results most of us had already seen ourselves. Somebody on the forums (sorry can't remember who) asked why developers seemed quite shy in stating our results, I obviously can't talk for everbody but the answer is probably a simple case of somebody had to first and whoever that person/company was, they better be able to handle the heat that it would produce.
Valve are fairly lucky, HL2 is probably the most eagerly awaiting tltle in the business, everybody has being doing everything to get the best results for this title. Everybody knows these guys are shit hot, they know what there doing and if they can't get good results, something is wrong and its likely not to be them. Smaller developers (and thats probably everbody except iD in the PC arena) don't have that luxery, If I had produced a similar performance table, the response of a lot of people would simply be, that the developers (i.e. me) don't know what there doing and obviously can't program properly. And why not? I don't have the reputation for quality that Valve or ID has, they've earnt there rights to be trusted that they know what there doing.
For Valve to do this, shows they were really annoyed, also the fact Microsoft issued a press release stating HL2 was the DirectX 9 benchmark also show how annoyed they were. To get these two massively important PC games companies to make such a public condemnation means you had to do something bad, just having bad performance wouldn't have been enough.
The basic problem that NVIDIA has caused has been the amount of extra work they've been requiring everybody else to do. Wether its benchmark's having to get smart and try and stop application specific optimisations, or developers having to write extra pipelines to even get half decent preformance at the high tech things its meant to be good at or MS having to upgrade the WHQL test to find spec violations. Everbody has been forced to pay for NVIDIA's mistakes, and that is what has caused the anger.
But in some ways it has had good consquences, quality should go up as loopholes are closed.
Future DX specs should now be much tigher.
WHQL testing to require pixel comparision tests.
Hardware must produce almost exact rendering of the same frame as the REFRAST
Self certification of WHQL, to make sure that WHQL driver will have bug fixes applied quicker without bypassing the quality checks.
Reviewers should be less quick to use 'special' drivers provided by the IHVs or test only under 'special' conditions.
Long term the biggest change this year long fiasco has caused will be to Microsoft and PC game developers. Microsoft have had to learn to protect its baby Direct3D, before its largely left quality and stability issues upto individual IHVs, now it knows that this is also its reputation thats damaged when IHVs play dodgy quality games. And us humble game developers have learnt we have to shout sometimes to protect our games from bad discisions made by IHVs, we can't just mop up the shit when it hits us. We have to be willing and able to communicate that certain things are NOT acceptable to our customers, so don't bother doing it. If your card is crap at something, at least be honest earlier on, don't make us find out when our games runs like a dog on your hardware, even though were using the techniques you've been suggesting for the last year.
The gossip on this was fairly dominant throughout the day about the benchmarks and comments putting NVIDIA in a bad light at Dx9 shaders.
Obviously ATI weren't that upset to see there hardware coming ahead so well, but what wasn't prehaps so expected was how glad everybody else was that HL2 results matched the results most of us had already seen ourselves. Somebody on the forums (sorry can't remember who) asked why developers seemed quite shy in stating our results, I obviously can't talk for everbody but the answer is probably a simple case of somebody had to first and whoever that person/company was, they better be able to handle the heat that it would produce.
Valve are fairly lucky, HL2 is probably the most eagerly awaiting tltle in the business, everybody has being doing everything to get the best results for this title. Everybody knows these guys are shit hot, they know what there doing and if they can't get good results, something is wrong and its likely not to be them. Smaller developers (and thats probably everbody except iD in the PC arena) don't have that luxery, If I had produced a similar performance table, the response of a lot of people would simply be, that the developers (i.e. me) don't know what there doing and obviously can't program properly. And why not? I don't have the reputation for quality that Valve or ID has, they've earnt there rights to be trusted that they know what there doing.
For Valve to do this, shows they were really annoyed, also the fact Microsoft issued a press release stating HL2 was the DirectX 9 benchmark also show how annoyed they were. To get these two massively important PC games companies to make such a public condemnation means you had to do something bad, just having bad performance wouldn't have been enough.
The basic problem that NVIDIA has caused has been the amount of extra work they've been requiring everybody else to do. Wether its benchmark's having to get smart and try and stop application specific optimisations, or developers having to write extra pipelines to even get half decent preformance at the high tech things its meant to be good at or MS having to upgrade the WHQL test to find spec violations. Everbody has been forced to pay for NVIDIA's mistakes, and that is what has caused the anger.
But in some ways it has had good consquences, quality should go up as loopholes are closed.
Future DX specs should now be much tigher.
WHQL testing to require pixel comparision tests.
Hardware must produce almost exact rendering of the same frame as the REFRAST
Self certification of WHQL, to make sure that WHQL driver will have bug fixes applied quicker without bypassing the quality checks.
Reviewers should be less quick to use 'special' drivers provided by the IHVs or test only under 'special' conditions.
Long term the biggest change this year long fiasco has caused will be to Microsoft and PC game developers. Microsoft have had to learn to protect its baby Direct3D, before its largely left quality and stability issues upto individual IHVs, now it knows that this is also its reputation thats damaged when IHVs play dodgy quality games. And us humble game developers have learnt we have to shout sometimes to protect our games from bad discisions made by IHVs, we can't just mop up the shit when it hits us. We have to be willing and able to communicate that certain things are NOT acceptable to our customers, so don't bother doing it. If your card is crap at something, at least be honest earlier on, don't make us find out when our games runs like a dog on your hardware, even though were using the techniques you've been suggesting for the last year.
