Futuremark Announces Patch for 3DMark03

[CorwinB]

You nicely examplify a standpoint I find fascinating - that actions performed by individuals for the material gain of a corporation are devoid of moral implications either for those individuals or the corporations, whereas if the same actions were performed for purely personal gain, they would have been regarded as morally unacceptable.

Seems strange to me, even though it fits nicely into the tradition that killing on command is fine, whereas killing on your own volition is unacceptable. The moral priviliges that were once the domain of nations (for the greater common good, presumably), are now extended to corporations. Hmm.

Sorry, I wasn't clear then. I just wanted to say that "evil" is too strong a word for that, not to imply that corporations have rights that individuals don't have. If an individual did exactly the same thing (shader replacement in a driver to protect its product), I would also find it deceptive, but not "evil". I prefer to reserve the word "evil" for worse things myself.

 

[Entropy]

You nicely examplify a standpoint I find fascinating - that actions performed by individuals for the material gain of a corporation are devoid of moral implications either for those individuals or the corporations, whereas if the same actions were performed for purely personal gain, they would have been regarded as morally unacceptable.

Sorry, I wasn't clear then. I just wanted to say that "evil" is too strong a word for that, not to imply that corporations have rights that individuals don't have. If an individual did exactly the same thing (shader replacement in a driver to protect its product), I would also find it deceptive, but not "evil". I prefer to reserve the word "evil" for worse things myself.

I'd say that the phenomenon - distancing people from the moral consequences of their actions - is evil in itself, and is the very means that allows a wide scale of ugliness. Unfortunately, its often the old grizzled veterans who perpetrates the attitude. -"Hey, its a business, their job is to make money, what do you expect!" implying that a business is a non-human entity that has a single purpose and thus has no responsibility for the consequences of pursuing that purpose.

A human individual that pursued such a single purpose, regarded people as an interchangeable commodity, held no regard for the consequences of others as he pursued his goals et cetera, would be diagnosed as a psychopath.

Of course, nVidias consistent cheating and deception has had relatively minor impact on individuals. Be happy that they are selling graphics cards, and not safety belt mechanisms, firearms, or medical equipment but do consider the consequences of such attitudes in other segments of business. Are they really acceptable?

 

[Rookie]

now ATI have joined the slover battle

Quote from gamersdepot:

"It's been claimed that Futuremark's changes have disabled compilers. This is complete nonsense. ATI has had a compiler since CATALYST 3.6 and it didn't have any problems with Futuremark's changes. Shader replacement and compilers are completely different operations.

ATI has had a compiler since CATALYST 3.6. We didn't have any problems with Futuremark's changes. The new build of 3DMark03 gives an honest picture of the relative DX9 game performance of graphics cards. It accurately reflects what gamers will see with titles such as Half-Life 2, and what they already see with today's DX9 games, such as Tomb Raider: Angel of Darkness.

Secondly, it is disingenuous to claim that Shader replacement better reflects the performance in games. Only a tiny fraction of games get the attention that 3DMark03 has had from our competitors. It requires too much programming time. Over 300 PC games are launched a year, and 100 of these will really tax the graphics hardware. Maybe a half-dozen - the ones most used as benchmarks - will receive the gentle caress of the driver engineer. An honest run of 3DMark03 will give a true indication of performance for the overwhelming majority of DirectX 9 games. Gamers need to be able to play any game they want; they don't want to be locked into the six that have had all their shaders replaced.

Even assuming that you somehow found the resources to replace all the shaders in every game, it's still not a practical solution. Shader replacement is a massive step back for reliability and game compatibility. Every year you'll be writing thousands of Shader programs that each have to be checked for image quality, taken through QA and supported. And changed whenever the developer issues a patch. Treating every game as a special case is a huge stability issue. Developers have come out against Shader replacement. John Carmack is on record as saying "Rewriting shaders behind an application's back in a way that changes the output under non-controlled circumstances is absolutely, positively wrong and indefensible." The opinions of Gabe Newell, Valve Software's CEO, on Shader replacement are well-known. Developers hate it. What if they release a new level, the gamer downloads it and performance sucks? The hardware vendor isn't going to get any grief, because all the user sees is the old levels working fine and the new one running like molasses in January. The problem's obviously with the game, right? Developers are worried about the support nightmare this approach will generate and the damage to their own brand when they get blamed.â€￾

Chris Evenden
PR Director
ATI Technologies"

 

[Quitch]

I'm glad to see that ATI are finally speaking out.

Over 300 PC games are launched a year, and 100 of these will really tax the graphics hardware. Maybe a half-dozen - the ones most used as benchmarks - will receive the gentle caress of the driver engineer

I do love that quote

 

[cthellis42]

Just talking to Derek Perez on IM at the moment. He has said there are shader replacements in the drivers (fine) and when asked if the compiler was still a work in progress he said that it was finished. Now, if the runtime shader compiler is "optimal" then it should be achieving as good, if not better, performance as a hand coded shader that is mathematically equivelent to the application shader - this being the case, why would you need hand coded shaders...?

Well hrm... Did you frame that question to HIM, we wonders? And did he reply? Heh...

Heck, why not also ask him regarding this point:

I've tried v330 and v340 shaders in another application. They showed exactly the same execution speed. So NVIDIA is doing replacement of these shaders ONLY in 3Dmark03. If a game uses the same shaders it won't have the enhancement.

(I'd love to see some actual numbers and demos out and about to show this point as well.)

Man-o-man, this is going to be one SERIOUSLY wacky week for PR blasting.

 

[Hanners]

The mental image of a nVidia engineer gently caressing a shader is both disturbing and strangely erotic at the same time... :?

 

[Dave Baumann]

Just talking to Derek Perez on IM at the moment. He has said there are shader replacements in the drivers (fine) and when asked if the compiler was still a work in progress he said that it was finished. Now, if the runtime shader compiler is "optimal" then it should be achieving as good, if not better, performance as a hand coded shader that is mathematically equivelent to the application shader - this being the case, why would you need hand coded shaders...?

Well hrm... Did you frame that question to HIM, we wonders? And did he reply? Heh...

Heck, why not also ask him regarding this point:

I've tried v330 and v340 shaders in another application. They showed exactly the same execution speed. So NVIDIA is doing replacement of these shaders ONLY in 3Dmark03. If a game uses the same shaders it won't have the enhancement.

(I'd love to see some actual numbers and demos out and about to show this point as well.)

Man-o-man, this is going to be one SERIOUSLY wacky week for PR blasting.

I did on both counts. No response from the test of the shaders in another app, awaiting a response on why hand coded shaders will remain better than their optimised ones.

 

[Trawler]

Just talking to Derek Perez on IM at the moment. He has said there are shader replacements in the drivers (fine) and when asked if the compiler was still a work in progress he said that it was finished.

Did he specifically say that the compiler was in the currently released driver set? Saying that it's 'finished' sounds a little odd to me.

Now, if the runtime shader compiler is "optimal" then it should be achieving as good, if not better, performance as a hand coded shader that is mathematically equivelent to the application shader - this being the case, why would you need hand coded shaders...?

Being an early iteration of the compiler, I doubt it would be fully optimal, as there's always large room for improvement in early revisions of software. However, I wouldn't expect to see such a huge disparity in results between the handcoded and compiled shaders at this stage.

Edit: Dave, could you ask Mr Perez if he could kindly supply us the source of the replacement shaders for 3dmark03. That way we could compare them to FM's shaders to see if they really are acceptable in terms of providing the same image quality.

Surely he will be happy to provide this? They have nothing to hide, after all...

 

[Exxtreme]

I've tried v330 and v340 shaders in another application. They showed exactly the same execution speed. So NVIDIA is doing replacement of these shaders ONLY in 3Dmark03. If a game uses the same shaders it won't have the enhancement.

Tridam,

can you extract the the vertex shader code and make the same tests?

 

[Magic-Sim]

It's just decent business practice. After all, it's ATI's job (and vested interest) to exploit the hardware's potential.
I don't intend to downplay ATI's achievements, on the contrary. It's just that IMO these kinds of things are best expressed through benchmarks and don't particularly require nor warrant press releases stating the obvious - ie "We are proud to announce that we now have managed to write a driver that finally does what it should have done eighteen months ago."

As has been stated, HLSL is layered ontop of DirectX Graphics, its results are passed to the driver through the plain jane PS2.0 interface. So while I obviously can't speak for ATI, it's rational to assume that the driver treats both types of shaders equally - it doesn't even know the difference, unless I'm hopelessly wrong

Thanks

 

[Dio]

We're always working to improve the optimizer. Stay tuned!

I have just one question : does this compiler work more for hand written shaders ? Or does it work the same for hand written and HLSL compiled shaders (globally, I know hand written shaders can be much better than automatic & genericly compiled shaders ?)

Code that comes out of HLL's tends to have a little more slack in it than asm code, and so one can expect the compiler can do a little better with it.

However, right now there aren't many applications which have really taxed the R300 series much at all - it's only now we're starting to see those that the compiler is going to start being extra smart.

 

[3dilettante]

Perhaps Mr. Perez should be asked how long he thinks the PR department can feasibly maintain operational control of Nvidia without there being irreversible consequences, given a charitable assumption that none have occurred yet.

Does he think the PR department is adequately staffed and briefed on driver and hardware development to be a sufficient substitute for central and even departmental management until such time that said management can resume stewardship?

Does he think Nvidia is sufficiently staffed and briefed on all products utilizing shaders either for sale or in development in order to allow for proper optimizations in order to not limit their current work only to benchmarks and benchmarked games?

Have all potential legal matters been studied and proper clearances made for replacing shaders, especially should there be hand-coded shaders for other programs besides 3dmark, or is he sure that the PR department can take over for the legal department as well?

 

[cthellis42]

I did on both counts. No response from the test of the shaders in another app, awaiting a response on why hand coded shaders will remain better than their optimised ones.

How much you want to bet he's watching old ST:TNG episodes for some Treknobabble inspiration?

Well, you see, we inverse the polarity of our shaders in the compiler, so if you don't regulate the flow to the plasma grid... Pretty soon all the press will be hopping into the Jeffries tubes and looking all which-way for what's going on...

 

[Marc]

Dave, could you ask Mr Perez if he could kindly supply us the source of the replacement shaders for 3dmark03. That way we could compare them to FM's shaders to see if they really are acceptable in terms of providing the same image quality.

Surely he will be happy to provide this? They have nothing to hide, after all...

Well, it will be great ... but i doubt they will

 

[Dio]

Now, if the runtime shader compiler is "optimal" then it should be achieving as good, if not better, performance as a hand coded shader that is mathematically equivelent to the application shader - this being the case, why would you need hand coded shaders...?

Generically compiled shaders should - and can - be every bit as good as hand-written shaders.

That is the exact goal for our shader compiler; to generate the hand-optimised shader in every single case. It's a far more effective use of expensive engineers' time to improve the compiler than each individual shader.

I'm not saying we'll achieve this, but it will be close, and if we find something that's not coming out right... well, we should be rewriting the compiler, not the shader!

(And there's another point: as someone who's seen quite a few 80-instruction shaders recently, let me say that hand-optimising five of them could take a month. When we get onto 2000-instruction shaders.... well, I'll be really glad we have a good compiler).

 

[Dave Baumann]

Did he specifically say that the compiler was in the currently released driver set? Saying that it's 'finished' sounds a little odd to me.

My exact question was "I assume the compiler is still a work in progress" to which he replied "No, its finished".

Being an early iteration of the compiler, I doubt it would be fully optimal, as there's always large room for improvement in early revisions of software. However, I wouldn't expect to see such a huge disparity in results between the handcoded and compiled shaders at this stage.

Well, this is the thing - I don't think it is an early iteration. As has already been said, they have been using it for numerous driver releases. It seems as though the 52.16 was the "final" push of it.

Edit: Dave, could you ask Mr Perez if he could kindly supply us the source of the replacement shaders for 3dmark03. That way we could compare them to FM's shaders to see if they really are acceptable in terms of providing the same image quality.

Surely he will be happy to provide this? They have nothing to hide, after all...

Yeah, that would never happen.

That is the exact goal for our shader compiler; to generate the hand-optimised shader in every single case. It's a far more effective use of expensive engineers' time to improve the compiler than each individual shader.

Yeah. And I flat out said that as they are still doing replaced shaders then that suggests to me they are not mathematically equivelent.

 

[PSarge]

Just a point on hand optimised shaders.

The one thing a compiler can't do when optimising a shader is say "These LSB's aren't important. I'll drop them, the effect will be minimal". A compiler must produce something which is mathmatically equivilent to the original shader. A human can use their judgement.

Now that the Fire, Hair, Fog diffs have been discounted I think what we're left with is lower precision, hand tuned shaders, which in nearly all cases produce the same ouput. It's those few bits that don't which are causing the speckles in those comparison shots.

One question.Dave, In the comparision tool you're using does the colour in the difference image correlate to the size of the difference? or is it doing some kind of auto-contast to make things visible? I have a feeling these PP versions of the shaders are very good, and it's only because people are on hyper alert that it got noticed.

Edit : looks like dave has made some of my post redundant above.

 

[Dave Baumann]

Brightness is increased.

One thing though - NVIDIA have two sets of FX's: NV35/38/36 with full float and NV30/31/34 with mixed float and Int. Someone has said to me the IQ look awful on a 5200 (NV34) so there might be other replacements for the boards with Int precision as well as float (the comparison shots were done on an NV35).

 

[Dio]

A human can use their judgement.

'Intuition' and similar things are a lot less important in optimisation than you might think. One might believe one is being instinctive, but I actually found that almost all of my hand-optimisation was rule-based - that surprised me.

The difficult bits are finding a data encoding that allows you to apply the rules, and coding the rules.

 

[Sharkfood]

Wow, am I ever impressed with Chris Evenden of ATI. His comments were spot on and truly prophetic of where such practices would evolve the gaming market.

"Gamers need to be able to play any game they want; they don't want to be locked into the six that have had all their shaders replaced."

I find it unbelievable that anyone would accept or otherwise not be appauled by the concept of writing custom shader's for games and benchmarks. Developers might as well simply leave blank stubs to call the IHV's custom shaders for their games if they didnt intend for their shader code to be used.

It's been pointed out what the process is.. It's been pointed out the selective targets (i.e. benchmarks/games with benchmarks) where this is being implemented, yet some sources (*cough* HardOCP *cough*) are still confused towards what impact this has and where it would evolve to.

I stand a bit baffled and bewildered how such simple concepts can be totally missed by the few.