Futuremark Announces Patch for 3DMark03

[Magic-Sim]

I follow your remark : even people with good background knowledge miss the point.

Rewriting shaders is a painful hassle, a loss of time and money, and it is just creting a pale copy of the original work.

Recompilation when done properly can save time for the developper, fot the IHV, and should normally output the good...... output.

Nevermind, it is sometimes very difficult to see all the facets of a problem. Mostly when it is flooded with awfully written PR-pieces.

I am pleased and suprised by the one issued by ATI. Clean, neat, factual, and thoughtful.

And they do not find necessary to quote ther Nasdaq reference etc copyright informations blablabla. This shows to that ATI truly want to communicate on this, and not play the PR-wheel.

 

[Rolf N]

Extremely OTish

A human can use their judgement.

'Intuition' and similar things are a lot less important in optimisation than you might think. One might believe one is being instinctive, but I actually found that almost all of my hand-optimisation was rule-based - that surprised me.

The difficult bits are finding a data encoding that allows you to apply the rules, and coding the rules.

Can I pull you aside for a second?
I've pondered the basic idea of treating every target as VLIW/MIMD for a while ...
Ie, find parallel dependency chains and 'drop' independent operations into slots of a VLIW packets until you find a best match, where "best" is defined as a combination of a)maximum source operations consumed adjusted by b)a weight attached to the type of instruction packet you end up generating.

Ummm ... like, several types of packets, pure texture sampling packets, MAD+MUL+use constant packets, TEX+TEX+LRP+ADD packets, etc. Each with a predetermined 'goodness' factor attached to them, which could be further adjusted wrt to the last issued packet (to model scheduling issues).

This was about CPU based runtime stream kernel compilation. Should be pretty Oish. I unfortunately didn't go very far yet
Does this sound sensible? 8)

 

[Rolf N]

I am pleased and suprised by the one issued by ATI. Clean, neat, factual, and thoughtful.

Hell yeah! They actually used human vocabulary
"Developers hate it."
"<...> performance sucks?"
"running like molasses in January"

 

[Magic-Sim]

I am pleased and suprised by the one issued by ATI. Clean, neat, factual, and thoughtful.

Hell yeah! They actually used human vocabulary
"Developers hate it."
"<...> performance sucks?"
"running like molasses in January"

Well, I had a big time to translate the statement in french. I was too much laughing sometimes

 

[XForce]

Well, I had a big time to translate the statement in french. I was too much laughing sometimes

I believe you did, as would I!

It's tragic that I can't share this with some of my friends, who have no clue WTF shaders are anyway.

Which leads us to the core of what substained NV for so long:
they've taken this to a technical level where the general public has no chance of understanding what's actually going on.

People love simple answers.

Example #1
"FM is siding with Ati to make NV look bad, don't use the evil evil performance crippling 340 patch"

Example #2
"Patch 340 can't disable the shader compiler, same as no other application could. Anyway, if it were possible and done, drivers which factually don't have this compiler yet wouldn't be affected, yet they are in exactly the same way"

Take your pick.
Ok, good choice.

Now go and ask the kid clerk in your local computer store.

 

[AndY1]

http://www.xbitlabs.com/news/video/display/20031114041519.html

Luciano Alibrandi, European Product PR Manager for NVIDIA Corporation, has made a correction in regards previous information about NVIDIA’s Unified Compiler and 3DMark03 benchmark after getting into details with the company’s engineers. Apparently, the statement claiming that NVIDIA’s Unified Complier deployed to optimize pixel shader performance is disabled by the new version of 3DMark03 is not fully correct.


“I would like to inform you that a part of my response was not accurate. I stated that the compiler gets disabled, by 3DMark and that is in fact not true,â€￾ he said.

So, after all NVIDIA denied the problems between the Unified Compiler technology and the latest version of popular 3DMark03 benchmark. As a result, we may now conclude that the accusations in Futuremark direction from Hans-Wolfram Tismer, a Managing Director for Gainward Europe GmbH were not correct at all.

In October 2003 Santa Clara, California-based NVIDIA Corporation introduced its Unified Compiler integrated in its ForceWare 52.16 drivers to optimize Pixel Shader code for NVIDIA GeForce FX architecture in an attempt to improve performance of graphics cards powered by NVIDIA’s latest GPUs in variety of demanding applications.

NVIDIA said that the Unified Compiler technology tunes DirectX 9.0 execution on the GeForce FX GPUs, and can be used to correct any similar conflict that arises with future APIs. NVIDIA indicated the Unified Compiler as an automatic tuning tool that optimizes Pixel Shader performance in all applications, not just on specific ones. Officials from NVIDIA again stressed today that one of the things the Unified Compiler does is to reinstruct the order of lines of code in a shader. By simply doing this the performance can increase dramatically since the GeForce FX technology is very sensitive to instruction order. So, if the re-ordering is not happening NVIDIA’s GeForce FX parts have a performance penalty.

Since the complier is still active with the new version of 3DMark03 there is currently no explanations for performance drops of certain GeForce FX parts in the latest build 340 of the famous 3DMark03.

“The only change in build 340 is the order of some instructions in the shaders or the registers they use. This means that new shaders are mathematically equivalent with previous shaders. A GPU compiler should process the old and the new shader code basically with the same performance,â€￾ said Tero Sarkkinen, Executive Vice President of Sales and Marketing for Futuremark Corporation – the developer of 3DMark03 application.

He was indirectly confirmed by an ATI official yesterday, who said: “ATI has had a compiler since CATALYST 3.6. We did not have any problems with Futuremark’s changes.â€￾

 

[Sharkfood]

Although this was discussed a bit on the first few pages of this thread (and maybe a new thread is in order)... it has now been totally glanced over during this whole debacle- what's to come of FutureMark?

FutureMark released a PDF on what they define to be "valid" drivers and have also included the 52.16s on their list of what they consider to be valid/verified drivers that are supposed to adhere to the guidelines set-forth in their PDF.

The problem is- the 52.16s clearly break most every "rule" that was portrayed in the FutureMark PDF. So they are validating/certifying drivers that have all evidence of NOT comforming to the very rules they have set forth for validation.

The recent 3DMark patch is a tool that is used to illustrate this point exactly, but it can in no way be certain they have captured or otherwise squelched all cheats that may exist in these drivers. By defeating *some* cheats, they have clearly illustrated these drivers (by their own rules lol) need to be disqualified.

I'm left scratching my head why the 52.16's hit the FutureMark "certified" list... and what it does to the (now useless) PDF. Without either completely discarding the PDF, or modifying it dramatically, it becomes a big paradox by it's own terms for as long as the 52.16's sit on their list of allowable drivers.

 

[nggalai]

http://www.xbitlabs.com/news/video/display/20031114041519.html

Since the complier is still active with the new version of 3DMark03 there is currently no explanations for performance drops of certain GeForce FX parts in the latest build 340 of the famous 3DMark03.

Err, LOL?

93,
-Sascha.rb

 

[CorwinB]

Since the complier is still active with the new version of 3DMark03 there is currently no explanations for performance drops of certain GeForce FX parts in the latest build 340 of the famous 3DMark03.

Well, there is a very simple explanation, more or less comfirmed by none other than Derek Perez himself... The compiler is doing both genuine generic optimizations (register reuse, reordering...), which are not hindered by the 340 patch, and specific shader detection/replacement based on shader fingerprint, which is against 3DMark's rules, and removed by the 340 patch...

How someone could call specific shader replacement part of a "unified compiler technology" still boggles my mind, though.

IMHO, M. Alibrandi's backpedaling is a direct result of the Internet backlash and the ATI press release regarding their ridiculous "compiler is disabled" statement... Or a realisation that saying that application had to be "compiler-aware" would be in the long term more devastating to their already much shattered reputation than a few hundreds 3DMark points. They are running out of options and wiggle-room very quickly on this case.

Officials from NVIDIA again stressed today that one of the things the Unified Compiler does is to reinstruct the order of lines of code in a shader. By simply doing this the performance can increase dramatically since the GeForce FX technology is very sensitive to instruction order.

Ok, no problem so far.

So, if the re-ordering is not happening NVIDIA’s GeForce FX parts have a performance penalty.

And that's where the problem is. Since the compiler is not disabled (as understood by anyone with a little technical knowledge, and admitted by Nv), then why on Earth would that re-ordering not take place ? Especially since when running the shaders from v330 and v340 in another program, they get the same result ? The only technical explanation is of course that there is no re-ordering taking place because this specific "optimization" does not rely on generic re-ordering, but in this case on app-detection, something explicitely forbidden by 3DMark.

As a result, we may now conclude that the accusations in Futuremark direction from Hans-Wolfram Tismer, a Managing Director for Gainward Europe GmbH were not correct at all.

I see Gainward moving to ATI soon... Their Managing Director is sure bound to love looking like a fool to buy Nvidia some time.

 

[PatrickL]

They just need to add something along theses lines:

What is a certified driver for FM?
- A driver that follows the guidelines
OR
- A driver with innacurate behaviour disabled by current built.

Would be maybe more clear for FM to write that down if not done already

 

[Exxtreme]

http://www.xbitlabs.com/news/video/display/20031114041519.html

Since the complier is still active with the new version of 3DMark03 there is currently no explanations for performance drops of certain GeForce FX parts in the latest build 340 of the famous 3DMark03.

Sorry but this is really ridiculous.

 

[whql]

IMHO, M. Alibrandi's backpedaling is a direct result of the Internet backlash and the ATI press release regarding their ridiculous "compiler is disabled" statement... Or a realisation that saying that application had to be "compiler-aware" would be in the long term more devastating to their already much shattered reputation than a few hundreds 3DMark points. They are running out of options and wiggle-room very quickly on this case.

no, the backpedaling is just because a number of people spoke before they had the full party line from HQ - yesterday there were two or three different conflicting messages from different parts of the NVIDIA family, but now those are being gathered up and mashed into a single message now that the US are dictating what should be said.

Still, the backpedaling will do nothing but confuse the uninitiated and make them doubt what they are saying.

 

[Simon F]

Which leads us to the core of what substained NV for so long:
they've taken this to a technical level where the general public has no chance of understanding what's actually going on.

People love simple answers.

Example #1
"FM is siding with Ati to make NV look bad, don't use the evil evil performance crippling 340 patch"

Example #2
"Patch 340 can't disable the shader compiler, same as no other application could. Anyway, if it were possible and done, drivers which factually don't have this compiler yet wouldn't be affected, yet they are in exactly the same way"

How about this answer -
(DISCLAIMER This is all speculation of course...)

#3 A hypothetical company currently has a poor quality compiler but it probably expects it to improve in future revisions. To make their system look better in the interim (until they can make the compiler more intelligent) they put in some customised, hand-coded examples. FM changed their code so it no longer matches those examples.

 

[Dave Baumann]

Simon, read a few posts back - I think that is out the window since work on the compiler has stopped, i.e. its already optimal for "mathematically equivelent" shaders. Its probable that the hand coded shaders are now wholly mathematical equivelent.

 

[Bouncing Zabaglione Bros.]

They just need to add something along theses lines:

What is a certified driver for FM?
- A driver that follows the guidelines
OR
- A driver with innacurate behaviour disabled by current built.

Would be maybe more clear for FM to write that down if not done already

According to FM it's the latter. They have stopped Nvidia from cheating, therefore the driver is valid for their benchmarks. It seems to be a way to get a baseline so that Nvidia is inlcuded in the Futuremark programme, but it could be possible that Nvidia never get another "approved" driver while they are still cheating.

Of course there are still doubts raised in this and other threads that FM have actually caught all the cheats.

 

[nggalai]

no, the backpedaling is just because a number of people spoke before they had the full party line from HQ - yesterday there were two or three different conflicting messages from different parts of the NVIDIA family, but now those are being gathered up and mashed into a single message now that the US are dictating what should be said.

Exactly. I work in PR on my day job (ok, we prefer to call it "marketing and communications" ), and the way NV handled the past couple of months seems to me like a number of people will be looking for new jobs come next quarter. Accidents DO happen, but everything concerning Futuremark has been handled rather ... let's just say not the way big corporations should do it.

I wonder who'll still be around 2004, of the PR blokes.

93,
-Sascha.rb

 

[Ostsol]

http://www.xbitlabs.com/news/video/display/20031114041519.html

Since the complier is still active with the new version of 3DMark03 there is currently no explanations for performance drops of certain GeForce FX parts in the latest build 340 of the famous 3DMark03.

Sorry but this is really ridiculous.

Heheh. . . In otherwords, it's a bug -- just like the custom clipping planes!

 

[Simon F]

Simon, read a few posts back - I think that is out the window since work on the compiler has stopped,

I saw that but disregarded it. It's like saying that work on the drivers has been stopped.

i.e. its already optimal for "mathematically equivelent" shaders. Its probable that the hand coded shaders are now wholly mathematical equivelent.

Did you instead mean to write "mathematically approximate" ?

 

[Dio]

Re: Extremely OTish

Can I pull you aside for a second?
I've pondered the basic idea of treating every target as VLIW/MIMD for a while ...
Ie, find parallel dependency chains and 'drop' independent operations into slots of a VLIW packets until you find a best match, where "best" is defined as a combination of a)maximum source operations consumed adjusted by b)a weight attached to the type of instruction packet you end up generating.

This was about CPU based runtime stream kernel compilation. Should be pretty Oish.

Scheduling is non-trivial and there's dozens of different ways of approaching the problem. The fundamental issue is that every instruction potentially affects every other instruction. What's right for the application depends on what your input looks like, and what your scheduling priorities are.

O scheduling algorithms often only have a fairly linear view of things, and therefore make bad guesses because they doesn't know what's coming later. Which isn't to say that it can't give good results, but it is certainly likely to be a bit on the wobbly side (in terms of the quality of output code) unless it's shored up carefully, perhaps using some multipass approach to derive data for tuning heuristics.

Scheduling is the one place I've found where a human can do better than a compiler because the human can better spot fourth-order changes, 'If I move this, swap this, invert that, and invert that, then this fits in this slot here'; but humans make frequent mistakes in scheduling (because it's terribly easy to miss some dependency) and the compiler often spots things that a human misses particularly as the code gets longer.

I'm glad you didn't mention algorithmic complexity on Tuesday. Something which worked perfectly reasonably for months turned out to have an O(2^n) pathological case. Ouch. Spent four hours debugging that assuming it was an infinite loop...

 

[nelg]

#3 A hypothetical company currently has a poor quality compiler but it probably expects it to improve in future revisions. To make their system look better in the interim (until they can make the compiler more intelligent) they put in some customised, hand-coded examples. FM changed their code so it no longer matches those examples.

Where have I heard that before?