Futuremark Announces Patch for 3DMark03

[madshi]

P.S: Now I'd like to see an editorial by Beyond3D or by another respected site with the topic "why NVidia's reply is bullshit", maybe with some nicer words. Dave, any chance for a front page comment on NVidia's recent PR crap?

 

[nggalai]

Kyle's approach:

http://www.hardocp.com/article.html?art=NTQ5

Derek again, I may add. This one made me laugh:

Clearly our compiler has gotten much better, as image quality remains exactly the same, the only thing that happens is a 10-15% drop in performance.

We're not sure why anyone would want to reduce their performance by 10-15% for the same image quality, but apparently Futuremark feels that is something relevant.

Overall, though, I have to say that now my brain hurts, and I needed a good smoke after reading that ... article.

93,
-Sascha.rb

 

[nggalai]

Hi andypski,

Correction: it's not different smoke from 330 to 340, but with 340, two different runs. i.e. one run 340, take screenshot, do another run, take screenshot -> different smoke.

Sounds like a pseudo-random number reseeding issue. They're probably seeding the generator once and not restoring the seed value between runs to ensure identical output.

The ATI user in question replied--no matter whether he re-runs GT2 without quitting 3DMark03, after quitting 3DMark03, or even a complete reboot, he never got two identical pictures, smoke-wise. Must be something other, then.

Wavey, can you provide some comparison input on that one? What does refrast say?

93,
-Sascha.rb

 

[chavvdarrr]

http://www.theinquirer.net/?article=12657

The unified compiler a collection of techniques that are not specific to any particular application but expose the full power of GeForce FX. These techniques are applied with a fingerprinting mechanism which evaluates shaders and, in some cases substitutes hand tuned shaders

 

[madshi]

http://www.theinquirer.net/?article=12657

The unified compiler a collection of techniques that are not specific to any particular application but expose the full power of GeForce FX. These techniques are applied with a fingerprinting mechanism which evaluates shaders and, in some cases substitutes hand tuned shaders

How can a hand tuned shader be not specific to a particular application?

 

[Deathlike2]

The unified compiler a collection of techniques that are not specific to any particular application but expose the full power of GeForce FX. These techniques are applied with a fingerprinting mechanism which evaluates shaders and, in some cases substitutes hand tuned shaders

In other words, specific application optimizations.

The unified compiler seems to be a whole different beast that originally thought to be...

The only oddity in the new scores.. and the old.. is that possibly the compiler IS being improved (PS theortical max performance does not change) and VS shaders have been "optimized" as well (as some of the tests involving VS have lost some score)...

Hm...

 

[vb]

A benchmark is worthless to me if overnight the results can change by 15% without proper explanation as to why exactly that happened. Futuremark knows very well what exactly has changed with their benchmark but has not filled in the public that pays attention to their tool. That is inexcusable.

Kyle's editorial

Well, now when Nvidia is making these kind of acusations, he does have a point.

ETA on the next Futuremark .pdf?

can it, please, include the word "cheating"?

 

[CorwinB]

I just love that part...

What we expect will happen is that we'll be forced to expend more engineering effort to update our compiler's fingerprinter to be more intelligent, specifically to make it intelligent in its ability to optimize code even when application developers are trying to specifically defeat compilation and optimal code generation.

Translation :
"Ok, we are cheating and got caught, but we will not amend our ways but rather try to cheat in a more clever way in the future".

Oh, and [K] doesn't have a point with this newest round of FM bashing, but it would take maturity to admit he was used by Nvidia in the initial round of 3DMark-FUD... So now he has resorted to bashing both NV and FM. I wonder when people like [K] will come out and start admitting that so far results of the first run of 3DMark 2003 (ie before the cheating spree) have been generally corroborated by all apps, including games, with regard to the NV3x shaders... Meaning that unless there is cheating at work, 3DM is a valuable tool. FM lost a lot of credibility with their backpedaling on the "C word" issue, but it seems they are at least back on track with the recent initiatives (guidelines, new patch...).

 

[Marc]

http://www.theinquirer.net/?article=12657

The unified compiler a collection of techniques that are not specific to any particular application but expose the full power of GeForce FX. These techniques are applied with a fingerprinting mechanism which evaluates shaders and, in some cases substitutes hand tuned shaders

How can a hand tuned shader be not specific to a particular application?

Because it's not specific to a particular application but to a particular shader ... note the difference

 

[CorwinB]

Because it's not specific to a particular application but to a particular shader ... note the difference

Stop giving them ideas, please...

"Please note that this shader-based optimization is consistent with our guidelines, since it would benefit any application (including games) using the exact same shader."

 

[Joe DeFuria]

What we expect will happen is that we'll be forced to expend more engineering effort to update our compiler's fingerprinter to be more intelligent, specifically to make it intelligent in its ability to optimize code even when application developers are trying to specifically defeat compilation and optimal code generation.

The whole point, Kyle / nVidia, is that if FX's current compiler and/or hardware is SO TWITCHY / SENSITIVE to FM's code changes, such that performance is so drastically altered....

Then your hardware / comiler is simply not particularly robust.

It amazes me how a product SHORTCOMING (FX "twitchiness") can be pushed by nVidia....and even accepted by people like Kyle to be some positive thing!

And it's further amazing to me that here we have a situation where Kyle is complaining about "inconsitency" of 3DMark....when that is exactly the point wrt to the FX. The FX performance is highly inconsitent!

There is nothing at all intelligent about a compiler that CAN optimize for patch 330 and CAN'T optimize for 340. That's called a "compiler that only works well when it knows ahead of time what's coming." In other words, a dumb-ass compiler.

ATI, on the other hand, either has an extremely smart compiler that actually does its job, or it has hardware that just doesn't "need" a "smart compiler" in the first place. (It's probably a bit a both.)

 

[Rookie]

Let hear what Nvidia said on it,Quote:

"An official from NVIDIA Corporation confirmed Mr. Tismer’s accusation that “patch 340 disables the GPU compiler. The compiler has to run on the CPU instead resulting in code harder to digest and taking away 20% of the performance.†“Yes, that is actually the case with the new patch 340 that Futuremark posted,†said an NVIDIA spokesperson on Wednesday.

“Few weeks ago we released our 52.16 driver that includes our brand new unified compiler technology. With the new patch the benchmark, our unified compiler gets not used by the app so it goes to CPU and we are definitely slower,†Luciano Alibrandi, NVIDIA’s European Product PR Manager, added.

“Our position is that our unified compiler delivers the best gaming experience possible, and as long as we produce the right image and simply do not accelerate just a benchmark then it is good to optimize and use a compiler,†the official from NVIDIA acknowledged.

Link:Xbitlabs

I remember Luciano Alibrandi is EX3dfx employee?

 

[nggalai]

Because it's not specific to a particular application but to a particular shader ... note the difference

Stop giving them ideas, please...

"Please note that this shader-based optimization is consistent with our guidelines, since it would benefit any application (including games) using the exact same shader."

Have a look at that NV document called "Unified Compiler Technology" (IIRC) and search for "intrinsic code". And weep.

93,
-Sascha.rb

 

[CorwinB]

I remember Luciano Alibrandi is EX3dfx employee?

Yep, he was technical PR manager for 3dfx France, and someone quite enjoyable to talk to. I really feel sorry for him for having to spew that much techno-babble nowadays.

 

[Neeyik]

Some things about of all this malarky are puzzling me. X-bits said this:

An official from NVIDIA Corporation confirmed Mr. Tismer’s accusation that “patch 340 disables the GPU compiler. The compiler has to run on the CPU instead resulting in code harder to digest and taking away 20% of the performance.â€￾ “Yes, that is actually the case with the new patch 340 that Futuremark posted,â€￾ said an NVIDIA spokesperson on Wednesday.

“Few weeks ago we released our 52.16 driver that includes our brand new unified compiler technology. With the new patch the benchmark, our unified compiler gets not used by the app so it goes to CPU and we are definitely slower,â€￾ Luciano Alibrandi, NVIDIA’s European Product PR Manager, added.

NVIDIA's compiler document only gives examples of PS2.0 code, and the way I read the whole thing, it seems to say that the compiler only works on PS2.0+ shaders, or at the very least, just DX9-level (ie. no VS1.x or PS1.x). Can anyone confirm that this is the case? Considering the relative maximum length/complexity of 2.0+ to 1.x shaders, I would have thought the latter wouldn't have that much to gain from reordering.

Another thing - the GPU compiler. What NV3x products have this "GPU compiler"? The document suggests to me that it's the drivers that are handling the compiling, which is obviously going to be done via the CPU. If I'm wrong here, and the GPU actually does the compiling, then I ask again - which products have it? It seems that all NV3x products take performance drops with the new patch; if that's the case and Gainward/NVIDIA are telling the truth, then they're saying that all NV3x products have this GPU compiler. So if that's true then why wasn't this feature announced when the GeForce FX was launched last year?

Oooh, this is better than a soap opera 8)

 

[Rolf N]

In case this hasn't been posted here already, Damien of TT hardware compared shaders in GT4.

edit:
Yeah, it originated here. I'm an idiot. Throw cake at me. I prefer chocolate
/edit
____________
AFAIK, Futuremark has not modified instructions order. They have made register name inversion (ie r0 -> r1 and r1 -> r0) and terms inversion (ie r0 + r1 -> r1 + r0).

Here's an exemple of a shader used in Mother Nature :

#v330
ps_2_0
dcl t0
dcl t1
dcl t2
dcl t3
dcl t4
dcl_2d s0
dcl_2d s1
dcl_2d s2
dcl_2d s3
dcl_cube s4
dcl_2d s5
texld r0 , t0 , s3
texld r1 , t3 , s3
texld r2 , t0 , s0
mad r2 , r2 , c1.xxxx , c1.yyyy
rcp r3.w , t1.wwww
mul r3 , r3.wwww , t1
mad r3 , r2 , c0 , r3
mad r0 , r0 , c1.xxxx , c1.yyyy
mad r1 , r1 , c1.xxxx , c1.yyyy
add r0 , r0 , r1
mul r0 , r0 , c2.xxxx
dp3 r1.x , r0 , t2
add r2.x , r1.xxxx , r1.xxxx
mad r0 , r2.xxxx , r0 , -t2
add r4.x , c1.wwww , -r1
pow r4.x , r4.xxxx , c1.zzzz
texld r0 , r0 , s4
texld r1 , r3 , s1
texld r2 , r3 , s2
texld r3 , t4 , s5
mul r4.x , r4.xxxx , r3.wwww
add r1 , r0 , r1
lrp r0 , r4.xxxx , r1 , r2
mul r0 , r0 , r3
mov oC0 , r0


#v340
ps_2_0
dcl t0
dcl t1
dcl t2
dcl t3
dcl t4
dcl_2d s0
dcl_2d s1
dcl_2d s2
dcl_2d s3
dcl_cube s4
dcl_2d s5
texld r0 , t0 , s3
texld r4 , t3 , s3
texld r3 , t0 , s0
mad r3 , c1.xxxx , r3 , c1.yyyy
rcp r2.w , t1.wwww
mul r2 , t1 , r2.wwww
mad r2 , c0 , r3 , r2
mad r0 , c1.xxxx , r0 , c1.yyyy
mad r4 , c1.xxxx , r4 , c1.yyyy
add r0 , r4 , r0
mul r0 , c2.xxxx , r0
dp3 r4.x , t2 , r0
add r3.x , r4.xxxx , r4.xxxx
mad r0 , r0 , r3.xxxx , -t2
add r1.x , r4 , -c1.wwww
pow r1.x , r1.xxxx , c1.zzzz
texld r0 , r0 , s4
texld r4 , r2 , s1
texld r3 , r2 , s2
texld r2 , t4 , s5
mul r1.x , r2.wwww , r1.xxxx
add r0 , r4 , r0
lrp r0 , r1.xxxx , r0 , r3
mul r0 , r2 , r0
mov oC0 , r0

_______________
I've plucked this from another forum where nggalai posted it. Blame him
Assuming this is accurate (extracted w 3Danalyze, I guess), the changes are absolutely trivial. Only register numbers have been swapped.
Any piece of software that deserves to be called a compiler would just proceed as usual, it wouldn't matter at all.

The real disturbing thing here is that NVIDIA's putting the name "compiler" onto something that, in reality, is the same old static application specific shader replacement.

This, ladies and gentlemen, is not a compiler.

 

[nggalai]

Hi zeckensack,

I've plucked this from another forum where nggalai posted it. Blame him

Guess where I got it from--right, this thread here. Sorry, should have added a link.

The real disturbing thing here is that NVIDIA's putting the name "compiler" onto something that, in reality, is the same old static application specific shader replacement.

This, ladies and gentlemen, is not a compiler.

Agreed. I wouldn't be surprised if, in the end, it all came down to petty semantics ...

93,
-Sascha.rb

 

[Rolf N]

Hi zeckensack,

I've plucked this from another forum where nggalai posted it. Blame him

Guess where I got it from--right, this thread here. Sorry, should have added a link.

*bites self*
Okay, I admit I didn't bother reading more than the last three pages. Maybe it was worth repeating, seeing how the thread developed. Hmm, let's see, where's that edit button ...

The real disturbing thing here is that NVIDIA's putting the name "compiler" onto something that, in reality, is the same old static application specific shader replacement.

This, ladies and gentlemen, is not a compiler.

Agreed. I wouldn't be surprised if, in the end, it all came down to petty semantics ...

93,
-Sascha.rb

Too late. The damage has already been done. Those twisted f**ks :?

 

[cho]

I've plucked this from another forum where nggalai posted it. Blame him
Assuming this is accurate (extracted w 3Danalyze, I guess), the changes are absolutely trivial. Only register numbers have been swapped.
Any piece of software that deserves to be called a compiler would just proceed as usual, it wouldn't matter at all.

The real disturbing thing here is that NVIDIA's putting the name "compiler" onto something that, in reality, is the same old static application specific shader replacement.

This, ladies and gentlemen, is not a compiler.

and i am interested in how did FM changed the VS ?

 

[Ostsol]

An official from NVIDIA Corporation confirmed Mr. Tismer?s accusation that ?patch 340 disables the GPU compiler. The compiler has to run on the CPU instead resulting in code harder to digest and taking away 20% of the performance.? ?Yes, that is actually the case with the new patch 340 that Futuremark posted,? said an NVIDIA spokesperson on Wednesday.

?Few weeks ago we released our 52.16 driver that includes our brand new unified compiler technology. With the new patch the benchmark, our unified compiler gets not used by the app so it goes to CPU and we are definitely slower,? Luciano Alibrandi, NVIDIA?s European Product PR Manager, added.

Um. . . ok. . . Shader code is tossed to the video card through the driver. From what I've learned during my time at this forum, the GPU itself cannot interpret shader code by itself. It requires the shader code to be compiled by the driver into a form usable by the GPU, just like assembler has to be compiled into a binary format usable by the CPU. As such, compilation occurs via CPU power anyway. However, assuming compilation takes place on the GPU, how would this be done? Hard code the compiler in silicon? No. . . I don't think anyone would be dumb enough to do that. . . For one it's a waste of transistors and second it makes the compiler impossible to tweak during the product's lifetime. The compiler is definitely in the drivers and powered by the CPU.

Another problem: shader code is not compiled during the running of the scene! It is compiled as soon as the driver receives it and anyone who uses shaders tosses them to the drivers long before the first frame of the scene is rendered. As such, if by some mysterious chance Futuremark had done something to slow down the compiler, there would be no affect of framerate. If anything, the initialization of the scene would take a few milliseconds longer -- and the speed of initialization is certainly not taken into account when determining the average frame rate of the scene.