shadermark 2.0 tomorrow!

[tEd]

shadermark 2.0 tomorrow

http://www.shadermark.de/start.html

looks good

 

[vrecan]

ANTI-DETECT-MODE fights against shader and texture specific optimisations <--- is the part that is the most interesting to me. Cant wait to see what will come of this.

 

[Arun]

Hmm, when you think about it, a basic anti-detect system that would work really nice ( against current techniques ) is an automatic system putting NOP instructions in the shader program randomly.
If you want to become more evil, try to do rescheduling that should theorically not impact performance on any known card ( but that becomes a potential problem for future cards, and is a mess to program... )


Uttar

 

[Simon F]

Hmm, when you think about it, a basic anti-detect system that would work really nice ( against current techniques ) is an automatic system putting NOP instructions in the shader program randomly.

I presume you don't mean the "NOP" opcode as I imagine that would be the first thing stripped out of the code by the runtime/driver/assembler/whatever.

Perhaps something functionally equivalent to Y=X*1.0 or Y=X+0.0 might work as a padding NOP.

 

[madshi]

Perhaps something functionally equivalent to Y=X*1.0 or Y=X+0.0 might work as a padding NOP.

Well, a clever optimizer might kill such an instruction, too.

What I could imagine is something like the following. The same shader could be either this:

X=75
Y=X*7.5

Or you could write it as:

Y=75
X=Y*7.5

If a benchmarks can change such things on the fly (optimal would be several variations), a cheater might get into problems.

 

[Nick[FM]]

It seems that it is now released. The same link as in the first post gets you to the list of mirrors. Downloading it now..

 

[Dio]

Well, a clever optimizer might kill such an instruction, too.

But isn't that the point?

The changes must be sufficient to disable 'cheating', but not so obtuse that only the most ridiculously aggressive optimising compiler can find every possible variation and get the same performance out...

If you make algorithmic changes, then performance may and even probably will change, because you may manage to make the compiler generate different code out the back.

 

[Miksu]

Looks promising. I tried the benchmark-mode with my R9700Pro and got these results:

shader  1 (                                           Pixel Shader Precision Test): pixel shader 1.1 full precision: s16e7
          (                                           Pixel Shader Precision Test): pixel shader 1.4 full precision: s16e7
          (                                           Pixel Shader Precision Test): pixel shader 2.0 full precision: s16e7
          (                                           Pixel Shader Precision Test): pixel shader 2.0 partial precision: s16e7

shader  2 (                                            Per Pixel Diffuse Lighting):	 199 fps	  5.0201 mspf	     996 rendered frames
shader  3 (                            Per Pixel Directional Light Shader (Phong)):	 137 fps	  7.2993 mspf	     685 rendered frames
shader  4 (                                  Per Pixel Point Light Shader (Phong)):	 140 fps	  7.1531 mspf	     699 rendered frames
shader  5 (                                   Per Pixel Spot Light Shader (Phong)):	 112 fps	  8.9606 mspf	     558 rendered frames
shader  6 (                                        Per Pixel Anisotropic Lighting):	 146 fps	  6.8587 mspf	     729 rendered frames
shader  7 (                                         Per Pixel Fresnel Reflections):	 126 fps	  7.9114 mspf	     632 rendered frames
shader  8 (                             Per Pixel BRDF-Phong/Anisotropic Lighting):	 109 fps	  9.1575 mspf	     546 rendered frames
shader  9 (                                          Per Pixel Car Surface Shader):	  99 fps	 10.1215 mspf	     494 rendered frames
shader 10 (                                         Per Pixel Environment Mapping):	 204 fps	  4.9068 mspf	    1019 rendered frames
shader 11 (                                    Per Pixel Environment Bump Mapping):	 177 fps	  5.6561 mspf	     884 rendered frames
shader 12 (                                                Per Pixel Bump Mapping):	 120 fps	  8.3612 mspf	     598 rendered frames
shader 13 (                                       Per Pixel Shadowed Bump Mapping):	  76 fps	 13.1234 mspf	     381 rendered frames
shader 14 (                                        Per Pixel Veined Marble Shader):	  76 fps	 13.0752 mspf	     383 rendered frames
shader 15 (                                                 Per Pixel Wood Shader):	 108 fps	  9.2737 mspf	     540 rendered frames
shader 16 (                                                 Per Pixel Tile Shader):	  65 fps	 15.3846 mspf	     325 rendered frames
shader 17 (                                  Fur Shader With Anisotropic Lighting):	  12 fps	 86.6110 mspf	      58 rendered frames
shader 18 (        Per Pixel Refraction and Reflection Shader with Phong Lighting):	  89 fps	 11.2108 mspf	     446 rendered frames
shader 19 (  Dual Depth Shadow Mapping With 3x3 Bilinear Percentage Closer Filter):	  24 fps	 42.2794 mspf	     119 rendered frames
shader 20 (                                High Dynamic Range Shader (cross blur)):	  46 fps	 21.9641 mspf	     228 rendered frames
shader 21 (                             High Dynamic Range Shader (gaussian blur)):	  49 fps	 20.2429 mspf	     247 rendered frames
shader 22 (                          Per Pixel Edge Detection And Hatching Shader):	  31 fps	 32.5690 mspf	     154 rendered frames
shader 23 (                                         Per Pixel Water Colour Shader):	  43 fps	 23.3285 mspf	     215 rendered frames

 

[madshi]

Well, a clever optimizer might kill such an instruction, too.

But isn't that the point?

The changes must be sufficient to disable 'cheating', but not so obtuse that only the most ridiculously aggressive optimising compiler can find every possible variation and get the same performance out...

If I imagine I would be a NVidia programmer trying to write a cheat for ShaderMark 2.0, I'd easily find out that ShaderMark adds just some dummy instructions at random locations of the shader. So before my shader detection I would simply clean up the shader by removing all those dummy instructions. I don't think that would be very difficult to do for a good cheater.

If you make algorithmic changes, then performance may and even probably will change, because you may manage to make the compiler generate different code out the back.

True.

Perhaps a benchmark could use the same shader, copy it 1000x and make algorythmic changes to it (which don't change the mathematical output). Then run them all one after the other. Just the sheer amount of shader variations might stop cheaters. Would that work?

 

[Dio]

Well, as long as you don't expect them all to perform the same... at which point I'm not sure what the use of it is...

 

[madshi]

Well, as long as you don't expect them all to perform the same... at which point I'm not sure what the use of it is...

I meant that each of the 1000 shaders should be (slightly) different to each other - but the whole collection of 1000 shaders should be (fully) identical in each full benchmark run. So each graphics card would have to execute exactly the very same 1000 shaders. As a result the benchmark results should be comparable.

Well, it was just a thought...

 

[Joe DeFuria]

Can someone post results with a GeforceFX variant card? (Both with anti-cheat mode on and off)?

 

[tEd]

5900 Ultra					
PS 2.0 - Partial Precision OFF	51,75	52,1			
shader  2 (                                            Per Pixel Diffuse Lighting):	101	133			
shader  3 (                            Per Pixel Directional Light Shader (Phong)):	70	85			
					
shader  5 (                                   Per Pixel Spot Light Shader (Phong)):	57	73			
shader  6 (                                        Per Pixel Anisotropic Lighting):	71	82			
shader  7 (                                         Per Pixel Fresnel Reflections):	118	143			
					
shader  9 (                                          Per Pixel Car Surface Shader):	27	39			
shader 10 (                                         Per Pixel Environment Mapping):	130	163			
shader 11 (                                    Per Pixel Environment Bump Mapping):	113	132			
shader 12 (                                                Per Pixel Bump Mapping):	49	69			
shader 13 (                                       Per Pixel Shadowed Bump Mapping):	34	40			
shader 14 (                                        Per Pixel Veined Marble Shader):	38	53			
shader 15 (                                                 Per Pixel Wood Shader):	52	52			
shader 16 (                                                 Per Pixel Tile Shader):		40			
shader 17 (                                  Fur Shader With Anisotropic Lighting):	5	6			
shader 18 (        Per Pixel Refraction and Reflection Shader with Phong Lighting):	21	30			

PS 2.0 - Partial Precision ON	51,75				
shader  2 (                                            Per Pixel Diffuse Lighting):	116	155			
shader  3 (                            Per Pixel Directional Light Shader (Phong)):	93	110			
					
shader  5 (                                   Per Pixel Spot Light Shader (Phong)):	69	89			
shader  6 (                                        Per Pixel Anisotropic Lighting):	92	123			
shader  7 (                                         Per Pixel Fresnel Reflections):	121	152			
					
shader  9 (                                          Per Pixel Car Surface Shader):	51	60			
shader 10 (                                         Per Pixel Environment Mapping):	132	163			
shader 11 (                                    Per Pixel Environment Bump Mapping):	123	141			
shader 12 (                                                Per Pixel Bump Mapping):	70	89			
shader 13 (                                       Per Pixel Shadowed Bump Mapping):	52	67			
shader 14 (                                        Per Pixel Veined Marble Shader):	51	85			
shader 15 (                                                 Per Pixel Wood Shader):	74	85			
shader 16 (                                                 Per Pixel Tile Shader):		58			
shader 17 (                                  Fur Shader With Anisotropic Lighting):	7	10			
shader 18 (        Per Pixel Refraction and Reflection Shader with Phong Lighting):	40	56			

PS 2.0a - Partial Precision OFF	51,75				
shader  2 (                                            Per Pixel Diffuse Lighting):	106	133			
shader  3 (                            Per Pixel Directional Light Shader (Phong)):	68	85			
					
shader  5 (                                   Per Pixel Spot Light Shader (Phong)):	49	73			
shader  6 (                                        Per Pixel Anisotropic Lighting):	69	82			
shader  7 (                                         Per Pixel Fresnel Reflections):	118	143			
					
shader  9 (                                          Per Pixel Car Surface Shader):	33	40			
shader 10 (                                         Per Pixel Environment Mapping):	131	163			
shader 11 (                                    Per Pixel Environment Bump Mapping):	113	132			
shader 12 (                                                Per Pixel Bump Mapping):	50	69			
shader 13 (                                       Per Pixel Shadowed Bump Mapping):	30	40			
shader 14 (                                        Per Pixel Veined Marble Shader):	41	53			
shader 15 (                                                 Per Pixel Wood Shader):	51	52			
shader 16 (                                                 Per Pixel Tile Shader):		42			
shader 17 (                                  Fur Shader With Anisotropic Lighting):	4	6			
shader 18 (        Per Pixel Refraction and Reflection Shader with Phong Lighting):	19	32			

PS 2.0a - Partial Precision ON	51,75				
shader  2 (                                            Per Pixel Diffuse Lighting):	122	155			
shader  3 (                            Per Pixel Directional Light Shader (Phong)):	91	110			
					
shader  5 (                                   Per Pixel Spot Light Shader (Phong)):	72	90			
shader  6 (                                        Per Pixel Anisotropic Lighting):	100	123			
shader  7 (                                         Per Pixel Fresnel Reflections):	122	152			
					
shader  9 (                                          Per Pixel Car Surface Shader):	53	61			
shader 10 (                                         Per Pixel Environment Mapping):	132	163			
shader 11 (                                    Per Pixel Environment Bump Mapping):	123	141			
shader 12 (                                                Per Pixel Bump Mapping):	74	89			
shader 13 (                                       Per Pixel Shadowed Bump Mapping):	50	64			
shader 14 (                                        Per Pixel Veined Marble Shader):	66	81			
shader 15 (                                                 Per Pixel Wood Shader):	74	85			
shader 16 (                                                 Per Pixel Tile Shader):		60			
shader 17 (                                  Fur Shader With Anisotropic Lighting):	7	10			
shader 18 (        Per Pixel Refraction and Reflection Shader with Phong Lighting):	43	56

 

[Joe DeFuria]

Thanks, Ted....now if you could just do something about that formatting!

 

[Tokelil]

Why aren't all shaders run on the 5900U? Missing driversupport? (Like MRT)

 

[Joe DeFuria]

And what's the difference between PS 2.0, and PS 2.0a? (Maybe I need to check the documentation...)

 

[tEd]

Why aren't all shaders run on the 5900U? Missing driversupport? (Like MRT)

missing fp texture support

 

[Quitch]

I'm not familiar with the program, what do the last two columns in tEd's output represent?

 

[Joe DeFuria]

Good question...I assume the first number is th FPS number....dunno about the second.

I don't know why ted's results are in what appears to be a different reporting format than Miksu's, for example....

 

[tEd]

And what's the difference between PS 2.0, and PS 2.0a? (Maybe I need to check the documentation...)

ps2.0a is only supported by the gffxs

i think it fits the funcionality of the gffx. On some shader ps2_a needs less instruction compared to ps2_0

check the shadermark pdf for more info