Test a DX8.1 1.1 Vertex Shader?

[flipper]

hi all. first time poster, sometimes lurker here.

a co-worker and i are writing a DX8.1 hobbyist "game" that features "life-like" fish movement. we posted a request on Gamedev.net for some help in testing one of the app's shaders on various hardware so we could see what mods, if any, we would need to make. the guys at Gamedev gave us a good set of numbers but unfortunately many didn't post their "shader off" numbers, which is when the app manipulates the vertices itself and constantly streams a new set to the video card for every frame.

so we thought maybe a somewhat benchmark-oriented site might be interested in helping us out? if you are, the ~400KB app's available for download here (http://www.geocities.com/flipper76108/). there's been a couple reports of failure and a known issue with the bounding box display that causes the app to crash with some drivers, but that's all the bad news that's been reported. what we're looking for are the number of fish it takes to just drop below 30fps, with the shader on and then again with the shader off. INS adds fish, cap A toggles the shader mode. cap H displays keys help.

thanks.

oh, and if you're interested, here's our current set of numbers, shader off sorted first then shader on sorted:

CPU/CARD Shader Off Shader On
2.4ghz/9700pro: 53 175
unknown: 46 134
2.1ghz/geforce4ti4600: 20 77
1.7ghz/geforce4mx440: 17 27
1.0ghz/geforce2go: 7 10
2.4ghz/9500: ? 42
2.0ghz/geforce4ti4400: ? 67
2.2ghz/8500: ? 54

CPU/CARD Shader Off Shader On
2.4ghz/9700pro: 53 175
unknown: 46 134
2.1ghz/geforce4ti4600: 20 77
2.0ghz/geforce4ti4400: ? 67
2.2ghz/8500: ? 54
2.4ghz/9500: ? 42
1.7ghz/geforce4mx440: 17 27
1.0ghz/geforce2go: 7 10

[Ichneumon]

hehe, probably not terribly interesting to add to the list, but i'm waiting for a phone call at work...

Numbers for Win2k, ATI Rage Mobility (M3), Mobile P3-850, 800x600
Shader Off:
3

Shader On:
3

:)

[Thowllly]

CPU: Athlon (Tbird) 900Mhz
MB: Abit KT7raid(kt133), 1GB 133mhz cl3 sdram, via 4in1 4.46
gfx: Geforce2 GTS 64MB, Detonator 40.71, DX9
OS: Windows XP pro, no service pack.

Screen resolution: 1400x1052x32@85Hz, 800x600 window (framerate was not affected by size of window)

# of fish 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Shader off 96 89 48 47 33 32 29 24 23 20 19 18 16 15 14 13 12 12 11 11
Shader on 96 94 49 48 47 40 32 31 28 24 24 22 19 19 17 16 16 15 14 13

The PC had over 7 days of up time when I ran this test, maybe I should have restarted it first... :?

[flipper]

thanks for the numbers guys.

Ichneumon, we think we can reduce the model's vertex/face count significantly w/o too much loss in quality, so it's good to see some numbers on the mobility. these are fairly high poly models compared to what's out there currently in commercial products. the models we're using have about 15 to 20 times more vertices/faces in them.

Thowllly, thanks for the breakdown by fish. seeing that halving of the fps for the 3rd fish just might help us write a better vb sizing algo.

[Ichneumon]

thanks for the numbers guys.

Ichneumon, we think we can reduce the model's vertex/face count significantly w/o too much loss in quality, so it's good to see some numbers on the mobility. these are fairly high poly models compared to what's out there currently in commercial products. the models we're using have about 15 to 20 times more vertices/faces in them.

Thowllly, thanks for the breakdown by fish. seeing that halving of the fps for the 3rd fish just might help us write a better vb sizing algo.

I can corroborate the extra drop with the 3rd fish. There was almost no drop in frame rate from one fish to two (heh, like 33fps to 32 fps)... but adding the 3rd fish dropped it by about 8 to around 24, and then in would come back up a couple in a few moments. then going to 4 fish only dropped it 1 or 2 fps again.

[BRiT]

System: AMD XP+ 2.2Ghz actual [11*200], Epox 8RDA+, 1Gig PC3500-C2 (4-2-2-2) in Dual-Channel, WinXP+SP1, 1600x1200x32bit@75Hz desktop, 8X AGP, FastWrites On
Video: Radeon 9700Pro (324x310.5 default clock)
ATI Control Panel: VSync Off, MipMap MAX, Texture MAX, 8X-AF Qual, 6x AA, TrueForm OFF, 75Hz DX Override

800x600x32 Window

Shader On (Major Transition Points):
1 = 64 fps
78 = 64 fps
79 = 63.02/62.03 fps [355421/552684]
80 = 62.03 fps [359920/559680]
81 = 61.05 fps [364419/566676]
83 = 60.06 fps [373417/580668]
171 = 29.61/30.06 fps [769329/1196316]
173 = 29.54 fps [778327/1210308]

Shader Off
1 = 64 fps
30 = 64 fps [134970/209880]
32 = 48/54 fps
57 = 32 fps [256443/398772]
59 = 28.55/31.03 fps [265441/412764]
62 = 21.33 fps [278938/433752]


I'll try running with vsync enabled later if these numbers seem wonky to anyone. Let me know if you'd like me to keep adding more fish or try overclocking the card some or the cpu (some more).

[BRiT]

Tried various modes, with application AA and AF settings - 800x600x32 windows - 1600x1200x32 full screen ... the numbers were pretty identical to 6xAA Q8xAF (given the randomness of the app).

[chavvdarrr]

TB2100+, GB VAX1394 (KT400), 512MB DDR@266MHz, Win2K SP3
R9000Pro, 128MB 275/250, Cat 3.2, DX9.0a, 4x AGP, FW off, Vsync off
1152x876x95Hz
texture&mipmap - Quality
Aniso - Application, Smoothvision - off

800X600X32 (d16)
number shader ON shader OFF
1 330 255
2 250 225
3 200 200
4 165 165
5 140 140
6 120 125
7 110 115
8 95 100
9 88 90
10 80 83
25 27-33
30 29-30

[Hyp-X]

On startup:

Generic application error. Enable debug output for detailed information.

Asus A7V266-E
XP 1800+
R9700Pro w/ Catalyst 3.2
Win2k + SP3
DX9.0 debug runtime

[flipper]

Ichneumon, thanks man. interesting pattern that, huh? we're going to re-run on the geforce2go we have here to see if the problem shows up with it as well.

BRiT, thanks for the thorough research. i should have passed along some info re: the setup. we tried to make the test so only the shader was a factor, i.e., it was independent of as many other things as possible. your testing sure helps us feel better that maybe we did okay.

hey chavvdarrr, that's another new card for us. thanks! those numbers look really interesting. notice how they're almost the same after the 3rd fish? what in the world is up with that 3rd fish!!!! guess we'll have to dig deeper than we thought.

thanks again all.

[flipper]

hi Hyp-X. the only thing i know of to cause that error would be if the fish model was not in the fish sub-directory? sorry if that's the case. i know my explanation of what to do was not very thorough.

if that's not the problem then i'm sort of at a loss since your setup is very similar to one we're using. and we used the DX 8.1 common framework for init to try and make sure we mucked up as little as possible.

[flipper]

hey BRiT. i just had a thought. those numbers start out as if the "sleep" mode was set. would you mind re-running with it off? it's the cap L key. no problem if not since the 30 fps numbers are the ones we're most interested in.

[Hyp-X]

hi Hyp-X. the only thing i know of to cause that error would be if the fish model was not in the fish sub-directory? sorry if that's the case. i know my explanation of what to do was not very thorough.

if that's not the problem then i'm sort of at a loss since your setup is very similar to one we're using. and we used the DX 8.1 common framework for init to try and make sure we mucked up as little as possible.

It works when I disable the debug version of Direct3D from the control panel.

[Thowllly]

I also had the sleep mode set. Here are the numbers with it off:
# of fish 1 2 3 4 6 8 10 12 16 20
Shader off 212 107 74 56 38 28 23 19 14 11
Shader on 239 128 90 69 48 36 29 24 18 14

[Ichneumon]

Heh... it appears I had sleep mode On as well.
whups.

Performance is much different with Sleep mode off on this system. (the mP3-850/ATI mobility m3). With shader off, the FPS fluctates by as much as 10fps (35-45) depending on the fish movement, where the performance is pretty steady with shader On. The numbers are rough averages of any fps fluctations...


# of fish 1 2 3 4
Shader off 40 35 28 23
Shader on 46 38 31 26


Seems that incremental drop in fps after 2nd fish is gone with sleep off... though on this system the fluctiations are significant as individual fish movement has a lot more effect with a couple fish rather than with a couple dozen...

[Crusher]

CPU: Celeron 566@850 (100 MHz bus)
MB: Abit BX6r2, 452MB PC133
gfx: Geforce2 GTS 32MB, Detonator 43.45, DX9
OS: Windows 2000 pro, SP2.

Screen resolution: 1280x1024x32 full screen, sleep mode off

# of fish 1 2 3 4 5 6 7
Shader off 116 63 43 31 26 21 19
Shader on 48 23 16 12 10 8 7

[BRiT]

System: AMD XP+ 2.2Ghz actual [11*200], Epox 8RDA+, 1Gig PC3500-C2 (4-2-2-2) in Dual-Channel, WinXP+SP1, 1600x1200x32bit@75Hz desktop, 8X AGP, FastWrites On
Video: Radeon 9700Pro (324x310.5 default clock)
ATI Control Panel: VSync Off, MipMap MAX, Texture MAX, 8X-AF Qual, 6x AA, TrueForm OFF, 75Hz DX Override

800x600x32 Window, Sleep Mode Off

Listings broken up per 10 fish (1-10, 11-20, 21-27). Towards the end there, individual fish adding had very slight effect on fps.


On : 1210, 964, 805, 705, 612, 550, 496, 451, 416, 384
Off: 1098, 670, 510, 400, 336, 296, 250, 222, 200, 187

On : 359, 335, 313, 295, 279, 265, 253, 240, 228, 219
Off: 172, 159, 143, 137, 126, 121, 112, 105, 100, 96

On : 210, 202, 194, 187, 180, 175, 168
Off: 91, 87, 84, 82, 77, 75, 73.85

[Bambers]

athlonxp 2000+, 64mb 8500 (overclocked a little), 512mb pc2700, 1152x864 full screen.


Fish on off
1 704 504
2 501 355
3 388 284
4 317 239
5 268 205
10 159 119
15 105 84
20 80 64
50 33.7 26.4
100 17.9 11.98

[flipper]

Hyp-X, thanks for the heads-up regarding the DX debug version. we have been "flying blind" here since we installed DX9 over DX8 and haven't bothered to install the DX9 SDK.

and thanks to everyone who re-ran with sleep mode off. and to Bambers and Crusher for the new set of numbers.

and i just want to say thanks again for all of this data. you've given us much more detail than we hoped to get. and after making some plots, which we did on a per fish basis thanks to you guys, we now have a pretty good idea where to concentrate our efforts.

although i have to admit that Crusher's numbers are confusing me. :)

[Crusher]

I get the same results every time I run it, with nothing else running in the background. Actually the "shader off" numbers are probably lower than they shoudl be, since you're using software vertex processing, instead of re-initializing the device for hardware T&L when you toggle the shader off. IIRC vertex shader 1.1 on the GTS are actually done by the drivers in software on the CPU, so "shader on" mode should be pretty slow on this system.

What what confuses me is that Thowllly's Athlon can do so much better when it's only running 50 MHz faster and on a 33 MHz faster bus... and also the fact that he somehow was getting 200+ FPS with sleep mode off... which I can't even come close to. I turned off the 2x anisotropic filtering and set the mip-map selection to best performance in the drivers, but it didn't make any noticable difference. I don't have FSAA on, and that would be the only other thing I could think of that would slow it down.

[flipper]

Crusher, you're absolutely correct about the test not resetting to hardware T&L when switching to non-shader use. we figured this would be the worst case scenario for those boards that had no on-board shader support, and this seemed like a good thing sense we were looking for high and low numbers.

but even then, all of the other numbers indicate that the driver/DX combo does better at manipulating the vertices than the application does, even on the GeForce2Go which also runs the shader in software mode.

what your numbers say to me is either you have a bad driver/DX combo or you have some kicka$$ cpu throughput that no one else has, or more likely, that we have something wrong in the code and that your setup is the only one that's catching the problem.

could it be that we have run into code in the version of the Detonator's that you're using that has not been optimized for a real game? in other words, we're doing something the nVidia folks assumed would not be done? kind of like with the new Futuremark demo? who knows.

[Crusher]

Well, the first test in 3DMark03 was slow as crap when I ran it with the older 41.09 drivers... and the vertex shader test of 3DMark01 wasn't any too fast either. It might be that NVIDIA optimized their software VS path for AMD and P3+ processors, and the first generation Celeron just can't handle them very well.

My driver history was DX8.1b -> install 41.09 -> install DX9.0a -> install 30.82 -> install 43.45.

The 30.82s were installed to see if it would fix some weird problems I've seen in newer games (pink screen when changing levels in Morrowind and weird colors when low on video memory in the IGI2 demo, both of which existed before updating to DX9). I uninstalled the video card every time I switched drivers. The only real driver issue I ever found for certain was that after installing DX9, all of my point sprites written in DX8 were incorrectly aligned. These latest 43.45 drivers have fixed that.

I suppose there *could* be an issue with having the DX8 SDK installed and using the DX9 runtime, but I wouldn't think that would cause problems (I haven't reinstalled the DX8 runtime through the SDK or anything since upgrading to DX9). DXDIAG doesn't show any issues.

I get 195 FPS in the DX8.1 SDK vertex shader code example at 1280x1024x32, and about 254 FPS in the vertex blend example at that resolution. If you want I can e-mail you with detailed system info and such.

[flipper]

If you want I can e-mail you with detailed system info and such.
thanks, but that's not necessary for now. it'd be great if you'd be willing to re-test after we come up with a new version of the code though. it should be something that's more "true-to-form" in non-shader mode.

[BenSkywalker]

Athlon XP @2192MHZ 163(326)MHZ FSB, 512MB PC2700 GF4 Ti4200 @290/560 Win2K SP3 45.31Dets Windowed mode
Shader Off

1- 760
10- 108
15- 74
20- 55
25- 45
30- 38
35- 33
40- 29
170- 7

Shader On-

1- 685
10- 190
20- 105
30- 73
40- 55
50- 45
60- 38
70- 34
80- 29
170- 14

With shaders on the single fist test is ~200FPS faster in fullscreen mode(883), although it comes in line when you add more fish with the windowed scores.

[flipper]

great numbers BenSkywalker. thanks for adding another card to our list.

[Dave Baumann]

Somthing a little newer: GFFX 5600U Reference:

Shader on:

1 - 64
27 - 57
31 - 49
35 - 44
40 - 39
45 - 34
50 - 31
52 - 30
53 - 29
55 - 28
60 - 26


Shader Off

1 - 64
16 - 38
17 - 32
18 - 32
31 - 32
32 - 22

Same rig as the 5600U Preview (http://www.beyond3d.com/previews/nvidia/nv31/)

[Dave Baumann]

NV30 Ultra

Shader on:

1 - 64
100 - 64
110 - 60
120 - 55
130 - 51
150 - 45
160 - 42
170 - 39
180 - 37
190 - 35
200 - 33
210 - 32
220 - 30
230 - 29

Shader off
1 - 64
20 - 64
21 - 63
30 - 32
44 - 32
45 - 31
47 - 23
50 - 21

[ector]

Athlon 1800+XP 512mb RAM Radeon9500Pro
1280x1024 32-bit fullscreen

Shader on: dips below 30FPS at 140 fish
Shader off: dips below 30FPS at 40 fish

With 1 single fish I get >600 fps with sleep mode off.

[flipper]

those 9500pro numbers look great ector; thanks.

Mr. Baumann, what can i say? did that test on the ultra ruin your Insert key? :) thanks for both sets of numbers.

[Dave Baumann]

I was more worried about bone compression in my finger - could have had an RSI suit on ya there! ;)

I'll try and run some numbers on 9600 PRO and 9800 PRO in a while.

[flipper]

I was more worried about bone compression in my finger - could have had an RSI suit on ya there! ;)

I'll try and run some numbers on 9600 PRO and 9800 PRO in a while.
:lol: don't blame us for your improper wrist alignment problems. :lol:

9600pro and 9800pro numbers would be great!

[Dave Baumann]

9800PRO

1 - 64
119 - 54
125 - 51.2
155 - 41.7
165 - 38.7
170 - 38
180 - 35.88
190 - 34
200 - 32.5
210 - 31.3
215 - 30.5
220 - 29.54

[Gubbi]

Athlon XP 10x200MHz + R9700:

Shader on: 176
Shader off: 53

Fps drops to 27.something when going from 53 to 54 fish with shaders OFF.

Cheers
Gubbi

[flipper]

thanks for the new numbers guys.

i'm afraid the project has ballooned into a full blown game development effort now so a new version of the shader probably won't be available for testing anytime soon. but we definitely appreciate all of the useful data.