Futuremarks technical response

[Joe DeFuria]

Thats it I would like to put out the challenge ATI vs Nvidia..Ice Hockey..name the place...

Already tried that....nVidia backed out...chicken sh*ts!

http://www.theinquirer.net/?article=5912

 

[Doomtrooper]

Seriously I don't understand the comments towards Canadians sometimes, I look at the US and Canada as partners...we are very much alike...alot of History.
I work for a Huge American based company, and travel to the US alot and in the end we all want the same thing and work towards the same goals.

 

[Tahir2]

How about a game of cricket? Or crocket!! Maybe NVIDIA would be happiest with a Crochet competition.. OK I'll stop now...

 

[Doomtrooper]

Thats it I would like to put out the challenge ATI vs Nvidia..Ice Hockey..name the place...

Already tried that....nVidia backed out...chicken sh*ts!

http://www.theinquirer.net/?article=5912

I missed that..

 

[Sabastian]

NVIDIA have got a real bee in their bonnet about PS1.4. I was talking to a few guys at Dusk-till-dawn about it and they are unhappy that Futuremark have supported a technique that 'only a bunch of Canadians' have. I pointed out that "well, it is a part of DirectX afterall" to which the responce was "well, it shouldn't have been".

The fact is, it is in DX, its a viable rendering option, and games are supporting it (I have a big list...).

Dave, it is good that you give such a candid account of your conversation with Nvidia reps. I very much appreciate your input, but for your sake are you not afraid of causing ... grief with that particular IHV or no?

This actually coninsides with a quote I read out of THG sometime ago, this seems to be a reoccurring theme. I have read other accounts suggesting the same thing. :? It really looks as this statement from Omid Rahmat is more true then I had originally thought.

http://www4.tomshardware.com/smoke/20010601/ati-06.html

I hate to say this, but the kind of aggression, even greed, that drives Silicon Valley companies eludes outsiders, and ATI is an outsider. It's going to be tougher for ATI out of Canada then if they were in the US. I wouldn't have said that 3 years ago, but in the present circumstances, I think that it is going to take an extraordinary effort by ATI..

 

[Silent_One]

DaveBaumann wrote:

The fact is, it is in DX, its a viable rendering option, and games are supporting it (I have a big list...).

Dave, just out of curiosity, what games are on the "big list"?

 

[MrBond]

Seriously I don't understand the comments towards Canadians sometimes, I look at the US and Canada as partners...we are very much alike...alot of History.
I work for a Huge American based company, and travel to the US alot and in the end we all want the same thing and work towards the same goals.

It's all in good fun No one really (well, no one I know anyway) takes these jokes seriously.

 

[Sabastian]

DaveBaumann wrote:

The fact is, it is in DX, its a viable rendering option, and games are supporting it (I have a big list...).

Dave, just out of curiosity, what games are on the "big list"?

Yes, Inquiring minds want to know...

 

[John Reynolds]

The first thing we do let's kill all the lawyers. --II Henry VI, IV:2

I had a shirt from the Folger's library in DC with that quote on it. Used to get me dirty looks in the legal department of Speedway SuperAmerica's corporate office when I worked there years ago. 8)

 

[tb]

My No, was to this part of the message "Game Test 2 and 3 don't change much between CPU vs GPU skinning because primarily they are Pixel Shader limited"

I think they are more vertex shader and fillrate limited than pixel shader calculation speed limited.

Based on what? Pixel shading performance hit may be the same on every card for the functionality exhibited in the test (I don't know how many clock cycles each pixel shader takes on each card, and that is something the 9500 non pro versus 9000 Pro could maybe tell us about how this has improved), but that doesn't mean the hit is not a significant factor in the benchmark.

Based on my results. If the pixel shader is the limiting factor, the fps should have increased much more when disabling it.

Your results from "no txt" show an increase, but your results from "no ps" show more of an increase when you raise the resolution to 1024x768. This supports the idea that the pixel fillrate determines the absolute limit of performance, but pixel shading limitations determine how much of that can be realized.

BTW, does "No txt" mean no textures read, or no pixel output (i.e, "No coloring" to my mind)?

No Txt means there will never be any texture set to dx9's "settexture", so the driver could ignore/optimize the txt reads and no coloring is made.

What your analysis (as I understand things, which of course may be in error) seems to ignore is the idea that just because the 9700 can run pixel shaders fast enough to not significantly limit pixel fillrate...

There is a line at the bottom of there test, where I say something like this: The actual limitation factor (vs, ps, fillrate) depends also much on the graphics hardware and the r9700 pro is taken as a baseline for future dx9 cards, because 3dmakr03 is a dx9 benchmark.

The limitation goes from the vertex shader to the fillrate (and a little bit of pixel shader) when you increase the resolution. Test 1 is most of the time vertex shader limited, but fillrate comes into play with some very high resolutions

But that is the case for any fillrate limited benchmark....it is fillrate limited until you lower the fillrate demands so that the impact is less than that of the other demands. The thing is that the test scales downwards wrt performance from 1024x768 and up, and as far as I can recall that is part of the baseline assumptions of the benchmark...so to say that it is "most of the time" vertex shader limited based on testing at 320x200 seems a distortion..

Thats true, but we didn't know before that 3dmark03 is that heavy fillrate limited. I meant "most of the time" in 320x200 only, in higher resolutions it's more fillrate (except test 1) limited than anything else.

On a 9700, bandwidth is not a factor for many situations, but it exhibiting this behavior for an application does not mean that the application doesn't depend on bandwidth, just that it isn't a good comparative benchmark for comparing cards that share this behavior.

If the 9500 non pro versus the 9000 pro gives us indication of equal performance under the right conditions, this illustrates that the GT 2 and GT 3 are fillrate limited for the purposes of that comparison, which should give us indication of how ps 1.4 execution characteristics of that card compare to the characteristics of the 9500 non pro (i.e., they are the same). This would be a good reason to start believing this reflects what the benchmark (as far as GT 2 and GT 3) is able to test (the usefulness of a benchmark in the real world is dependent on the set of applicable items there are in the real world to be benchmarked), pending confirmation of actually evaluating other cards as they are released.

Right, I would love the have a radeon 9500 / 9500 pro. The radeon softmod should work in the other direction r9700->r9500, I'll see what I can do.

Thomas

 

[vrecan]

I dont know if it will help I underclocked my 9700pro I dont know if the scores will help at all.

320x200 VGA Memory Clock 258 MHz
VGA Core Clock 275 MHz
3DMark Score 8 K3DMarks
GT1 - Wings of Fury 195.9 fps
GT2 - Battle of Proxycon 77.8 fps
GT3 - Troll's Lair 56.1 fps
GT4 - Mother Nature 34.4 fps
Vertex Shader 13.0 fps
Pixel Shader 2.0 101.4 fps
Ragtroll 30.7 fps

1024x768mem 258 MHz
VGA Core Clock 275 MHz
3DMark Score 4012 3DMarks
GT1 - Wings of Fury 143.2 fps
GT2 - Battle of Proxycon 25.7 fps
GT3 - Troll's Lair 24.0 fps
GT4 - Mother Nature 22.9 fps
Vertex Shader 12.7 fps
Pixel Shader 2.0 34.4 fps
Ragtroll 16.8 fps

software VS 320x200
3DMark Score 1721 3DMarks
GT1 - Wings of Fury 64.3 fps
GT2 - Battle of Proxycon 17.2 fps
GT3 - Troll's Lair 9.3 fps
GT4 - Mother Nature 4.7 fps
Vertex Shader 2.3 fps
Pixel Shader 2.0 13.4 fps
Ragtroll 6.4 fps
:edit I can put the reg scores at default clocks if needed also:

 

[just me]

Thats it I would like to put out the challenge ATI vs Nvidia..Ice Hockey..name the place..

I think Chinese checkers would be more appropriate for Mssrs. Ho & Huang. 8)

 

[Evildeus]

DaveBaumann wrote:

The fact is, it is in DX, its a viable rendering option, and games are supporting it (I have a big list...).

Dave, just out of curiosity, what games are on the "big list"?

+1

 

[demalion]

There is a line at the bottom of there test, where I say something like this: The actual limitation factor (vs, ps, fillrate) depends also much on the graphics hardware and the r9700 pro is taken as a baseline for future dx9 cards, because 3dmakr03 is a dx9 benchmark.

So, we both agree that pixel shading is not a significantly limiting factor (in the GT 2 and GT 3 tests) for the 9700, but it may possibly be so for the benchmark? And that we have yet to determine if it is for the benchmark, but observing differences between the 9500 non pro and 9000 pro (we want as apples to apples as we can get...namely 4x1 and 128-bit bus, so a 9500 pro does not enter into this question about pixel shading, and perhaps we might want to turn off Hyper Z features) would be a better basis for observing whether the benchmark is useful for testing this performance than just observing the 9700 (the R300 with all 8 pipelines and a 256-bit bus) compared to itself? Finally, do we agree that we won't know in a final sense whether it is useful for comparing pixel shader performance until comparing to more widely divergent architectures (would most likely have to be slower architectures than the R300 since it already shows little limitation)?

Sorry I missed that text, things are a bit more cramped than I'm used to and I found the translation tricky to decipher in some spots.

Thats true, but we didn't know before that 3dmark03 is that heavy fillrate limited. I meant "most of the time" in 320x200 only, in higher resolutions it's more fillrate (except test 1) limited than anything else.

That bolded statement is still confusing me. Let me quote what I was replying to initially:

Test 1 is most of the time vertex shader limited, but fillrate comes into play with some very high resolutions.

Talking about GT 1 specifically:

The default test resolution is 1024x768. Framerate drops significantly as it is increased. Thus, why would you say GT 1 is not fillrate limited? Well, you show 320x200 resolution, where it is not. While the default resolution of 1024x768 may be "very high resolution" compared to 320x200, it seems odd to term it the way you did.

Right, I would love the have a radeon 9500 / 9500 pro. The radeon softmod should work in the other direction r9700->r9500, I'll see what I can do.

Thomas

Hmm...what I would find interesting, atleast as far as currently available cards go, is a comparison between: 8500/9100 @ 275/275, 9000 Pro (275/275), 9500 non pro (275/275, 128-bit bus), all with 128MB memory. With Hyper Z features turned on and off, at various resolutions and in all tests (well, the first 3 game tests and shader tests). Not very much data in the ORB yet, unfortunately, and it would be a big question mark of system config details and driver settings even if there were.

I also wonder how much "driver telling the GPU to do stuff via the AGP bus" overhead there is and how it the framerate for that compares to the "driver gets told to do nothing" you showed on that system.

 

[boobs]

Thats it I would like to put out the challenge ATI vs Nvidia..Ice Hockey..name the place...

Already tried that....nVidia backed out...chicken sh*ts!

http://www.theinquirer.net/?article=5912

Joe, seriously, honestly, do you work for Ati?

Instead of sitting here and arguing about 3DMark, how about some of the demo writers here code up a quick benchmark for DX 9.0 using HLSL or Cg so that Dave or Kyle can use it to check out what optimized code can do on Gffx and R300?

 

[tb]

So, we both agree that pixel shading is not a significantly limiting factor (in the GT 2 and GT 3 tests) for the 9700, but it may possibly be so for the benchmark? And that we have yet to determine if it is for the benchmark, but observing differences between the 9500 non pro and 9000 pro (we want as apples to apples as we can get...namely 4x1 and 128-bit bus, so a 9500 pro does not enter into this question about pixel shading, and perhaps we might want to turn off Hyper Z features) would be a better basis for observing whether the benchmark is useful for testing this performance than just observing the 9700 (the R300 with all 8 pipelines and a 256-bit bus) compared to itself? Finally, do we agree that we won't know in a final sense whether it is useful for comparing pixel shader performance until comparing to more widely divergent architectures (would most likely have to be slower architectures than the R300 since it already shows little limitation)?

yes.

The default test resolution is 1024x768. Framerate drops significantly as it is increased. Thus, why would you say GT 1 is not fillrate limited? Well, you show 320x200 resolution, where it is not. While the default resolution of 1024x768 may be "very high resolution" compared to 320x200, it seems odd to term it the way you did.

Okay, sorry for the confusion. I meant not fillrate limted at all, I meant not as much fillrate limited as the other tests.

Hmm...what I would find interesting, atleast as far as currently available cards go, is a comparison between: 8500/9100 @ 275/275, 9000 Pro (275/275), 9500 non pro (275/275, 128-bit bus), all with 128MB memory. With Hyper Z features turned on and off, at various resolutions and in all tests (well, the first 3 game tests and shader tests). Not very much data in the ORB yet, unfortunately, and it would be a big question mark of system config details and driver settings even if there were.

I also wonder how much "driver telling the GPU to do stuff via the AGP bus" overhead there is and how it the framerate for that compares to the "driver gets told to do nothing" you showed on that system.

yes, would be nice to investigate this isues...

Thomas

 

[Hellbinder]

Joe, seriously, honestly, do you work for Ati?

Instead of sitting here and arguing about 3DMark, how about some of the demo writers here code up a quick benchmark for DX 9.0 using HLSL or Cg so that Dave or Kyle can use it to check out what optimized code can do on Gffx and R300?

Because that is not the way its supposed to work. Also you are insinuating that 3dmark03's code is not optomized, or is somehow unworthy.

1. 3dmark03's shader routines were written with HLSL.

2. Code is *supposed* to be optomized for DX8,DX9 or OpenGL. It is then the companies problem if they cant run the damn thing.

Ati is not upset at the methodology used in developing 3dmark. either Sis or Any of the other Beta companies have seen fit to make a public statement. The bottom line is that 3dmark03 comes as close as you can to simulating the condistions in future games.

For a Synthetic benchmark that is all you can ask for.

 

[Hyp-X]

]
1. 3dmark03's shader routines were written with HLSL.

Only in the pixel shader 2.0 test.

 

[just me]

This is posted @ FM & I copied it for your reading pleasure:

A lot of fuzz has been raised about us using vertex shader skinning, and the fact that the skinning then needs to be performed repeated times. Our official statement explained the technical details behind our choice to still skin in the hardware.

Our vertex shader test gives an estimate that hardware vertex shader skinning is 5 times faster than CPU skinning on a system with both a high-end CPU and graphics card. This is a feature test, where there isn't much else to do than skinning. The questioned tests, Game Test 2 and 3, have a lot more going on than just skinning. In fact, our beta members and we agree that the bottleneck is above all the pixel shader, not the vertex shader. That is why the vertex shader feature test shows clear performance advantage of hardware vertex shader skinning, but the game test takes much less hit from it (this is where we compare our current implementation with a CPU skinning implementation, which is not possible to compare in the final released 3DMark03).

One aspect speaking for vertex shader skinning, that was not yet brought up as clearly, is that the whole idea behind vertex shaders (as it was with its 'kind of predecessor' HW T&L) is to offload graphics tasks from the CPU. Vertex shader skinning reduces the CPU workload during 3D game play, leaving the CPU more time to deal with physics (collision, ragdoll, cloth etc.), artificial intelligence, visibility optimization and a lot more. Hardware vertex shader skinning significantly improves the total system throughput, since the workload between CPU and graphics card is more balanced. You can try the ragtroll test (if your system is up to it). That test has a lot of both CPU work (physics) and graphics load. Check the frame rate difference between the default run and forced software vertex shaders and see how the total system throughput gets improved when using vertex shader skinning.

I could write a whole novel about this, but let's leave it here for now

Patric - 3DMark®03 Producer

Hope this helps in the understanding of what FM did & why.

just me

 

[Morris Ital]

I think Ed Stroglio summed it up best at www.overclockers.com :-

"It's fair to say there's plenty of hypocrisy and intellectual dishonesty to go around in all the pieces. If you think any one of the parties is as pure as the virgin snow, you are biased. "

As I see it nvidia come out badly in the test so disown it
Ati come out well so grab it with two hands.
Futuremark probably wish they had made it less agressive and more cpu dependent, after all they could bring out the SE edition in 1 years time with more polygons to test the current crop of cards more severely then.

I don't see many people entering this "happening" with an objective and neutral viewpoint .