Futuremark: 3DMark06 [Archive] - Page 2

Nick[FM]

18-Jan-2006, 23:29

So because your art direction was already set, you couldn't implement the technique because no scene required it? While deep freeze is graphically very impressive, that might have been your candidate for a different look and set piece that included some of the techniques people are asking for. You can't undo what's done, though.
Our artists have very much freedom to design the scenes as they see fit, but of course our programmers work with them to make sure that we get all needed stuff in there. None of the scenes really would benefit from POM (at least I can't think of any places where it would benefit and make the scene look better). In any case, we don't want to add effects just because they exist. I doubt that any game developer would add 1001 effects to games just because they exist.

I get the very distinct impression that this release of 06 was very much an in-house thing, with little to no consultation and discussion with your major industry (non-IHV) and media BDP partners.
Let me quote our press release:
Developed in conjunction with BDP members AMD, ATI, Dell, Imagination Technologies, Intel, Microsoft, NVIDIA, S3, SIS, Velocity Micro and XGI, this latest offering from Futuremark is a collaborative effort that reflects the projected workloads of game content two years from today.
That covers the BDP partners. We always collect information from the media (either via discussions, posts, reviews, comments etc.) and work by that. In some cases we work directly with them, but it is tricky due to NDA's between us and other companies. Discussions like these also give us a pretty good picture what users/media feels about our products, and what we could do better. We have also discussed our approaches and ideas with other developers.

Seems like a waste of resources to push out a (knowingly, I bet) controversial update to 3DMark this close to Vista and D3D10, and it smells IHV-led, on the face of it.
3DMark06 is our last DX9 benchmark before moving to new grounds. 3DMark05 lacked SM3.0 tests, and we also wanted to push the envelope for DX9 one more time. There were quite a few tricks and treats we wanted to have in our benchmarks before leaving the good'n'old DX9. :smile:

Nick[FM]

18-Jan-2006, 23:32

Can anyone answer me as to why my SLI'd GTX's are getting just about 52xx? I just swapped out a DFi SLI board, for a Asus one, and SLI is working fine in CoD2. Which is all that matters I guess, but Im a little confused as to why its not working in 3dmark06.

The drivers have a built in profile for 06, but the score seems to be well below what it should be.

Worm?
Sounds strange. Which drivers do you have installed? 3DMark doesn't actually "enable" SLI or "disable" it either. It is up to the drivers to determine if SLI should be on or off.

Dave Baumann

18-Jan-2006, 23:35

Wow, all things in favor of the big green. And within a couple hours of launch 3dmark06 has already been branded as an Nvidia sellout (notwithstanding the fact that it was probably developed on NV40/G70). Looks like 2006 is gonna be more of the same :roll:
Thats not what he was saying - the point being is that if you stick with something common there is then any accusations that are levelled are easily deflected; supporting that for one and that for another muddies things considerably.

Kombatant

18-Jan-2006, 23:35

FX55 with X1900 Series card
http://img37.imageshack.us/img37/8070/3dmark06fx553dr.th.gif (http://img37.imageshack.us/my.php?image=3dmark06fx553dr.gif)

FX60 with X1900 Series Card
http://img37.imageshack.us/img37/9137/3dmark06fx602zp.th.gif (http://img37.imageshack.us/my.php?image=3dmark06fx602zp.gif)

AMD64 3200+ with GTX512(NOC)
http://img37.imageshack.us/img37/475/3d06sinoc6af.th.jpg (http://img37.imageshack.us/my.php?image=3d06sinoc6af.jpg)

AMD64 3200+ with GTX512(OC)
http://img37.imageshack.us/img37/4517/3d06oc0cw.th.jpg (http://img37.imageshack.us/my.php?image=3d06oc0cw.jpg)

Something tells me that this picture seems very wrong.

US Hmm... if you think that
a) I get 4047 with my 1800xt, stock speeds
b) I get 1040 from my 3200+ o/c at 2.7GHz
c) see the FX60 score there,

that'd mean that, if i had a FX60, I'd get something a score of 4940 with my current card right? So isn't 5348 kinda low for a 1900? Unless it's not a high-end model, of course.

Moloch

18-Jan-2006, 23:36

']And which choices are the ones you refer to? It'd be good to know.

In order to avoid any wrong use of the scores (AA scores in this case) this was the best solution. As said, this applies to all cards where one setting (or value) disables any of the tests. We require that what tests are available must also be available with any settings, otherwise no 3DMark score is being outputted.

Depends on how you look at things really. In this case the HDR/SM3.0 tests require something the 6200 is not capable of even with default settings. AA is an optional thing, not in as default.

We wanted SM2.0 hardware to get a score in 3DMark06 as well. 3DMark05 is a great SM2.0 benchmark, but 3DMark06 is even better.

Not sure about this one.. Refer to SM2.0, SM3.0 or SM3.0 with FP16 blending support? :???:

Cheers,

Nick
Having nvidia and ati Sm3 hardware run different code because you chose to require an optional DX9 feature when the work around for cards which don't support results in in a clear disadvantage for the card which doesn't not support an optional feature.
Maybe you should have used 3dc because nvidia can support it though a hack:wink:

Razor1

18-Jan-2006, 23:37

First two will be the dual core drivers helping out the x1900 :wink: But the boost seems unatural though a bit too much.

Anyone can test a gtx 512 with a 4800+ and overclock the CPU?

Nick[FM]

18-Jan-2006, 23:38

Nick, can you find out a little more about the conditions that Fetch4 is used in please. I'm genuinely trying to find a useful Fetch4 test, and was asking about 3DM's use and I got a rather confusing message back that Fetch4 is only used in the Shader Model 2.0 tests - if thats the case, whats the reason for not using it on the SM3.0 tests (especially as the only ATI hardware that has Fetch4 is SM3.0 and ATI's actual SM2.0 parts need to use a different algorithm because they don't support Fetch4!).
FETCH4 is being used for all shadows in the SM2.0 graphics tests (for hardware that supports it of course). Due to the sampling method in the HDR/SM3.0 graphics tests, we weren't able to use neither FETCH4 or PCF in those tests. It simply wouldn't have worked due to the rotated grid we use.

Unknown Soldier

18-Jan-2006, 23:38

Yes, I was hoping for 3D nirvana :lol: I need to re-read the thread to get a better understanding of exactly how ATi got the shaft - right now it's just a jumble of acronyms and accusations.

I think it's pretty simple.

1) DST16 could've been used in the benchmark but wasn't (developers(or IHV's) preference I suppose)
2) DST24 is used, but cards that don't support the feature is forced to use F32(which might cost more but is an unknown factor atm).
3) Fetch4 used in SM2.0 which doesn't support Fetch4 but an algorithm of it is used.
4) SM3.0 supports Fetch4 yet it isn't used in the SM3.0 benchmark.
5) AA doesn't get scored with a certain IHV's card.

Anything else to add?

US

JasonCross

18-Jan-2006, 23:40

Jason, did you push for parallax mapping's inclusion during the 06 development process, as part of ET's participation in the BDP? Wavey, did you do anything for 06? Hanners?

Neither myself nor the ET editor Loyd Case were ever asked. We didn't see or hear anything about 3DMark06 until January (as in, this month, not last January).

Had someone asked me last summer, I would have said parallax occlusion mapping with a variable sample rate depending on distance from camera and angle, and sub-surface scattering (on flesh, liquids) would be two primary target areas. But hindsight is always 20/20. And I can certainly understand Futuremark's desire to prevent 3DMark from becoming "design by committee" or letting just their partners "buy" their way into having influence over the app.

I still can't reconcile why a GeForce 7800 gets no score with AA enabled. Enabling AA puts it in the same boat as an X800 or GeForce 6200 - able to complete only the CPU and SM2.0 tests. I think it should use the forumla for those cards in that situation.

There should be less of a hassle over optional features (knock on wood!) with the DX10 version of 3DMark. DX10 is much more rigirous in its requirements. There's a straight-up list of features, and either you have them and you're a DX10 card, or you don't have them and you're not. None of this hooey about various different shader models, optional texture formats, etc.

Nick[FM]

18-Jan-2006, 23:43

Having nvidia and ati Sm3 hardware run different code because you chose to require an optional DX9 feature when the work around for cards which don't support results in in a clear disadvantage for the card which doesn't not support an optional feature.
Maybe you should have used 3dc because nvidia can support it though a hack:wink:
They both run the same code really! The other simply doesn't support FP16 filtering, and thus needs to use a fallback, which I already said is very efficient! We do not require FP16 filtering. We require FP16 blending, which we have no fallback for.

3Dc would have increased the package by 2x (or something like that)... Same thing as when we discussed this topic with 3DMark05. Nothing has changed since that.

Nick[FM]

18-Jan-2006, 23:51

1) DST16 could've been used in the benchmark but wasn't (developers(or IHV's) preference I suppose)
We want to ensure that the shadows are rendered correctly, no matter how much we tune or tweak the cameras in the scenes. We chose 24 bit already back in 3DMark05, and we continued to do so in 3DMark06. We don't want to in a situation where we would possibly need to limit the scene due to the shadows not being properly rendered.

2) DST24 is used, but cards that don't support the feature is forced to use F32(which might cost more but is an unknown factor atm).
We have multivendor hardware shadow mapping support, and for hardware that do not have it, uses R32F (just as in 3DMark05).

2) Fetch4 used in SM2.0 which doesn't support Fetch4 but an algorithm of it is used.FETCH4 is supported in the SM2.0 graphics tests for any hardware with DF24 and FETCH4 support.

3) SM3.0 supports Fetch4 yet it isn't used in the SM3.0 benchmark.
We weren't able to use neither FETCH4 or PCF in the HDR/SM3.0 graphics tests due to the way we smoothen the shadows.

More? :wink:

Unknown Soldier

18-Jan-2006, 23:57

Ye, can you explain why this is happening? (http://www.beyond3d.com/forum/showpost.php?p=677040&postcount=250) I mean the R580 supports Fetch4, supports D24X8, has more pixel shader power(although most probably less vertex shader) but still falls behind the GTX512?

And it can't all be drivers.

US

Cowboy X

18-Jan-2006, 23:58

I think we would all still like an answer regarding the 7800's score of 0 with AA enabled .

fallguy

19-Jan-2006, 00:00

']Sounds strange. Which drivers do you have installed? 3DMark doesn't actually "enable" SLI or "disable" it either. It is up to the drivers to determine if SLI should be on or off.

Well, perhaps it is working. I disabled SLI, and the score dropped about 1500 points. After re-looking at scores I was comparing it to, they had X2 A64's, I do not. I guess thats where the extra 1000 points came from. Didnt know the CPU impacted scored so much. So I guess its case closed. Thanks for replying though.

Geo

19-Jan-2006, 00:01

I still can't reconcile why a GeForce 7800 gets no score with AA enabled. Enabling AA puts it in the same boat as an X800 or GeForce 6200 - able to complete only the CPU and SM2.0 tests. I think it should use the forumla for those cards in that situation.

Oh my. I just realized what you guys are pointing at here. Of course that means that non-AA scores will be used in comparitive reviews while AA ones can't be. No wonder there are large green smiles today, given that X1800 shines best when AA is used.

Moloch

19-Jan-2006, 00:04

Well I get 3399 3dmarks with a 7800GT at 445/1070 *factory speeds* and a 3200+ winchester.
http://service.futuremark.com/compare?3dm06=8617
Time for some fun ;)

SugarCoat

19-Jan-2006, 00:05

Hmm... if you think that
a) I get 4047 with my 1800xt, stock speeds
b) I get 1040 from my 3200+ o/c at 2.7GHz
c) see the FX60 score there,

that'd mean that, if i had a FX60, I'd get something a score of 4940 with my current card right? So isn't 5348 kinda low for a 1900? Unless it's not a high-end model, of course.

Dual core seems to help substantially. Using my results i posted before, a dual core 2.0 scores a CPU score of about 1400. Quite a bit more then yours at 2.7. May not be a big deal but when you overclock your processor by 35% and score lower (small margine of error due to my cards settings) thats a rather large boost from dual core. There will be some real problems with these CPU scores and trying to make a performance basis off your total score like people are use to doing. One can dream that this will be a performance indication of future games though with the effect of dual core :)

I'm going to disable a core and see what happens to the total score as well as the CPU score. I would like to see if its the driver SMP or the 3Dmark program making such a substantial difference.

Personal request, Can someone with a GTX force on x16 HQ AF please and post scores in 2.0/3.0 and total?

digitalwanderer

19-Jan-2006, 00:07

Oh my. I just realized what you guys are pointing at here. Of course that means that non-AA scores will be used in comparitive reviews while AA ones can't be. No wonder there are large green smiles today, given that X1800 shines best when AA is used.
Yeah, there are a few questions being raised that'll I'll be interested in watching the answers develop to too....right now I'm still just trying to keep my head above water following along with the discussion. :oops:

Thanks for taking the time/abuse to answer Nick, it's most appreciated. I imagine you're having a rather busy day today... :lol:

Nick[FM]

19-Jan-2006, 00:09

Nick[FM]

19-Jan-2006, 00:13

Thanks for taking the time/abuse to answer Nick, it's most appreciated. I imagine you're having a rather busy day today... :lol:
Of course I will spend time to answer important questions. It seems that there are too much false (or distorted) information in regards to the hardware shadow mapping etc. Still, I'm prepared to follow up the discussions and make sure that everyone knows as much as possible what is being supported and what is not. Busy day? Yep! :grin:

Unknown Soldier

19-Jan-2006, 00:13

Ye .. a weeks saving grace I suppose. ;)

Guess the flak cannons will be out in full force then. I'd hate to be you.

Thanks for the answers. I like what you've did with your product.

US <-- Off to bed

Reverend

19-Jan-2006, 00:14

I haven't d/l this thing yet but from the comments :

3DM05 started on That Road I didn't like. This one appears to have gone further down That Road.

micron

19-Jan-2006, 00:31

I haven't d/l this thing yet but from the comments :

3DM05 started on That Road I didn't like. This one appears to have gone further down That Road.
If you're one thing Rev.....it's predictable.

....and in your case, it's not a good thing.

SugarCoat

19-Jan-2006, 00:35

Richteralan

19-Jan-2006, 01:16

ANova

19-Jan-2006, 01:19

']FETCH4 is supported in the SM2.0 graphics tests for any hardware with DF24 and FETCH4 support.

More? :wink:

Maybe you can explain to me why DF24 is required in order to use fetch4 and DFC? Also, why did you decide against HDR+AA on the 7x00 series (since it is not supported) but for DST24 even though it is also not supported on the X1800? This is what I mean by a double standard.

Apple740

19-Jan-2006, 01:37

']We want to ensure that the shadows are rendered correctly, no matter how much we tune or tweak the cameras in the scenes. We chose 24 bit already back in 3DMark05, and we continued to do so in 3DMark06. We don't want to in a situation where we would possibly need to limit the scene due to the shadows not being properly rendered.

If i compare a 3DMark05 Ati/Nv image, i can see that the Nv shadows (due to DST) are more aliased than Ati's shadows. I have never seen a reference image so i don't know which card is "correct", but i assume Ati (shadows have softer edges).
So your statement about 'ensuring correct shadow rendering' surprises me a bit, because it seems you didn't care for that in 3DM05.

How it is in the new 3DM06 i don't know.

trinibwoy

19-Jan-2006, 01:53

I think it's pretty simple.

1) DST16 could've been used in the benchmark but wasn't (developers(or IHV's) preference I suppose)
2) DST24 is used, but cards that don't support the feature is forced to use F32(which might cost more but is an unknown factor atm).
3) Fetch4 used in SM2.0 which doesn't support Fetch4 but an algorithm of it is used.
4) SM3.0 supports Fetch4 yet it isn't used in the SM3.0 benchmark.
5) AA doesn't get scored with a certain IHV's card.

Anything else to add?

US

Thanks :) But I would like to know that if you were a game developer targeting today's hardware and trying to implement features similar to those in 3dmark06, what decisions would you have made differently concerning the above points. Is there anything that could have been done to improve ATi's performance and not just decrease Nvidia's performance?

Cowboy X

19-Jan-2006, 02:00

SugarCoat

19-Jan-2006, 02:09

I'm not sure why FM included the CPU mark in final 3DMark. So, a computer with 6600GT but couped a dual-core CPU can score higher than a 6800GS with single-core CPU.
Does it corresponds to actual gaming experience?

I dont really understand either as your total score takes into account your CPU now in 3Dmark06, you have to look at the graphic scores more specifically if you want just your cards output. As you can see keeping both cores enabled or disabled basically does nothing to the graphic bench scores, but the total score is pretty drastic in differences. For instance, look at the screenshots of the FX60 and the X1900 card, look at its results in SM2 and SM3, and compare them to the results of the OC'd GTX 512 with a 3200+, you'll notice the GTX 512 is outputting a significantly higher score in the graphics tests, but because of the CPU tests, the X1900 actually recieves a very high score for the differences in the graphic tests between the cards.

I'm not too sure im happy with the way the CPU tests were implimented in the way they were either. I actually would of preferred something like modified 3Dmark01 tests using graphics cards to show CPU power, and then CPU benchmarks like the ones used as a seperate score and benchmark. I like where they were going, but not the confusion this might cause. Total score is now more worthless in showing a computers graphic horsepower in my opinion basically.

Moloch

19-Jan-2006, 02:27

Here's my mild OC run.. A64 at 2156 (but HTT at 862 mhz) and 7800GT at 520/1200 (wowzers!!)
For some reason 3dmark thinks my 7800GT is at 445mhz:???:
It also is using the CPU HTT I boot at not the one I set in clockgen.
http://service.futuremark.com/compare?3dm06=10482
38XX 3dmarks
Not a huge change between 445mhz/1070 and 520/1200.
Me thinks I need more GPU bandwidth and CPU clockspeed.

/brag I'm number one for A64s from 2000 mhz to 2155 and 7800gt's.
On a side note, futuremark needs to update the search and compare so non pro users when they search and compare and limited to the res they tested at.
Only guy above me ran at 1024 res...

Sunday

19-Jan-2006, 02:39

Galduta

19-Jan-2006, 02:44

AMD64 3200+ with GTX512(NOC)
http://img37.imageshack.us/img37/475/3d06sinoc6af.th.jpg (http://img37.imageshack.us/my.php?image=3d06sinoc6af.jpg)

AMD64 3200+ with GTX512(OC)
http://img37.imageshack.us/img37/4517/3d06oc0cw.th.jpg (http://img37.imageshack.us/my.php?image=3d06oc0cw.jpg)

Something tells me that this picture seems very wrong.

US

Is a GTX256 SLI ;)

Pete

19-Jan-2006, 03:19

I think we would all still like an answer regarding the 7800's score of 0 with AA enabled .I'm still not quite clear on that, but given that we have 3DM06's scoring formula, can't we just create it ourselves?

I'm not sure why FM included the CPU mark in final 3DMark. So, a computer with 6600GT but couped a dual-core CPU can score higher than a 6800GS with single-core CPU.
Does it corresponds to actual gaming experience?Well, if 3DM06 is actually meant to predict game performance in two years (somewhat odd, given the name, but I guess it's in keeping with previous 3DMs and in the sense that low-end models will run it acceptably in that time frame), including dual-core CPUs might make sense if games (and drivers) are going to go multi-threaded in a big way. They might, thanks to the new consoles, and at least one game points the way (http://www.amdzone.com/modules.php?op=modload&name=Sections&file=index&req=viewarticle&artid=225&page=1).

SugarCoat

19-Jan-2006, 03:26

This is one dreadful “farewell”!

3Dmark has totally transformed! 03 was the peak, and with this one they have hit the ground! For crying out loud, they did not changed the scenes, only added one, that is pretty useless form the real game design, just like any other??? Is this the gamer’s benchmark anymore?

And everyone is giving thanx to Nick? What for? He’s just doing his PR job, and not to well I must say. He’s missing to answer some more important questions giving a lousy PR spin around! Why? Is this community of the smartest 3D enthusiast on Net? We don’t need no PR spinning! If he would like to answer “real” questions, then fine, if not he might as well try not to quote their PR announcement on this thread!

05 was fallback in artistic and conceptual sense, and this one hit the bottom (and you require $500 for a pro license?!!?) Only thing that can bring you to the senses is:
1. Boycott
2. Any competition
When you stop making that much many you’re making now, maybe you will ask your self “maybe we did not drink all of the wisdom of this world”!
I don’t have any more expectations from you, mine next prime DX10 benchmark will be CryEngine 2! Maybe some one smart enough will find that this is the best time to launch some serious commercial benchmark, and give you, and your teammates some creative farewell!

Sunday

yea i wont be buying this version, but for some reason i dont think it will take them nearly as long to release 3Dmark07 with DX10 and all new benchmarks. They have to develop products just like the graphics card companies in line with Direct X releases. Theres only so much in my opinion they can do with DX9.0C that 3Dmark05 didnt cover. HDR honostly seems to be the biggest highlight.

Part of me wonders if they dont already have their DX10 3Dmark done or mostly and have been sitting on it through Vista push backs. Not to knock their work but this seemed to take them an exceptionally long time to release for what was changed.

CMAN

19-Jan-2006, 03:50

I'm not sure why FM included the CPU mark in final 3DMark. So, a computer with 6600GT but couped a dual-core CPU can score higher than a 6800GS with single-core CPU.
Does it corresponds to actual gaming experience?

I don't think it is necessarily a bad thing. If you look at how the results screen is set up, then I would assume they are getting away from a single score and going towards multiple scores. They seem much more interested in testing shader version power than anything else, and the lack of CPU score/importance has been fixed (maybe overshot though). As what I'd call a minor (05 to 06), I'm impressed by how well the older scenes look with just a little more elbow grease (especially the forest scene).

Rur0ni

19-Jan-2006, 04:26

Well if im not mistaken, there are some games that do get a boost from dual core cpus and recent drivers. That should be the trend too. We're going to dual, then quad, then multi(+). Processor speed isnt really going up all that well. So i'm glad to see dual core systems to get that boost in score.

poly-gone

19-Jan-2006, 04:39

']Have you tried all our official mirrors yet? I know most of them are getting hammered, but some seem to work pretty ok (Guru3D and MajorGeeks).
My download got cut-off at 80.0 MB on Guru3D the first time, 62.5 MB the next time, 126.3 MB on Major Geeks!!! Now I'm trying your Futuremark torrent, so let's hope that goes well :(

Chalnoth

19-Jan-2006, 04:45

My download got cut-off at 80.0 MB on Guru3D the first time, 62.5 MB the next time, 126.3 MB on Major Geeks!!! Now I'm trying your Futuremark torrent, so let's hope that goes well :(
For normal downloads, I usually use GetRight for this sort of stuff. You can use the program to download from many servers at once, and it's got really reliable download restarting. But at the moment I'm trying the Torrent (though this was probably a bad decision....seems to be running slowly, but it's almost halfway done, so I'm not going to stop now).

SugarCoat

19-Jan-2006, 04:48

My download got cut-off at 80.0 MB on Guru3D the first time, 62.5 MB the next time, 126.3 MB on Major Geeks!!! Now I'm trying your Futuremark torrent, so let's hope that goes well :(

right click and save target as on this, it will take 1 minute to start but should be fast:
http://www.webnallen.se/download/3DMark06_v102_installer.exe

and give a swede a hug

Reverend

19-Jan-2006, 04:58

If you're one thing Rev.....it's predictable.

....and in your case, it's not a good thing.
Well, do you know what I was talking about exactly for you to pass such a comment about me? Here's a hint : look at Dave's comments in this thread, look at the reasons that ultimately made me leave the BDP (FM also kicked me out at the exact same time, if you'll remember) back then.

3DM05 is the percursor of what we see in 3DM06. The one-fork road 3DM05 was on has now turned into one with many forks, leading to not just two different destinations but a few. I love looking at all 3DM tests but that's not the point, is it?

Reverend

19-Jan-2006, 05:01

']In any case, we don't want to add effects just because they exist. I doubt that any game developer would add 1001 effects to games just because they exist.
This is just a suggestion but since your app is supposed to be a forward-looking benchmark (and not a game), perhaps it's not such a bad idea to incorporate as many effects/features as possible in order to inform the public just what the heck those effects/features look like or perform like...?

I know FM likes to make a benchmark like a game developer makes a game but this is, to be blunt, impossible.

poly-gone

19-Jan-2006, 05:09

For normal downloads, I usually use GetRight for this sort of stuff. You can use the program to download from many servers at once, and it's got really reliable download restarting. But at the moment I'm trying the Torrent (though this was probably a bad decision....seems to be running slowly, but it's almost halfway done, so I'm not going to stop now).
Yeah, I was using FDM (another excellent download manager) but it said "Unknown Server Error" all three times...

right click and save target as on this, it will take 1 minute to start but should be fast:
http://www.webnallen.se/download/3DM..._installer.exe

and give a swede a hug
Thank you :).

Chalnoth

19-Jan-2006, 05:21

I know FM likes to make a benchmark like a game developer makes a game but this is, to be blunt, impossible.
Yeah. Personally, while Futuremark's method of developing benchmarks is really great-looking, I don't think it's the best in terms of really comparing to how these cards should perform in games.

This is my personal idea as to what would be the best type of synthetic benchmark.

Firstly, there are two separate purposes for developing a synthetic benchmark. The first would be to compress game benchmarks into one easy to run suite that is still representative of game performance. The second would be to produce a benchmark that seeks to separate out the performance of particular features of a graphics card, and maybe even attempt to analyze how these apply to games.

I'm just going to look at the first purpose. The only way to effectively do it is to talk to software developers. I know that Futuremark has been in heavy collaboration with hardware developers, but there doesn't seem to be much of any collaboration with game developers. If you really want to make a benchmark that is representative of games, either current or future, then talking to game developers seems an obvious choice.

Now, with this idea in mind, the second part would involve attempting to develop a robust suite for analyzing the makeup of games. One could look at things like mean values for overdraw, alpha testing/blending, shadowing algorithms, and, most importantly, shaders. This would be a very difficult thing to do, and there would be a fair amount of arbitrariness in the weighting across different games and across different situations within games. But it seems that it would at least be possible to attempt it.

Then, once you have a good amount of data, both of current games and of games in development, of typical usage statistics, you could attempt to produce a benchmark that would replicate these different scenarios. It wouldn't be pretty-looking, but if you do it right, it could be an excellent way to determine how different graphics cards should compare to one another in real games. And you could even adjust the weighting of various aspects of the benchmark in order to get a score as it relates to old games, new games, and future games.

Reverend

19-Jan-2006, 05:34

The only way to effectively do it is to talk to software developers. I know that Futuremark has been in heavy collaboration with hardware developers, but there doesn't seem to be much of any collaboration with game developers. If you really want to make a benchmark that is representative of games, either current or future, then talking to game developers seems an obvious choice.
I think FM has tried but generally speaking I think game developers would rather not be associated with FM/3DM. This is based on my experience (a prominent ex-FM staff asked me to ask Tim to help, or join their BDP sometime during 3DM05... I can't really remember the specifics... I am also sure FM approached John). Whether you want to look at it financially or from an altruistic POV, the reason(s) should be fairly obvious. FM may beg to differ, of course.

Brent

19-Jan-2006, 05:38

Or you could just use games to find out how games perform...

Moloch

19-Jan-2006, 05:48

Or you could just use games to find out how games perform...
what will [H] be doing with regards to 3dmark06?
it favors nvidia so perhaps you should incorporate it into your test suite :wink:

Brent

19-Jan-2006, 05:50

oh please, i'm not even going to respond to that

i know, using games to find out gaming performance, wild concept eh

Hubert

19-Jan-2006, 05:53

To be fair, fetch4 is supported.

However, I'm starting to agree...seems like quite a few inconsistent decisions have been made.

My question is: did they have thorough knowledge of the R520 architecture when they took these decisions ? This alone would explain a lot, IMO.

Richteralan

19-Jan-2006, 06:04

I don't think it is necessarily a bad thing. If you look at how the results screen is set up, then I would assume they are getting away from a single score and going towards multiple scores. They seem much more interested in testing shader version power than anything else, and the lack of CPU score/importance has been fixed (maybe overshot though). As what I'd call a minor (05 to 06), I'm impressed by how well the older scenes look with just a little more elbow grease (especially the forest scene).

I do agree with your points.
I do think including CPU tests are essential.
But what I know 3DMark is for testing 3D? CPU dual-core wise will help physics and AI that's not quite related to 3D?
And if it counts, why include the CPU score in final 3DMark score? It can be a independent score just like 3DMark05 and before. Do we will get benifits from dual-core graphic wise? Even few years later(assume FM won't release another 3DMark?)

Mark

19-Jan-2006, 06:06

what will [H] be doing with regards to 3dmark06?
it favors nvidia so perhaps you should incorporate it into your test suite :wink:Even though there are some, shall we say... "curious"... design decisions with 3Dmark06, they did manage to get HDR+AA in there for ATI X1x00 cards and it looks really nice (particulairy the Deep Freeze test).

The inclusion of the CPU score into the final 3DMark score takes the "3D" out of 3DMark though. That was a terrible decision in my opinion. For that, and other reasons, I don't think I'll be using it in my reviews. In fact, I think I've just decided to drop all versions of 3Dmark from my benchmark suite.

Moloch

19-Jan-2006, 06:12

Even though there are some, shall we say... "curious"... design decisions with 3Dmark06, they did manage to get HDR+AA in there for ATI X1x00 cards and it looks really nice (particulairy the Deep Freeze test).

The inclusion of the CPU score into the final 3DMark score takes the "3D" out of 3DMark though. That was a terrible decision in my opinion. For that, and other reasons, I don't think I'll be using it in my reviews. In fact, I think I've just decided to drop all versions of 3Dmark from my benchmark suite.
Ya that's nice but as mentioned int his thread whats the deal with nvidia cards getting no score.. rather than a reduces score as with SM2 cards.

As for cpu scores counting.. I think that is another bad decision.
If want cpus to count why not have a game test that's more cpu limited.. say having it doing lots of physics calculations since they are partnered withe ageia.

boltneck

19-Jan-2006, 06:35

Reverend

19-Jan-2006, 06:56

Or you could just use games to find out how games perform...
Brent, unless you're on a crusade, it is best we don't re-read your stance again and again over here. Feel free to do so at the site you write for, however.

The way I have responded in this thread isn't about using what is best for measuring 3D hardware performance in games (which is what they are primarily created for, financially for the IHV). This thread, I think, is about FM and its 3DM app. Unless you want to discuss this instead of "Games are best" one-liner posts, I think you're wasting space.

IOW, we get your message(s). No need to for us to view your participation here as utterly boring and one-topic.

Hubert

19-Jan-2006, 07:22

How does 3Dmark06 favor Nvidia?

You would think that by default it should favor ATi with all the work they have done on optimizing SM3 shader execution.

Have they dont something thats just a little to obvious here?

Well, what I heard, support for Dynamic Branching is one of the major things in SM 3.0. Ati focused on that with R5xx, but this does not show in 3DMark 2006. As we know from other tests Ati indeed succeded in creating a strong performer in this department, we only can think that either the amount of dynamic branching shaders is low compared to other ones or they are not using significant branching.

rwolf

19-Jan-2006, 07:41

chavvdarrr

19-Jan-2006, 07:49

Kombatant

19-Jan-2006, 07:54

I'm not sure why FM included the CPU mark in final 3DMark. So, a computer with 6600GT but couped a dual-core CPU can score higher than a 6800GS with single-core CPU.
Does it corresponds to actual gaming experience?
That is my main concern at this point as well - the dual core CPUs alter the score in such a great way, it even makes a Pentium D 2.8 GHz score 30% higher than an AthlonFX 57! If I use it, I don't think I'll use the total 3DMark score; but just the SM2/SM3 scores.

rwolf

19-Jan-2006, 08:06

http://www.firingsquad.com/hardware/3dmark_06/page11.asp

We’ve only begun benchmarking with the latest build of 3DMark in the past 18 hours, so we haven’t had enough time to come to any firm conclusions on how the X1600 performs so strongly, while it can be said that the X1800 XT disappoints, often finishing behind the GeForce 7800 GTX 256MB in many tests, even though real-world testing with today’s latest games indicates the opposite. We’re sure conspiracy theories will begin popping up in various forums shortly.

This conclusion is bang on.

dizietsma

19-Jan-2006, 08:07

To me the "games" are more to do with league tables and competition rather than being the determinent how good the card is or is not in a specific test; the individiual tests should be used for this instead. The games are a bit of fun that has made 3dmark popular, the tests give you the raw data.

By Futuremark giving the games differing point scoring means that subjective opinon has already crept in, never mind what to include or not include in the games. A classic example of this was 3dmark03 GT1 which not only had a mere 7 points per fps because it was DX7 and not very stressful but also was mainly single textured which favoured the Ati R300 series cards ( remember how nvidia got upset about that and there was that HUGE 3 post thread on b3d because nvidia was not favoured ? :p )

To me 06 is how 05 should have been, it does better a job of estimating current trends in games, especially with the inclusion of the cpu test which sure will be offloaded more and more work from the gpu's. If you have quad cores in 2007/2008 you need to have something to justify them !

Neeyik

19-Jan-2006, 08:24

question:
3.0GHz iP4 - i get 1 frames every few seconds... WHAT FOR ?! unable to do something that looks so-so and will run with reasonable (>10fps) speed ? What kind of system will be able to run smooth this test ? 4 sockets , 8 ore Opteron rig ?
Nice visuals, but too less balance IMHO.
Read the manual - it's never going to run at 10+ fps because the tests are set to a fixed frame rate of 2.

Hubert

19-Jan-2006, 08:26

question:
Why FM decided that no current generation card will be able to make 25 fps !?
Why not make a bench that can look well and run SMOOTH on current cards (6800/X800 + ), and next gen to run even better (like 50 vs 25 fps)
yea, the user right now is able to "see" how smoother 3dmark06 will run, after he buys 6 months later AGAIN a new 5-6-700$ videocard ...

personally i'd prefer NOT to be hit so hard in face "upgrade, upgrade, UPGRADE, UPGRADE"
nice point, maybe because they are FutureMark and testing future games' behaviour is the target. And, sadly, future games will run on actual systems just as 3DMark 2006 does. Talking about games from not the immediate future. But FutureMark will never get it right, it's hard to predict the future, IMO. :)

and CPU test is... a joke
3.0GHz iP4 - i get 1 frames every few seconds... WHAT FOR ?! unable to do something that looks so-so and will run with reasonable (>10fps) speed ? What kind of system will be able to run smooth this test ? 4 sockets , 8 ore Opteron rig ?
Nice visuals, but too less balance IMHO.

With the CPU test, I read that there is a limit of 2 fps, in order to minimize the effect of the videocard. You know, it's not the CPU churning the vertexes as in 3DMark 2003, but it only does physix and AI. Of course, the scene needs to be rendered, and that's done by the GPU, but in order to avoid the influence on the results, it is set this very low limit. I hope I got it right ...

IgnorancePersonified

19-Jan-2006, 08:44

having a long - shit day but d/led this at work and just ran it:
A64 X2@2100 9800pro 370core/340 mem
3D Mark Score= 606
SM2.0 = 281
Cpu Score = 1589.

Only watched a bit of it but it was a slideshow.

Poor old arthritic 9800 was getting flogged.

ZoinKs!

19-Jan-2006, 08:47

Or you could just use games to find out how games perform...
And which games help show how well cards will keep up with techniques that'll be used in games in 1-2 years from now?

Part of the point of 3dMark is to help indicate future performance of hardware. How well it actually manages to do this varies from version to version: '03 predicted future trends of the early sm 2.0 cards perfectly, I'm not so sure about '05, and as you can see from this thread there's good reason to doubt how useful '06 will be.

But when in the hands of those who know what they're doing 3dMark has often been a useful tool for discovering how strong hardware is and how well it'll keep up. But for clueless people who don't understand what it's for or how to use it (i.e. someone who goes around continually complaining that it doesn't present exact correlation with current games) it's quite useless.

Nick[FM]

19-Jan-2006, 09:05

Maybe you can explain to me why DF24 is required in order to use fetch4 and DFC? Also, why did you decide against HDR+AA on the 7x00 series (since it is not supported) but for DST24 even though it is also not supported on the X1800? This is what I mean by a double standard.
Good morning! Sorry it took me a while to get back here, but I had to get some sleep.
:smile:

DF24 works with FETCH4, but certainly has nothing to do with DFC (or do you mean PCF?). DF24 and FETCH4 go hand in hand, as D24X8 and PCF does. Dynamic Flow Control (DFC) has nothing to do with FETCH4, PCF, DST etc. We didn't decide against HDR+AA (a DX feature) since we support it in 3DMark06. If any hardware doesn't support that feature, then it .. simply doesn't. :???: DF24 is supported since it helps in shadow rendering on hardware that supports the feature (just like D24X8 on other cards). I am still not sure how this can be seen as "double standard". We now have multivendor hardware shadow mapping support, which we didn't in our previous 3DMark. What's wrong with us supporting more vendors' hardware shadow mapping?

I think that all these abbreviations confuse a lot of people..

* edit: corrected some errors in the post.

Cheers,

Nick

Nick[FM]

19-Jan-2006, 09:12

I've noticed he is answering just about every question except that .
If "he" is me, I'm sorry but I need to sleep sometimes.. :wink: Anyway, I do answer all questions the fastest I can.

Cheers,

Nick

dizietsma

19-Jan-2006, 09:12

Can someone clear up for me the cpu side of it ? From the white paper it seems that you have

1 thread or cpu doing the main logic which calls 1 thread or cpu doing physics and 1 or more threads or cpu doing AI dending on how many cores you have extra

So ideally the new Intel 955 with 2 cores with hyperthreading ( 4 logical processors ) would benefit best as you have 1 logic, 1 physics and 2 AI.

If the AGEIA chip is present does that do all the work and what "cpu" score does it get ? Nick can you clarify ?

JasonCross

19-Jan-2006, 09:13

Read the manual - it's never going to run at 10+ fps because the tests are set to a fixed frame rate of 2.

It's not that the test runs at a fixed frame rate...it runs faster or slower depending on the speed of your CPU(s). It's worded poorly in the help file.

The CPU tests use fixed frame-based rendering, meaning that there are X number of frames per second of gameplay (in this case, 2). So when it renders 40 frames, that always equals the exact same 20 seconds of gameplay. Quake/Doom timedemo are like this, though I think it's usually 20 frames per second in those. They made the CPU tests run with frame-based rendering instead of time-based like the graphics tests to ensure that all CPUs end up running the same number of pathfinding and physics intervals on the same number of units.

In theory, if your CPU was fast enough, you could run the tests at 3 or 4 frames per second: it would still render 40 and 60 frames for the two tests, equalling 20 and 30 seconds and performing calculations for the same number of physics and pathfinding intervals. Anyone got a seriously overclocked dual-CPU, dual-core Opteron system to test it out? :)

People shouldn't be concerned that it runs at an extremely low frame rate. It's irrelevant, since you're just running those tests to get an overall score. But it *is* appropriate that the CPU is included in the overall score. 3DMark strives to be an overall gaming benchmark, not a pure graphics benchmark. It hasn't done a good job of this in the past, being fully graphics-bound nearly all the time. But this time, the CPU test stresses CPU functions from games almost exclusively: D* Lite pathfinding algorithm and Ageia physics, as well as game logic. It's multithreaded, so it's pretty forward-looking as only a couple of games so far take advantage of dual core CPUs (though that will become very common in the future).

mongoled

19-Jan-2006, 09:14

I still can't reconcile why a GeForce 7800 gets no score with AA enabled. Enabling AA puts it in the same boat as an X800 or GeForce 6200 - able to complete only the CPU and SM2.0 tests. I think it should use the forumla for those cards in that situation.Yup this is sticking out like a sore thumb, im still waiting to see an answer to this question. I am way out of my depth when reading alot of the technical stuff discussed over here at B3D, but the point JasonCross makes is very clear to me. So could a 3Dmark representative give an explanation to a non-tech savvy person as myself as to why (7800 AA score give no score *confused* ) this is the case, as it obviously scews the result when sites use your benchmark to make comparisons?

JasonCross

19-Jan-2006, 09:15

So ideally the new Intel 955 with 2 cores with hyperthreading ( 4 logical processors ) would benefit best as you have 1 logic, 1 physics and 2 AI.

If the AGEIA chip is present does that do all the work and what "cpu" score does it get ? Nick can you clarify ?

It doesn't support the PhysX PPU, no.

I think in the case of 4 logical CPUs (P4 955), you'd have 1 main game logic thread, 1 physics, and *4* pathfinding threads. At least, this is how FutureMark explained it to me.

Nick[FM]

19-Jan-2006, 09:33

As for cpu scores counting.. I think that is another bad decision.
If want cpus to count why not have a game test that's more cpu limited.. say having it doing lots of physics calculations since they are partnered withe ageia.
And what's the difference if we would have physics & AI in the graphics tests, eliminating pure GPU benchmarking? :???: We now have 4 graphics tests and 2 CPU tests (which do use lots of physics, AI etc.), and we are able to output a 3DMark score based on your systems' gaming performance, and sub scores for pure graphics & CPU benchmarking. I'm sorry, but I don't see the logic in your post since that's what we did. Only difference is that we separated those two aspects (CPU & GPU) in order for people to do more in-depth benchmarking.

Cheers,

Nick

Nick[FM]

19-Jan-2006, 09:40

I think the solution is for FM to create a patch with a setting that makes the NV card use the same pixel shader workaround for 24-bit depth stencil textures. This way we could see if 3DMark06 is a benchmark or an NV demo. In fact why not make a patch that makes all cards use one code path through a setting.
This confuses me a bit.. Workaround for 24bit depth stencil textures? Simply disable the hardware shadow mapping from the benchmark settings, and no card uses the hardware shadow mapping. It is already in there and works for all cards. We also have an option "Force software FP filtering" if you'd like to disable the hardware FP16 filtering support. Everything is in there, it is just a matter of taking a look in the benchmark settings. :smile:

Cheers,

Nick

Nick[FM]

19-Jan-2006, 09:43

The inclusion of the CPU score into the final 3DMark score takes the "3D" out of 3DMark though. That was a terrible decision in my opinion. For that, and other reasons, I don't think I'll be using it in my reviews. In fact, I think I've just decided to drop all versions of 3Dmark from my benchmark suite.
Sad to hear that you will go that route. I don't really see what bad there is with adding the CPU score to the final 3DMark score, since the 3DMark score should reflect future gaming performance. Why not use the sub scores for pure CPU and GPU reviews/benchmarking? Do you think they are useless or why won't they be useful for you?

Cheers,

Nick

Nick[FM]

19-Jan-2006, 09:45

Yup this is sticking out like a sore thumb, im still waiting to see an answer to this question. I am way out of my depth when reading alot of the technical stuff discussed over here at B3D, but the point JasonCross makes is very clear to me. So could a 3Dmark representative give an explanation to a non-tech savvy person as myself as to why (7800 AA score give no score *confused* ) this is the case, as it obviously scews the result when sites use your benchmark to make comparisons?
It doesn't get a 3DMark score, but it does get all available sub scores which are as useful for comparison as the final 3DMark score.

Cheers,

Nick

Unknown Soldier

19-Jan-2006, 09:46

Thanks :) But I would like to know that if you were a game developer targeting today's hardware and trying to implement features similar to those in 3dmark06, what decisions would you have made differently concerning the above points. Is there anything that could have been done to improve ATi's performance and not just decrease Nvidia's performance?

It isn't about making Nvidia slower .. it's about making ATI faster. It seems things could've been done to increase the performance of ATI hardware i.e. R520 but it was overlooked.

I'm still interested in the X1900XT benchmarks but I guess this will wait till next week.

US

19-Jan-2006, 09:50

']I am still not sure how this can be seen as "double standard". We now have multivendor hardware shadow mapping support, which we didn't in our previous 3DMark. What's wrong with us supporting more vendors' hardware shadow mapping?

I believe the same reason was given by FM in your previous 3DMark. Supposedly there was another IHV (S3?) who used DST24/PCF and that was the reason given by FM to justify it's use in 3DM05. So here you are saying that there wasn't multivendor support during 3DM05 which basically means that all scores on GF6 cards should be revised since it adds quite a significant amount of points to the final score.

Kombatant

19-Jan-2006, 09:55

mongoled

19-Jan-2006, 10:02

']It doesn't get a 3DMark score, but it does get all available sub scores which are as useful for comparison as the final 3DMark score.

Cheers,

Nick
Thanks for taking the time to answer, but you still seem to have side-stepped my question
So could a 3Dmark representative give an explanation to a non-tech savvy person as myself as to why (7800 AA score give no score *confused* ) this is the case?

-EDIT- The emphasis in this question is placed on the WHY! the 7800 with AA gets no score

Surely it would make more 'sense' tht a score is given when AA is enabled. I say 'sense' because its definiton is meaningful in this context, and in the context of my eyes , the action your company has taken makes no sense to me. It seems illogical........

Nick[FM]

19-Jan-2006, 10:04

I believe the same reason was given by FM in your previous 3DMark. Supposedly there was another IHV (S3?) who used DST24/PCF and that was the reason given by FM to justify it's use in 3DM05. So here you are saying that there wasn't multivendor support during 3DM05 which basically means that all scores on GF6 cards should be revised since it adds quite a significant amount of points to the final score.
I see your point. In 3DMark05 we already had multivendor DST support which is true, but we only supported D24X8 & PCF (only available hardware shadow mapping at the time). Now we support D24X8 & PCF and DF24 & FETCH4. Maybe I should call in "extended multivendor DST support"? :smile:

Cheers,

Nick

Nick[FM]

19-Jan-2006, 10:10

Nick, one of my main gripes is the amount of influence the CPU has in the final 3DMark score; especially dual core processors raise performance too much (in my opinion, of course). I gather that you folk believe that multi-core processors will result in such a performance increase in future games, and hence you took that decision - do you have a timeframe in mind where you believe that we'll see such performance increases from multi-core processors in real games, as portrayed in 3DMark06?
We are already seeing games supporting dual cores, but when they will support it to the same extent as 3DMark06 does is a bit difficult to predict. I would presume during this year since dual cores are becoming more "mainstream" already. It is more or less up to the developers to decide how much resources they want to put into supporting dual cores. We see it (DC) as a great thing, and 3DMark06's CPU tests are proof that if the support is done efficiently, the gains in CPU utilization/performance can be pretty big. Really, it is up to the game developers to decide if they really want to utilize the possibilities DC's have to offer.

Cheers,

Nick

Nick[FM]

19-Jan-2006, 10:13

Thanks for taking the time to answer, but you still seem to have side-stepped my question

-EDIT- The emphasis in this question is placed on the WHY! the 7800 with AA gets no score

Surely it would make more 'sense' tht a score is given when AA is enabled. I say 'sense' because its definiton is meaningful in this context, and in the context of my eyes , the action your company has taken makes no sense to me. It seems illogical........
I recall that I already answered this question a couple of pages back? :???:

Anyway, as said, the point is that we require that the hardware needs to be able to run all tests which are available with default settings, no matter what settings & options are being used. I also want to remind that this is not an IHV specific thing! Any hardware with lack of some optional (from the benchmark options) feature support will work in the same way. Some of you seem to think this is an NVIDIA only thing, but it isn't.

Cheers,

Nick

mongoled

19-Jan-2006, 10:23

Kombatant

19-Jan-2006, 10:37

']We are already seeing games supporting dual cores, but when they will support it to the same extent as 3DMark06 does is a bit difficult to predict. I would presume during this year since dual cores are becoming more "mainstream" already. It is more or less up to the developers to decide how much resources they want to put into supporting dual cores. We see it (DC) as a great thing, and 3DMark06's CPU tests are proof that if the support is done efficiently, the gains in CPU utilization/performance can be pretty big. Really, it is up to the game developers to decide if they really want to utilize the possibilities DC's have to offer.

Cheers,

Nick My point is that, the way the benchmark is constructed, CPU performance is abstracted from GPU performance. Allow me to explain what I mean. SM2 and SM3/HDR scores are more or less independent from the type of CPU you use - which is great as far as consistency is concerned. And yes, multi-core processors are seeing more support and will see more support as they become more mainstream. But (you knew there was a 'but' coming, didn't you) what I am saying is that:

a) right now, as I have said in one of my previous posts, we have results that have no relation to actual game performance. A PentiumD 2.8GHz processor will give you a 30% bigger CPU score than an AthlonFX-57 - which sounds a bit, you know, out of this world. As you said, you have no way of knowing the level of multi-core improvements developers will bring to the table, so I was interested to find out how you guys decided on the amount of influence dual-core processors (even if they are weaker in performance, like the Pentium Ds) have in your benchmark.

b) CPU scores and GPU scores are distinct, as it is now. Meaning that, if you pair a lesser gfx card with a top-of-the-line dual core processor (call this Scenario A), you'll get a score which is bigger than someone with a top-of-the-line gfx card and a single-core processor (call this scenario B). In games, on the other hand, both the CPU and the GPU influence the framerates you get; aka you'll never see a scenario like the one I described above. From what I read, the overall 3DMark score is there to give you an indication of your system performance (although I thought PCMark was doing that already - please correct me if I am wrong) and not just games/gfx card performance. So, what I am trying to ask is, what do you believe the overall mark is portaying exactly? How are the scenarios I mentioned justified? How is scenario A better than scenario B in real-life usage, since 3DMark's tests are focused mainly on the gfx card?

Jawed

19-Jan-2006, 10:40

SM2 and SM3/HDR scores are more or less independent from the type of CPU you use - which is great as far as consistency is concerned.
We've got lots of results posted already in this thread that indicate 3 of the 4 graphics tests are CPU-limited.

Jawed

Kombatant

19-Jan-2006, 10:52

We've got lots of results posted already in this thread that indicate 3 of the 4 graphics tests are CPU-limited.

Jawed

Thanks for pointing it out man, I stand corrected then on that.

GraphiX

19-Jan-2006, 11:10

']I recall that I already answered this question a couple of pages back? :???:

Anyway, as said, the point is that we require that the hardware needs to be able to run all tests which are available with default settings, no matter what settings & options are being used. I also want to remind that this is not an IHV specific thing! Any hardware with lack of some optional (from the benchmark options) feature support will work in the same way. Some of you seem to think this is an NVIDIA only thing, but it isn't.

Cheers,

Nick

I have to agree with many people in this Forum, that giving NV no score is illogical. When it comes to AA and HDR, 7800 behaves like a SM2.0 card so it should get 2.0 scoring. It does all SM2.0 Tests with AA but is unable to run HDR/SM3.0 tests. It does render exactly the same in this scenario like a X850 card. The X850 gets a score but the 7800 does not... although the render work is absolutely comparable!

3DM06 AA Test:
X850 does run SM2.0 tests with AA -> SM2.0 score counting (because it cannot run SM3.0/HDR tests)
7800 does run SM2.0 Tests with AA -> No score counting (because it cannot run SM3.0/HDR tests)
7800 does run SM2.0 Tests with AA -> SM3.0 score counting (because it can run all tests)

Where is the logic???

In real games, GF7 owners have to do the same decission: Use HDR OR AA. It is a limitation of the hardware and this should be reflected in a score... this is what 3DM was meant for. A gamers benchmark that shows what the PC is capable of, reflected by a score.

The reason why this is important, is that 3DM is seen as a standard by many people. They just count the overall score. But there is no score for NV so nobody will take care of the important AA situation. NV can claim that it's not comparable and just use the 2.0 scoring.

Even just counting the single result scores does not help here, because ATI can not benefit in terms of scoring for their AA+HDR work. They have a score, NV is not "judged" by not supporting it. You leave it just neutral in this case althought the situation isn't...

Klaus

GraphiX

19-Jan-2006, 11:12

Typo... should be:

3DM06 AA Test:
X850 does run SM2.0 tests with AA -> SM2.0 score counting (because it cannot run SM3.0/HDR tests)
7800 does run SM2.0 Tests with AA -> No score counting (because it cannot run SM3.0/HDR tests)
1800 XT does run SM2.0 Tests with AA -> SM3.0 score counting (because it can run all tests)

Klaus

Nick[FM]

19-Jan-2006, 11:30

a) right now, as I have said in one of my previous posts, we have results that have no relation to actual game performance. A PentiumD 2.8GHz processor will give you a 30% bigger CPU score than an AthlonFX-57 - which sounds a bit, you know, out of this world. As you said, you have no way of knowing the level of multi-core improvements developers will bring to the table, so I was interested to find out how you guys decided on the amount of influence dual-core processors (even if they are weaker in performance, like the Pentium Ds) have in your benchmark.
3DMark06 is an ahead looking benchmark (as all our new 3DMark's are at the time when they are released). There are already games coming out which have some sort of dual core support, but it will take a while before game developers tune their engines to fully support dual core CPU's. I am sure that with time games will use DC's to the same extent as we do in 3DMark06. It is just a matter of time. I don't see any reason why not? I also have a hunch that when Xbox360 games (which fully utilize the multicore CPU) will be ported to PC's, we will see performance benefits of dual cores.

b) CPU scores and GPU scores are distinct, as it is now. Meaning that, if you pair a lesser gfx card with a top-of-the-line dual core processor (call this Scenario A), you'll get a score which is bigger than someone with a top-of-the-line gfx card and a single-core processor (call this scenario B). In games, on the other hand, both the CPU and the GPU influence the framerates you get; aka you'll never see a scenario like the one I described above. From what I read, the overall 3DMark score is there to give you an indication of your system performance (although I thought PCMark was doing that already - please correct me if I am wrong) and not just games/gfx card performance. So, what I am trying to ask is, what do you believe the overall mark is portaying exactly? How are the scenarios I mentioned justified? How is scenario A better than scenario B in real-life usage, since 3DMark's tests are focused mainly on the gfx card?
Look at it from this point of view. If you have a high-end DC CPU but a lower end GPU, which one will be the more obvious limiting factor in upcoming titles? Then again, if you have a single-core CPU and a high-end GPU, which one will become the limiting factor as soon as games utilize dual cores and use more complex AI & physics? The point in the CPU tests is to support the latest technology and show the benefits of it. Dual cores really are efficient CPU's if the application supports them well. In 3DMark06 I think we have done an excellent job in showing the real benefits of dual core technology. I can't see why games wouldn't do the same? If I was a game developer, I would certainly put resources into optimizing the CPU side of things in my game (to support dual cores efficiently). 3DMark06 is the proof of the benefits.

Cheers,

Nick

ANova

19-Jan-2006, 11:54

']DF24 works with FETCH4, but certainly has nothing to do with DFC (or do you mean PCF?). DF24 and FETCH4 go hand in hand, as D24X8 and PCF does. Dynamic Flow Control (DFC) has nothing to do with FETCH4, PCF, DST etc.

The X1800 supports fetch4, it does not support DF24, therefore it is not running fetch4 due to your requirements. The X1800 also supports rather good dynamic branching yet it does not seem to use this either, why? I understand that you believe 16 bit DST was not up to standards but it would have been nice to have it as an option.

We didn't decide against HDR+AA (a DX feature) since we support it in 3DMark06. If any hardware doesn't support that feature, then it .. simply doesn't. :???: DF24 is supported since it helps in shadow rendering on hardware that supports the feature (just like D24X8 on other cards). I am still not sure how this can be seen as "double standard". We now have multivendor hardware shadow mapping support, which we didn't in our previous 3DMark. What's wrong with us supporting more vendors' hardware shadow mapping?

The difference is that you are forcing standards supported by nvidia onto ATI but not the other way around. HDR+AA is not being forced onto the 7x00 series like D24X8 is on the X1800 (instead you just give the 7 series an N/A score), and since the X1800 does not support D24X8 it has to fall back to R32F which has an impact on bandwidth. So tell me, how is 24 bit to 32 bit a fair comparison? It's apples to oranges. So while the 7800s are running well on 24 bit with PCF, the X1800 is being compared to it on 32 bit without any fetch4 or DFC support. Thus it is not a relevant test to compare the two's capabilities imo.

Kombatant

19-Jan-2006, 12:01

']The point in the CPU tests is to support the latest technology and show the benefits of it. Dual cores really are efficient CPU's if the application supports them well. In 3DMark06 I think we have done an excellent job in showing the real benefits of dual core technology. I can't see why games wouldn't do the same? If I was a game developer, I would certainly put resources into optimizing the CPU side of things in my game (to support dual cores efficiently). 3DMark06 is the proof of the benefits.

Cheers,

Nick
So, all in all, the overall score is a proof of concept of what would happen if developers head that way when they develop their games? Or am I getting this all wrong?

Dave Baumann

19-Jan-2006, 12:07

The X1800 supports fetch4
X1800 is the odd one out of the X1000 line - it, and only it, does not support Fetch4.

inefficient

19-Jan-2006, 12:08

Hanners

19-Jan-2006, 12:14

The X1800 supports fetch4, it does not support DF24, therefore it is not running fetch4 due to your requirements.

The Radeon X1800 doesn't support Fetch4. The Radeon X1600 and X1300 (and indeed R580) do.

EDIT: Bah, Dave beat me to it. :p

dizietsma

19-Jan-2006, 12:24

Nick[FM]

19-Jan-2006, 12:29

So, all in all, the overall score is a proof of concept of what would happen if developers head that way when they develop their games? Or am I getting this all wrong?
The CPU tests are proof of what can be "milked out of" dual cores if it is done properly. That was my point. I thought we were discussing the CPU tests only?

The overall 3DMark score is calculated based on the CPU tests and the graphics tests. I think that everyone in this room also understand that what you see in 3DMark06 is something we don't really have in current games. 3DMark06 is an ahead looking benchmark, while still enabling today's hardware to run the benchmark and get comparable results. We are convinced that as soon as game developers have had enough time to optimize the CPU side of things in their games, we will see an increased performance with dual cores (some released games already proove this) which reflects with the score in 3DMark06.

Cheers,

Nick

Nick[FM]

19-Jan-2006, 12:32

Damn what kind of monster CPU do you need to get the CPU test running at over 1fps?

I feel like the the whole benchmark is really really high end compared to previous 3dmarks. I thought my PC was still in the high end (X2 4200 7800 GTX256) but not a single test was at watchable frame rates. The 3d tests are all bellow 15fps average and the CPU tests bellow 1fps.

It's a shame too. Because the graphics are really pretty. It's just not fun to watch at 15fps
The CPU tests won't ever run at high fps since they run at fixed frame based rendering. :wink: I suggest that you take a peek at the whitepaper for more in-depth info about the CPU tests, and why they are as they are: http://www.futuremark.com/companyinfo/3DMark06_Whitepaper_v1_0_2.pdf

Cheers,

Nick

overclocked

19-Jan-2006, 12:34

Nick about the dualcore issue raised here or complains.

Dont you think it would have been better to indeed include advanced Dualcore/cpu test BUT done it *in* the benchmark so too say. Ie as you have SM3/HDR tests dont you think it would have made more sense to add a "Dualcore" graphics test with either SM3 or SM2?
With a test like this you could hopefully have send a clearer message out why Dualcore is the future in gaming, and been the first to show it at a big scale, dont you agree?

I think that would have justified the whole benchmark as "the gamers benchmark" better.

Kombatant

19-Jan-2006, 12:40

']The CPU tests are proof of what can be "milked out of" dual cores if it is done properly. That was my point. I thought we were discussing the CPU tests only?

The overall 3DMark score is calculated based on the CPU tests and the graphics tests. I think that everyone in this room also understand that what you see in 3DMark06 is something we don't really have in current games. 3DMark06 is an ahead looking benchmark, while still enabling today's hardware to run the benchmark and get comparable results. We are convinced that as soon as game developers have had enough time to optimize the CPU side of things in their games, we will see an increased performance with dual cores (some released games already proove this) which reflects with the score in 3DMark06.

Cheers,

Nick
We are discussing CPU scores, but they influence the overall score a great deal, hence my question. In any case, I believe you answered what I wanted to know, thanks for taking the time to do so :)

Sunday

19-Jan-2006, 12:46

It's a shame too. Because the graphics are really pretty. It's just not fun to watch at 15fps
This is not meant to be funny benchmark, but it end up being one!

There’s no way that DualCore will impact any game performance in a way you trying to paint the picture! System with DualCore CPU and slower GFX will NOT play games faster then SingeCore CPU with faster GFX!

Since Nick has stated that there is no wrong in dong multi vendor path (even if that meant hurting one side implementing something that it doesn’t support, but relying on it heavily in some crucial tests), ‘cos hell “game developers are doing it” does it mean that is OK now for the vendors to do shader replacement trough drivers, or to do self made patches for 3DMark06 to improve performance (we’ve all prized Humus for his work on DOOM3)?

You’re so full of pride and joy Nick do you honestly cannot see what have you done to the 3DMark? You say there was no artistic reason to put Parallax Occlusion? Well god damn you’ve should create different art! There is a bunch of games that are designed right now in such way that this type of mapping is crucial for immersion, and what you giving us is some un-usable fireflies (this time two), one big dragon fish, skinned silly, AGAIN and one non playable Antarctic scene (good for some in game cinematic, and nothing more).

You wore either lazy, to create something truly new and usable, or you just don’t know how! Anyway you turn it up you’ve failed with 06!

Does anyone find it ironic that company that pushed for 2.0 SM for so long is now losing it dominance in that pair of tests, and “Power of Three” has comparable worst results in SM3 part (GTX 256 vs. R520XT situation)!

Mark

19-Jan-2006, 12:48

']3DMark06 is an ahead looking benchmark...I'm not so sure how far ahead it's looking. It doesn't include parallax mapping (which has already been included in games like FEAR) or a decent level of dynamic flow control (which is one of the more important SM3.0 features). If I would say anything, 3Dmark06 seems more like a modern benchmark, not a future one. In that case (and at the risk of soundling like I just graduated from the [H] school of thought) modern games would serve as a better guide to graphics performance.

Jawed

19-Jan-2006, 12:49

The X1800 supports fetch4, it does not support DF24, [...]

[...] and since the X1800 does not support D24X8 it has to fall back to R32F which has an impact on bandwidth. So tell me, how is 24 bit to 32 bit a fair comparison? It's apples to oranges. So while the 7800s are running well on 24 bit with PCF, the X1800 is being compared to it on 32 bit without any fetch4 or DFC support. Thus it is not a relevant test to compare the two's capabilities imo.
Not only is dynamic branching a victim of R520's seriously tardy arrival, in a fashion that prevented devs from gaining any quality time with it, R520 is somewhat of a runt in terms of features - RV530 is reasonable evidence, with its Fetch4 support.

I strongly suspect ATI will support the "NVidia" technique in the not too distant future (but not with X1900). From the XB360 GPU Overview:

Additionally, the back buffer can be resolved to the DXT3A_AS_1_1_1_1 format and the depth-stencil buffer can be resolved to the 24:8 fixed-point or 24:8 floating-point formats
I'm no expert, but that appears to mean that XB360 already supports D24X8. Though whether it also supports Fetch4, I'm not clear.

It would be nice if someone who truly understands this stuff clears up my misapprehensions...

In the end, even with X1900, ATI has seemingly left out a feature that it apparently knows is due for the big time (D24X8). It's up to ATI to demonstrate that the suggested 16-bit shadowing would have enough quality.

Jawed

inefficient

19-Jan-2006, 13:04

I'm not so sure how far ahead it's looking. It doesn't include parallax mapping (which has already been included in games like FEAR) or a decent level of dynamic flow control (which is one of the more important SM3.0 features). If I would say anything, 3Dmark06 seems more like a modern benchmark, not a future one. In that case (and at the risk of soundling like I just graduated from the [H] school of thought) modern games would serve as a better guide to graphics performance.

Since parallex mapping doesnt stress the GPU that much more than plain old normal mapping, for purely benchmark purposes I don't think it was nessasary.

It would been trivial for them to change their shader to support parallex mapping - but it would have just been a checkbox feature. There just aren't many opportunities to notice it in these tests.

GraphiX

19-Jan-2006, 13:19

Applying AA once gets you off the standard run, in a standard run AA is not applied in 06, instead they have gone for higher resolution. Once you start changing the standard settings at all then in my view it is better to individually run tests and give frame rates rather than a score which is artificial anyway by it's very nature.

If the standard test had AA then this would be more of an issue.

Up to 06, there was no difference made betwen the standard run or special feature runs with AF/AA or different resolution. You always got a score/meassurement for gaming performance. Many people used it with higher resolutions and AA/AF to set it to "HD Gaming" standards.

GF6/7 also deserves an overall score for the AA environment.

I did not test it, but I think GF7 also receives score if you force AF only!?

Klaus

Mariner

19-Jan-2006, 13:24

It strikes me that Futuremark's design decisions, however honestly conceived, penalise ATI's current high-end chip for not supporting Fetch4. On the other hand, the R5X0 series of chips are able to support AA + HDR, something no NVidia chip is able to do, yet these NVidia chips are not penalised in the same way.

We know that chips which don't support the required depth textures for the PS2.0 shadowing are forced into a relatively expensive shader workaround which is fine by me as Futuremark have decided 24-bit accuracy is required. On the other hand, if this is acceptable, why aren't chips which are not able to support AA + HDR also forced into a shader workaround?

I note that in this interview (http://www.bit-tech.net/bits/2005/07/11/nvidia_rsx_interview/3.html), David Kirk explains NVidia's decision to not support AA + HDR thus:

"It would be expensive for us to try and do it in hardware, and it wouldn't really make sense - it doesn't make sense, going into the future, for us to keep applying AA at the hardware level. What will happen is that as games are created for HDR, AA will be done in-engine according to the specification of the developer."

This being the case, surely it would have been logical for Futuremark to include a shader workaround for SM3.0 cards unable to support AA + HDR which generates the AA in-engine - the technique recommended by NVidia's Chief Scientist? Not being as technically minded as some others on this board, I'm assuming here that this is entirely feasible although it would undoubtedly add a great deal of complexity to the shaders.

This all seems a pretty reasonable idea to me. Any holes in my logic?

Neeyik

19-Jan-2006, 13:38

I suppose one could argue (in a very silly manner) that ATI and NVIDIA deserve what's happening to them in 06. For example, it doesn't look good stating that "It would be expensive for us to try and do it in hardware, and it wouldn't really make sense" when your nearest competitor has done it; it also doesn't look good offering FETCH4 on your mid- and low-end models but not your top end (although this might change with the R520) when your nearest competitor offers PCF across the whole range to which it is appropriate.

But like I said, all very silly...

Bouncing Zabaglione Bros.

19-Jan-2006, 13:40

Not only is dynamic branching a victim of R520's seriously tardy arrival, in a fashion that prevented devs from gaining any quality time with it, R520 is somewhat of a runt in terms of features - RV530 is reasonable evidence, with its Fetch4 support.

Hang on, wasn't dynamic branching demoed by Nvidia with their launch of NV40? Why is it that R520's late arrival stopped devs using dynamic branching, when Nvidia have been offering it for two product generations?

Dynamic branching should be in any forward looking benchmark, and would be a big advantage for ATI as they have spent a lot of transistors on it in R520/R580 - but for some reason Futuremark decided not to test this important aspect of newer SM3.0 hardware.

Razor1

19-Jan-2006, 13:50

Hang on, wasn't dynamic branching demoed by Nvidia with their launch of NV40? Why is it that R520's late arrival stopped devs using dynamic branching, when Nvidia have been offering it for two product generations?

Dynamic branching should be in any forward looking benchmark, and would be a big advantage for ATI as they have spent a lot of transistors on it in R520/R580 - but for some reason Futuremark decided not to test this important aspect of newer SM3.0 hardware.

Well for the most part other then terrain, at least in the Futuremark art assets that can use dynamic branching to improve visuals by using parrallex occlusion mapping or similiar bump mapping ( and this is a minimal improvement in quality because as explained earlier parrallex amount will be too small to be appreciated), don't see where else it would be very useful.

Nick where is dynamic branching being used right now? From what I've seen in the benchmark I'm guessing for the multiple light sources, and possibly the water in game 3.

inefficient

19-Jan-2006, 13:51

Dave Baumann

19-Jan-2006, 13:52

it also doesn't look good offering FETCH4 on your mid- and low-end models but not your top end (although this might change with the R520) when your nearest competitor offers PCF across the whole range to which it is appropriate.
AFAIK there are IP and usage issues related to PCF, though. IIRC the PCF operation is actually an SGI patent which NVIDIA inherited back when SGI transferred a bunch of technology and engineers to NVIDIA and NVIDIA implemented it after that. The actual PCF operation is atypical for any kind of previously documented texture operation in DX, but because it was included in the XBOX, and documented, developers started using it and then it also happended to "work" under DX. MS do openly talk about it now, but when the capability is undocumented and support kind of happens through osmosis it going to take time for a competitor to actually get in hardware (and do it in such a fashion that doesn't step on existing IP).

Dave Baumann

19-Jan-2006, 13:55

Well for the most part other then terrain, at least in the Futuremark art assets that can use dynamic branching to improve visuals by using parrallex occlusion mapping or similiar bump mapping ( and this is a minimal improvement in quality because as explained earlier parrallex amount will be too small to be appreciated), don't see where else it would be very useful.
With the jittered sampling from the shadowmaps I believe dynamic branching could be used to decide whether its in, out or on the edge of a shadow.

Razor1

19-Jan-2006, 14:00

With the jittered sampling from the shadowmaps I believe dynamic branching could be used to decide whether its in, out or on the edge of a shadow.

Very true

Nick[FM]

19-Jan-2006, 14:14

Since Nick has stated that there is no wrong in dong multi vendor path (even if that meant hurting one side implementing something that it doesn’t support, but relying on it heavily in some crucial tests), ‘cos hell “game developers are doing it” does it mean that is OK now for the vendors to do shader replacement trough drivers, or to do self made patches for 3DMark06 to improve performance (we’ve all prized Humus for his work on DOOM3)?
I have said that doing multivendor paths (as in totally different paths) in games is sure ok, but in 3DMark06 (and all previous) we use only one path with certain fallbacks to enable more hardware to be able to run the tests. We will not allow shader replacements or such. The driver optimization guidelines we set a couple of years ago still are in full effect.

You’re so full of pride and joy Nick do you honestly cannot see what have you done to the 3DMark? You say there was no artistic reason to put Parallax Occlusion? Well god damn you’ve should create different art! There is a bunch of games that are designed right now in such way that this type of mapping is crucial for immersion, and what you giving us is some un-usable fireflies (this time two), one big dragon fish, skinned silly, AGAIN and one non playable Antarctic scene (good for some in game cinematic, and nothing more).
You think we should create our scenes just to promote one single effect that has been used in one (are there more released games (not engines/tech demos!) which use POM?) game? :???: Certainly not! Personal likings is one thing I won't go into any more than this. If you don't like the artwork, it is ok, but forcing some POM into them wouldn't have changed them visually much at all. It is down to the fact that not all effects are feasible to use just because they exist.

You wore either lazy, to create something truly new and usable, or you just don’t know how! Anyway you turn it up you’ve failed with 06!
If that's your personal view on how things work and are, then so be it. I can't convince you to like what we have created if you simply don't like it.

Anyhow, thanks for the feedback!

Cheers,

Nick

digitalwanderer

19-Jan-2006, 14:22

Where is the cow in this one?

Nick[FM]

19-Jan-2006, 14:24

I'm not so sure how far ahead it's looking. It doesn't include parallax mapping (which has already been included in games like FEAR) or a decent level of dynamic flow control (which is one of the more important SM3.0 features). If I would say anything, 3Dmark06 seems more like a modern benchmark, not a future one. In that case (and at the risk of soundling like I just graduated from the [H] school of thought) modern games would serve as a better guide to graphics performance.
Tell me what makes Parallax Mapping such an effect that it must be used in all games/apps? I fail to understand why this one effect is such a big issue. You name one game which supports it (FEAR), but I don't see people bashing other game benchmarks or games for not haing POM. Just doesn't make sense.

Sure, future engines may support POM (even we in the future if we find use for it), but we didn't find use for it in 3DMark. As I said, it is simply one effect amongst many others.

Cheers,

Nick

Nick[FM]

19-Jan-2006, 14:25

Where is the cow in this one?
:wink: You need to dig (no pun intended) deeper...

digitalwanderer

19-Jan-2006, 14:28

']:wink: You need to dig (no pun intended) deeper...
Well, can I at least find out if I need SM3.0 to see it? If so I'll have to borrow my son's rig for a bit.... :oops:

Nick[FM]

19-Jan-2006, 14:32

Well, can I at least find out if I need SM3.0 to see it? If so I'll have to borrow my son's rig for a bit.... :oops:
Nope. Doesn't require SM3.0. :wink:

Nick[FM]

19-Jan-2006, 14:37

Guys,

while I'd love to hang in here 24/7, I need to continue to work. We will release a Technical FAQ soon where we have answered the most frequent asked questions about 3DMark06. It seems that too many simply load up the last page of this thread and I keep repeating myself when answering questions.

I'll keep you guys posted as soon as we have the FAQ ready and released!

If anyone has anything important to report (something that I haven't answered here, or available in the 3DMark06 Whitepaper) feel free to email me at, nick at futuremark.com. Please keep in mind that though we are very open to discuss our products and are willing to help everyone understand how the benchmarks work, there are certain limitations to what info we can share. I hope you respect that since we don't want to step on anyone's toes.

Cheers,

Nick

Jawed

19-Jan-2006, 14:37

Hang on, wasn't dynamic branching demoed by Nvidia with their launch of NV40? Why is it that R520's late arrival stopped devs using dynamic branching, when Nvidia have been offering it for two product generations?
In the same way that vertex texturing is not a core function within 3DMk06 - it performs like a dog on NVidia hardware - dynamic branching is a complete waste of time on NVidia hardware in any form other than as a shortcut to static branching (or in other words it's prolly being used instead of having multiple, similar, shaders).

3DMk06 clearly doesn't use per-pixel dynamic branching in any meaningful fashion, because if it did the NVidia hardware would be down the toilet.

This is why I had quite a lot of confidence that we wouldn't see any significant use of dynamic branching - because FM prefers to "even things up" (though in the 05 and 06 that process favours NVidia heavily, hmm, strange huh?).

Dynamic branching should be in any forward looking benchmark, and would be a big advantage for ATI as they have spent a lot of transistors on it in R520/R580 - but for some reason Futuremark decided not to test this important aspect of newer SM3.0 hardware.
I agree - which is why I think this is the final nail in the coffin for FM, they've proven once and for all that the 3DMk series is irrelevant as a benchmark. Hoisted themselves by their own petard. And very prettily done, it has to be said.

Jawed

digitalwanderer

19-Jan-2006, 14:40

']Nope. Doesn't require SM3.0. :wink:
Thanks! Now I have a project for the day....

(Yes, I have no life. :( )

Mark

19-Jan-2006, 14:42

']...

Sure, future engines may support POM (even we in the future if we find use for it), but we didn't find use for it in 3DMark. As I said, it is simply one effect amongst many others.

Cheers,

NickWell what is 3DMark06 then, is it future looking or not? You say in one post that it is, but in another that it won't sport features that aren't in (enough) current games?

Neeyik

19-Jan-2006, 14:45

3DMk06 clearly doesn't use per-pixel dynamic branching in any meaningful fashion, because if it did the NVidia hardware would be down the toilet.

Surely a game's performance shouldn't hinge on the use of dynamic branching though? What dev house would use it in such a manner to totally bork a huge amount of the user hardware base?

AFAIK there are IP and usage issues related to PCF, though.

Oh I wasn't suggesting that ATI should be using PCF but rather that they implement FETCH4 across either the entire X1000 series or, at the very least, on their top-end model rather than just the mid- or low-end.

Hanners

19-Jan-2006, 14:46

In the same way that vertex texturing is not a core function within 3DMk06 - it performs like a dog on NVidia hardware - dynamic branching is a complete waste of time on NVidia hardware in any form other than as a shortcut to static branching (or in other words it's prolly being used instead of having multiple, similar, shaders).

With that in mind though, is dynamic branching likely to be used to any great extent in any future titles currently in development? In a sense, it's a bigger disappointment (to me anyway) that dynamic branching wasn't included in some way as an additional feature test (in the same sort of way vertex texture fetch was utilised in the shader particles test), rather than as part of the main graphics tests.

N00b

19-Jan-2006, 14:50

Well what is 3DMark then, is it future looking or not?Can you tell me where in your opinion in 3DMark06 POM would improve image quality noticable? Which effect should have been dropped in favour of POM? (I assume that futuremark does not have unlimited resources and fixed shedule, so they probably would have had to drop something else in order to support POM)

Joe DeFuria

19-Jan-2006, 15:03

With that in mind though, is dynamic branching likely to be used to any great extent in any future titles currently in development? In a sense, it's a bigger disappointment (to me anyway) that dynamic branching wasn't included in some way as an additional feature test (in the same sort of way vertex texture fetch was utilised in the shader particles test), rather than as part of the main graphics tests.

Exactly...even if it didn't make it into the game tests in a meaningful way, there should at LEAST be a feature test that is basically a "branching test", similar to the vertex fetch test. I mean really...BOTH vendors support branching...and it is the single stand-out feature of PS 3....

Joe DeFuria

19-Jan-2006, 15:05

Can you tell me where in your opinion in 3DMark06 POM would improve image quality noticable?

That is a separate question actually. It seems to me that both SM3.0 tests are very similar in terms of quality and features that they stress...why is this? Why, if you're going to have two test, would you not make them very different in terms of what they are stressing?

Neeyik

19-Jan-2006, 15:06

The theoretical/feature tests are always very low down, if not the last, on the list of FM's test priorities for 3DMark. God only knows how many times Worm has heard me twine about it...

Mark

19-Jan-2006, 15:09

Can you tell me where in your opinion in 3DMark06 POM would improve image quality noticable? Which effect should have been dropped in favour of POM? (I assume that futuremark does not have unlimited resources and fixed shedule, so they probably would have had to drop something else in order to support POM)It isn't about improving image quality, this isn't a game or a eye-candy demo, it's supposed to be a benchmark. If one of your "tests" doesn't have a suitable place for POM, then make a test that does. Even if was a simple feature test, one that doesn't factor into the final score (like the perlin noise or shader particles tests) which stresses nothing but the parallax mapping feature would have been fine. Futuremark chose instead to completely ignore it even though it's probably going to be an often used feature in many upcoming games.

Bouncing Zabaglione Bros.

19-Jan-2006, 15:10

3DMk06 clearly doesn't use per-pixel dynamic branching in any meaningful fashion, because if it did the NVidia hardware would be down the toilet.

This is why I had quite a lot of confidence that we wouldn't see any significant use of dynamic branching - because FM prefers to "even things up" (though in the 05 and 06 that process favours NVidia heavily, hmm, strange huh?).

I agree - which is why I think this is the final nail in the coffin for FM, they've proven once and for all that the 3DMk series is irrelevant as a benchmark. Hoisted themselves by their own petard. And very prettily done, it has to be said.

Jawed
Which is exactly what I am alluding to. Futuremark made some questionable choices in 3D Mark 06, ones that benefit one IHV and actively hamstring the other. Strange eh? I guess they are still worried about what would happen if one company took their green-backs away, started "leaking" internal documents about how terrible Futuremark are, and threatened them with a bunch of lawyers.

Edit: I just think they gone to far in placating Nvidia by not including one of the most forward looking and lauded SM3.0 techniques (ie dynamic branching), even though Nvidia is the company that has been pushing this the longest of all! Leaves a bad taste in the mouth and asking what value is 3DMark06 if it doesn't even include dynamic branching, one of the keystones of SM3.0?

dizietsma

19-Jan-2006, 15:11

it's a bigger disappointment (to me anyway) that dynamic branching wasn't included in some way as an additional feature test (in the same sort of way vertex texture fetch was utilised in the shader particles test), rather than as part of the main graphics tests.

I agree with this statement, perhaps the single feature tests for next time need to be padded out so that as many techniques and effects can be measured, this at least will take away the claim that some features of a card are favoured or not favoured.

I would imagine ( perhaps naively) that lots of small tests are easier to chunk out than those impressive games so you can cover more effects. Futuremark is not the worlds biggest company of course so obviously they do have a limited amount of developement resources and manpower.

You still need a scored "game" though at least to get the crowds in .

NocturnDragon

19-Jan-2006, 15:12

It strikes me that Futuremark's design decisions, however honestly conceived, penalise ATI's current high-end chip for not supporting Fetch4. On the other hand, the R5X0 series of chips are able to support AA + HDR, something no NVidia chip is able to do, yet these NVidia chips are not penalised in the same way.

We know that chips which don't support the required depth textures for the PS2.0 shadowing are forced into a relatively expensive shader workaround which is fine by me as Futuremark have decided 24-bit accuracy is required. On the other hand, if this is acceptable, why aren't chips which are not able to support AA + HDR also forced into a shader workaround?

I note that in this interview (http://www.bit-tech.net/bits/2005/07/11/nvidia_rsx_interview/3.html), David Kirk explains NVidia's decision to not support AA + HDR thus:

I really liked Mariner post, and I think it got overlooked.

ATI decided not to implement FP16 texture filtering, saying that in the future, developers will want to use different kernel from the default box. So what Futuremark is doing, is using this "very efficient" (we don't have any information on what "very efficient" means.
Which is good, that's what ATI was looking for, let the developed do it with PS.
(One could argue that FM could have used a custom filter, to rais quality and implement it with PS on all hardware, as it's probably what future developers will do, but it's still a personal point of view)

Now again as all of you remember, Nvidia told anyone that multisaple with HDR was useless, because developers in the future will do the AA inside PSs.
Then why didn't FM do that with a PS on hardware not supporting it natively?
Why not do software SSAA?
Wouldn't it be "very efficient"? Too bad for the hardware not supporting it!
Would show that Nvidia cards (and any other cards that don't support it) are not future proof? Surely!
(But protecting some hardware is not what a "objective" benchmark should do anyway!)

It surely is just a coincidence that all the weird decision made go against ATI.. But still!

Jawed

19-Jan-2006, 15:15

Surely a game's performance shouldn't hinge on the use of dynamic branching though? What dev house would use it in such a manner to totally bork a huge amount of the user hardware base?
Shader effects in games normally come with options to turn them on or off, perhaps with a default based on the shader model of the hardware.

So, it's easy to see that in future games sophisticated eye-candy will make use of optional shaders, some of which will be heavily dependent on dynamic branching - with some pixels running shaders for 150 or 200+ cycles while most other pixels stick to an upper limit of about 100, say. If your future SM4 card is at the low-end, then you prolly won't be turning on such eye-candy options :razz:

So, in the end, a future game's performance hinges on the max eye-candy options the user sets. If some of those options are predicated on having viable dynamic branching performance, then the game's performance hinges on the capability of the GPU - exactly as it does today. FEAR can be played on an FX5600 by turning every option off or to the minimum. Admittedly it looks shite, but I've seen it run.

In a so-called future-aware benchmark, you'd expect a key feature of SM3 and all suceeding versions to play a significant role. Well, I would expect so. FM's only viable excuse is the tardiness of R520. I call that unimaginative.

Jawed

N00b

19-Jan-2006, 15:18

That is a separate question actually. It seems to me that both SM3.0 tests are very similar in terms of quality and features that they stress...why is this? Why, if you're going to have two test, would you not make them very different in terms of what they are stressing?If it had been your decision, how would the second SM3.0 test look like (features, quality, settings,...)?

digitalwanderer

19-Jan-2006, 15:20

And on a lighter note, a post from Hanners over at EB (http://www.elitebastards.com/forum/viewtopic.php?t=13419) about NGO HQ's latest article on 3dm2k6 entitled, "Is 3DMark Really the “Gamers Benchmark” ?" (http://www.ngohq.com/home.php?page=Articles&go=read&arc_id=66):

The program’s authors, Futuremark, refer to it as “The gamers’ benchmark”. However, there is a dark side to this innocuous little app. 3DMark uses various game tests which help to decide your score, and all of these game tests are based on the DirectX9 platform. The question remaining on the silent majority’s lips is “Where are the OpenGL game tests?” In response Futuremark have claimed that OpenGL is not deemed popular enough to warrant incorporation into the program. Is that really so?

Does the name “Counter-Strike” ring a bell? Most people have heard of this game (in fact there are pygmy tribes in Central Africa who have never seen a PC that are familiar with it!). And I’m not talking about Counter-Strike: Source. Even today the original Counter-Strike remains very popular among the gamers. What about Quake 3, Quake 4, Doom 3, Enemy Territory, The Chronicles of Riddick and Call of Duty? All of these big titles are OpenGL based.
http://69.93.88.162/forum/images/smiles/rofl.gifhttp://69.93.88.162/forum/images/smiles/rofl.gifhttp://69.93.88.162/forum/images/smiles/rofl.gif

I'm sorry, but this one is slaying me! :lol:

Joe DeFuria

19-Jan-2006, 15:23

If it had been your decision, how would the second SM3.0 test look like (features, quality, settings,...)?

For example, one with likely with less emphasis on shadowing and HDR (which the first test covered), and more emphasis on implementing other features that are in the emerging stage in games. Some parallax mapping implementation, other skin shaders...

Cowboy X

19-Jan-2006, 15:26

Sadly the responses to the lack of dynamic branching and HDR with AA scores on Nvidia cards leave much to be desired and I have to give this a big thumbs down . I am not even mentioning the .2 fps or so that my 3200+ at 2503 Mhz gets in the CPU test .

Illogical decisions always result in a very very fishy smell ................ why can't FM seem to understand this ?

N00b

19-Jan-2006, 15:31

It isn't about improving image quality, this isn't a game or a eye-candy demo, it's supposed to be a benchmark. If one of your "tests" doesn't have a suitable place for POM, then make a test that does. Even if was a simple feature test, one that doesn't factor into the final score (like the perlin noise or shader particles tests) which stresses nothing but the parallax mapping feature would have been fine. Futuremark chose instead to completely ignore it even though it's probably going to be an often used feature in many upcoming games.I disagree that it's not about improving image quality. Including a feature in a (game) test only for the sake of it, if it does not add anything to the image quality or overall experience, does not seem right to me.
But I agree that Futuremark should have included one or more feature tests.
Yet I think that it is all about time and resources. DX10 is around the corner they probably already work on 3DMarkX. So pushing the schedule of 3DM06 in order to include more tests most certainly was not an option for futuremark. I agree with some posters here who stated that 3DM06 would have looked different it the R520 had been out in time.

NocturnDragon

19-Jan-2006, 15:34

A64 X2@2100 9800pro 370core/340 mem
3D Mark Score= 606
SM2.0 = 281
Cpu Score = 1589.

Only watched a bit of it but it was a slideshow.

Poor old arthritic 9800 was getting flogged.
My result:
3DMark Score: 996
SM2.0 Score: 480
CPU Score: 1006

GFX: RADEON 9800 XT @434MHz/370MHz
CPU Intel Pentium M 1.60GHz @ 2.4GHz

Looks like my little PM is still kicking (counting that the DDR400 memory is underclocked!)

Richteralan

19-Jan-2006, 15:39

Ok about CPU score.
I understand CPU testing is essential.
But I don't understand why CPU scores can influence the final 3DMark score in such a degree.
Can a dual-core CPU gives you better shader performance?
Can a dual-core CPU gives you HDR+AA?
Can a dual-core CPU gives you SM3.0?

While dual-core CPU IS influential in real game situations, but it won't be much influential as GPU does.
So just imagine one with fast CPU+6600GT get a 3DM06 scores higher than one with slower CPU+6800GT, I really can't and won't find a game is CPU limited in this way, now plus future.

N00b

19-Jan-2006, 15:39

For example, one with likely with less emphasis on shadowing and HDR (which the first test covered), and more emphasis on implementing other features that are in the emerging stage in games. Some parallax mapping implementation, other skin shaders...That actually a bit vague. Can you go into detail a bit? Like describe a complete game scenario.

Cowboy X

19-Jan-2006, 15:41

This is where I am with vcard at stock :

3DMark Score 1735 3DMarks
SM 2.0 Score 883 Marks
SM 3.0 Score N/A
CPU Score 948 Marks

http://service.futuremark.com/compare?3dm06=19050

Joe DeFuria

19-Jan-2006, 15:45

That actually a bit vague. Can you go into detail a bit? Like describe a complete game scenario.

Of course it's vague...I'm not a 3D programmer or artist. ;)

How about a hand-to-hand compbat scene where the emphasis is more on the quality of charaters (skinning technoloy), as opposed to the environment?

All I'm saying is, if you're going to make two tests...they should not be as similar as they are in terms of what you are showcasing. HDR and shadowing are great, and an advanced and computationally heavy version absolutely SHOULD be in at least one of the tests...but not both when there are other things to explore.

Richteralan

19-Jan-2006, 15:47

N00b

19-Jan-2006, 16:09

Of course it's vague...I'm not a 3D programmer or artist. ;)

How about a hand-to-hand compbat scene where the emphasis is more on the quality of charaters (skinning technoloy), as opposed to the environment?

All I'm saying is, if you're going to make two tests...they should not be as similar as they are in terms of what you are showcasing. HDR and shadowing are great, and an advanced and computationally heavy version absolutely SHOULD be in at least one of the tests...but not both when there are other things to explore.I think that shadowing is here to stay and the significance it has in 3DM06 is absolutely justified (even if the implementation hurts one party more than the other). Not so sure about HDR, though. But I agree with you on the advanced and computationally heavy part.
So here is what I would have liked: A game scenario where a knight fights a dragon in a castle ruin.
Featuring: Subsurface scattering on the knight's and dragon's skin.
Displacement mapping (castle bricks are great for this).
Procedural textures for things like wooden furniture and dragon skin.
A test like this probably would run at max. 5 FPS on the most fastest boards out, but would be a nice look ahead into the future.

Hubert

19-Jan-2006, 16:15

rwolf

19-Jan-2006, 16:21

']I have said that doing multivendor paths (as in totally different paths) in games is sure ok, but in 3DMark06 (and all previous) we use only one path with certain fallbacks to enable more hardware to be able to run the tests. We will not allow shader replacements or such. The driver optimization guidelines we set a couple of years ago still are in full effect.

You think we should create our scenes just to promote one single effect that has been used in one (are there more released games (not engines/tech demos!) which use POM?) game? :???: Certainly not! Personal likings is one thing I won't go into any more than this. If you don't like the artwork, it is ok, but forcing some POM into them wouldn't have changed them visually much at all. It is down to the fact that not all effects are feasible to use just because they exist.

If that's your personal view on how things work and are, then so be it. I can't convince you to like what we have created if you simply don't like it.

Anyhow, thanks for the feedback!

Cheers,

Nick

So why can't we run all cards using the fall back method so we can compare apples to apples. If the results are the same then there is no issue.

Hanners

19-Jan-2006, 16:29

So why can't we run all cards using the fall back method so we can compare apples to apples. If the results are the same then there is no issue.

In the case of both PCF/Fetch4 and floating-point filtering, you can.

boltneck

19-Jan-2006, 16:31

The Radeon X1800 doesn't support Fetch4. The Radeon X1600 and X1300 (and indeed R580) do.

EDIT: Bah, Dave beat me to it. :p

So you are telling me that I just spend a ton of money on a card that is missing basic features of the new generation???

Can you explain why this is acceptable?

How big an impact wiill not having fetch 4 affect me in the Future?

Jawed

19-Jan-2006, 16:39

In the case of both PCF/Fetch4 and floating-point filtering, you can.
It's not the fetching or filtering that's causing the significant performance difference, though (as far as I can tell).

The problem is that it's taking the ATI cards 3x bandwidth to create the shadow maps compared with the NVidia cards.

Whereas the 16-bit fallback that ATI are supposedly suggesting would work equivalently for both IHVs.

Hope I've got that right.

Jawed

Cowboy X

19-Jan-2006, 16:40

So you are telling me that I just spend a ton of money on a card that is missing basic features of the new generation???

Can you explain why this is acceptable?

How big an impact wiill not having fetch 4 affect me in the Future?

It means that your 3Dmark score will be teh suxx0rs .

Lol I couldn't resist .

It is all up to the individual developer what techniques and features to use . Both sides leave out certain features or abilities in their current gen products . Should a particular dev make heavy use of a feature that your card maker didn't implement then look for a performance hit , or a missing quality checkbox in the game . Largely though developers will avoid such and try to find common ground so from a less technical standpoint I doubt it would affect you much in real games ................... but i am certain that there are several members here who can give long dissertations about the usefulness or uselessness of Fetch 4 etc .

Hubert

19-Jan-2006, 16:42

So you are telling me that I just spend a ton of money on a card that is missing basic features of the new generation???

Can you explain why this is acceptable?

How big an impact wiill not having fetch 4 affect me in the Future?

My question, too. That's what about my pm towards Jawed was asking. I still could cancel the card ... :) But I think this is off topic here.

boltneck

19-Jan-2006, 16:43

After reading through these posts it does seem to be a reasonable conclusion that for some reason, behind the scenes, Futuremark partnered with Nvidia on this project. It does not look like "coincidence" at all. Specific choices where made over several months that had specific impacts on different vendors.

Choices were made that frankly favor Nvidia at this time. How can having the best SM3 processing hardware not be allowed to have an impact in a supposed "future" looking benchmark?

Once the X1900XT gets released things are going to change in a hurry though. Looks like this one might put a beat down on the competition.

Hanners

19-Jan-2006, 16:46

It's not the fetching or filtering that's causing the significant performance difference, though (as far as I can tell).

The problem is that it's taking the ATI cards 3x bandwidth to create the shadow maps compared with the NVidia cards.

With hardware shadow mapping disabled in 3DMark06, I believe it also reverts to using R32F depth maps on all hardware, as well as removing the use of either PCF or Fetch4.

Cowboy X

19-Jan-2006, 16:48

digitalwanderer

19-Jan-2006, 16:51

I am not so sure about the 1900Xt suddenly ruling all in this benchmark unless it is far far away better than the 7800GTX 512 .
Then you can now be sure. :cool:

Monday gonna be interesting... ;)

Cowboy X

19-Jan-2006, 16:54

Then you can now be sure. :cool:

Monday gonna be interesting... ;)

Are those snippets of soon to be revealed common knowledge or , just hope :)

digitalwanderer

19-Jan-2006, 16:58

Are those snippets of soon to be revealed common knowledge or , just hope :)
Yes. http://mastabeta.com/forum/images/smilies/yep.gif

Mariner

19-Jan-2006, 17:01

With hardware shadow mapping disabled in 3DMark06, I believe it also reverts to using R32F depth maps on all hardware, as well as removing the use of either PCF or Fetch4.

This being the case I suppose that (in theory at least) we can roughly extrapolate the performance hit on the X1800 by seeing what the hit is on NVidia cards when hardware shadow mapping is disabled. I'm pretty sure that I saw some NVidia benchmarks with/without hardware shadowing on EB's website. Unfortunately, their site is borked at the moment so I can't check it this!

Hanners

19-Jan-2006, 17:04

This being the case I suppose that (in theory at least) we can roughly extrapolate the performance hit on the X1800 by seeing what the hit is on NVidia cards when hardware shadow mapping is disabled. I'm pretty sure that I saw some NVidia benchmarks with/without hardware shadowing on EB's website. Unfortunately, their site is borked at the moment so I can't check it this!

Elite Bastards is working fine for me, although it is playing its bi-weekly game of "whore's drawers" because I dared to post some content. :roll: :wink:

Anyhow, from looking at the charts I can tell you that a 7800GT (on an Athlon64 3500+ system with 1GB of RAM) is losing just under 300 points in the Shader Model 2.0 tests. This equates to about 2 frames per second lost each in graphics tests one and two.

digitalwanderer

19-Jan-2006, 17:07

Elite Bastards is working fine for me, although it is playing its bi-weekly game of "whore's drawers" because I dared to post some content. :roll: :wink:
DAMN JOU!!! DAMN JOU TO HELL!!!!!!

I keep telling you, if you keep it up like this the place will be getting all touristy. :roll:

;)

Unknown Soldier

19-Jan-2006, 17:08

X800Pro @ Bios'd XT/PE Speeds 520/560
1Gig DDR 400
7NNXP Gigabyte Mobo
AMD 3200+ @ 2200 (def.)

First Pic CAT5.13
http://img458.imageshack.us/img458/8403/3dmark06cat5131as.th.jpg (http://img458.imageshack.us/my.php?image=3dmark06cat5131as.jpg)

Second Pic Cat6.1
http://img458.imageshack.us/img458/9034/3dmark06cat614ag.th.jpg (http://img458.imageshack.us/my.php?image=3dmark06cat614ag.jpg)

Jawed

19-Jan-2006, 17:10

With hardware shadow mapping disabled in 3DMark06, I believe it also reverts to using R32F depth maps on all hardware, as well as removing the use of either PCF or Fetch4.
Ah, ok. I think I'm getting more confused now. I shoulda stuck with my attitude from yesterday: hold my tongue until someone writes an in-depth article on the intricacies of 3DMk06 shadowing.

Jawed

Moloch

19-Jan-2006, 17:11

']And what's the difference if we would have physics & AI in the graphics tests, eliminating pure GPU benchmarking? :???: We now have 4 graphics tests and 2 CPU tests (which do use lots of physics, AI etc.), and we are able to output a 3DMark score based on your systems' gaming performance, and sub scores for pure graphics & CPU benchmarking. I'm sorry, but I don't see the logic in your post since that's what we did. Only difference is that we separated those two aspects (CPU & GPU) in order for people to do more in-depth benchmarking.

Cheers,

Nick
Exactly.
You took the graphics card completely out of the equation, where as a regular game test (1280x1024) that just hit the CPU much harder than the other tests would be more realistic.
Well actually since some of the tests are already heavily cpu limited why didn't you make them hit the GPU more?
It seems the harder you guys try to create a good benchmarking tool the harder you guys fall.

Razor1

19-Jan-2006, 17:18

It's not the fetching or filtering that's causing the significant performance difference, though (as far as I can tell).

The problem is that it's taking the ATI cards 3x bandwidth to create the shadow maps compared with the NVidia cards.

Whereas the 16-bit fallback that ATI are supposedly suggesting would work equivalently for both IHVs.

Hope I've got that right.

Jawed

Doesn't even look like that, that means the x1600 would perform noticable worse then other cards in its catagory while in the sm 3.0 path. Since from what Nick has said these features aren't aviable in the sm 3.0 tests? Sure doesn't seem like its getting hurt much. Actually its doing very very well.

Razor1

19-Jan-2006, 17:21

Exactly.
You took the graphics card completely out of the equation, where as a regular game test (1280x1024) that just hit the CPU much harder than the other tests would be more realistic.
It seems the harder you guys try to create a good benchmarking tool the harder you guys fall.

Well they didn't fail, its a synthetic test, its good for analyse of weak and strong points of gpu tech, for true benchmark tests games are the only way to go. Lets see there was only one benchmark out there and it was the game FEAR, would it be a great benchmarking tool? Well yeah for that one game, but Doom 3 will be different, Far Cry would be different, etc, etc.

Jawed

19-Jan-2006, 17:52

Doesn't even look like that, that means the x1600 would perform noticable worse then other cards in its catagory while in the sm 3.0 path. Since from what Nick has said these features aren't aviable in the sm 3.0 tests? Sure doesn't seem like its getting hurt much. Actually its doing very very well.
I'll be honest, I've reached a sort of indefensible muddle as to what 3DMk06 is doing, when.

I'm just gonna wait now till some in-depth analysis. Should anyone be bothered.

Jawed

Chalnoth

19-Jan-2006, 17:56

The problem is that it's taking the ATI cards 3x bandwidth to create the shadow maps compared with the NVidia cards.
Why would it require 3 times the bandwidth?

Whereas the 16-bit fallback that ATI are supposedly suggesting would work equivalently for both IHVs.
Maybe. Or maybe there would be graphical anomalies.

Jawed

19-Jan-2006, 17:58

Why would it require 3 times the bandwidth?
Ask ATI - that's what they're saying :wink:

Maybe. Or maybe there would be graphical anomalies.
If you read Nick's qualification of why 16-bits are not enough, you'll realise that his answer is not satisfactory. :wink:

The winks, by the way, are because you apparently haven't read the thread.

Jawed

boltneck

19-Jan-2006, 18:06

I am not so sure about the 1900Xt suddenly ruling all in this benchmark unless it is far far away better than the 7800GTX 512 .

My reason is its 48 Shader units and it has fetch-4.

boltneck

19-Jan-2006, 18:15

The more i look at this the more just wrong it seems.

http://www.anandtech.com/video/showdoc.aspx?i=2675&p=3

There is just no way, that the X1800XT should be slower than a GTX in a benchmark that is supposedly heavy on SM2 and SM3.

To me this is as blatantly fixed as you can get. Its on the same level as the Troy palumalu INT in the Steelers/Colts game that was suddenly a "incomplete pass".

Whats the point in going out of your way to make a Card that is completely geared towards Shader rendering if companies because of money or whatever just find ways to work around it so you still lose?

The X1800Xt should be within a few % of the GTX 512. Anything that shows any different is not worth the hard drive space its saved on.

I am now officially on the futuremark needs to go the way of the dodo bandwaggon. Tehy need to dissapear of the scene and now before they do any more damage.

Hubert

19-Jan-2006, 18:16

Are R5xx and G70 PS limited in 3DMark 2006 ?

Razor1

19-Jan-2006, 18:29

Are R5xx and G70 PS limited in 3DMark 2006 ?

Thats what confusing :grin: , it looks that way from benchmarks, but then when CPU's are overclocked there are gains in 3dmark. But then again the CPU we have seen that was overclocked was a 3.2 would like to see a fx-57 overclock would be able to get a better understanding of where the bottlenecks are.

dizietsma

19-Jan-2006, 18:30

We've got lots of results posted already in this thread that indicate 3 of the 4 graphics tests are CPU-limited.

Jawed

and

Ok about CPU score.
I understand CPU testing is essential.
But I don't understand why CPU scores can influence the final 3DMark score in such a degree.
//snip//
So just imagine one with fast CPU+6600GT get a 3DM06 scores higher than one with slower CPU+6800GT, I really can't and won't find a game is CPU limited in this way, now plus future.

and

Well actually since some of the tests are already heavily cpu limited why didn't you make them hit the GPU more?

Is this really what is happening though ?

I just ran the test 3 times with my Fx-55 at 2200, 2400 and 2600Mhz. The cards are 2 x 6800GS at 485/1100.

format is MHz
Score SM2/SM3/cpu

*******************
2200Mhz

4066 => 2214/1935/731

2400Mhz

4401 => 2228/1936/938

2600Mhz

4503 => 2234 / 1938 / 1016

********************

400Mhz on an Fx-55 gives 20 points for SM2, 3 points for SM3, 300 points for the cpu test ( which seems reasonable).

Of course with dual top end cards with a touch of exotic cooling it might be different but I doubt 06 will become cpu "limited" for most folks before 07 arrives. Feel free to run your own tests to see for yourself. Certainly this seems less cpu affected ( I hate limited as it suggests a cap ) than 05 and is in my view a far better bench.

People seem to be writing their own version of history before actually doing any tests it seems ??

Razor1

19-Jan-2006, 18:32

Razor1

19-Jan-2006, 18:33

and

Is this really what is happening though ?

I just ran the test 3 times with my Fx-55 at 2200, 2400 and 2600Mhz. The cards are 2 x 6800GS at 485/1100.

format is MHz
Score SM2/SM3/cpu

*******************
2200Mhz

4066 => 2214/1935/731

2400Mhz

4401 => 2228/1936/938

2600Mhz

4503 => 2234 / 1938 / 1016

********************

400Mhz on an Fx-55 gives 20 points for SM2, 3 points for SM3, 300 points for the cpu test ( which seems reasonable).

Of course with dual top end cards with a touch of exotic cooling it might be different but I doubt 06 will become cpu "limited" for most folks before 07 arrives. Feel free to run your own tests to see for yourself. Certainly this seems less cpu affected ( I hate limited as it suggests a cap ) than 05 and is in my view a far better bench.

People seem to be writing their own version of history before actually doing any tests it seems ??

Speak of the devil hehe, Thx dizietsma

It truelly is pixel shader bottlenecked. This explains quite a bit :)

dizietsma

19-Jan-2006, 18:38

Speak of the devil hehe, Thx dizietsma

It truelly is pixel shader bottlenecked. This explains quite a bit :)

I am trying to pay for the pro version so I could run 1600x1200 as well but it seems stuck, the 1 minute it says it might take to process my payment has now last 25 minutes. I'm worried it has put me into the cpu test by mistake !! :D

Joe DeFuria

19-Jan-2006, 18:41

As expected, many people are probably comparing complete 3DMark scores (which includes the CPU test) and deciding it's not a GPU "benchmark." (In other words...people don't have the ability to separate out scores.)

That being said, the graphics tests do not necessarily indicate that it's the pixel shaders that are the bottleneck...only the graphics subsystem as a whole. (Could be pixel or vertex shaders, bandwidth limitations, or something else).

On a related note, the CPU score is simply weighted too heavily in the final 3D Mark score calculation. If it were given much less weight, then we wouldn't be hearing so many (valid) complaints about the CPU impacting the score "too much".

boltneck

19-Jan-2006, 18:44

Jawed

19-Jan-2006, 18:47

I just ran the test 3 times with my Fx-55 at 2200, 2400 and 2600Mhz. The cards are 2 x 6800GS at 485/1100.
http://www.beyond3d.com/forum/showpost.php?p=676993&postcount=238

Jawed

boltneck

19-Jan-2006, 18:49

Ok I take the above post back.

I am still going to get HD decoding, and AA for my Wide screen format games. It will also be fast enough for the next year.

*sigh*...

Razor1

19-Jan-2006, 18:50

http://www.beyond3d.com/forum/showpost.php?p=676993&postcount=238

Jawed

Seems like a 3200+ is definilty CPU bound, but when using faster CPUs like the fx55 seems like the bottleneck shifts over to pixel shaders. This is probably even more prevalent on dual cores if the physics engine utilizes both cores.

Jawed

19-Jan-2006, 18:58

dizietsma

19-Jan-2006, 19:00

http://www.beyond3d.com/forum/showpost.php?p=676993&postcount=238

Jawed

What do your own tests show ?

Razor1

19-Jan-2006, 19:03

In this pair of comparisons, the FX55 is running at 2.2 and 2.6GHz versus 2.0 and 2.5GHz for the Venice.

I think you're mistaking the graphics-limited SLI-ed 6800GS for the apparently not graphics-limited SLI-ed 7800GTX.

Or, in other words, 193 CPU points on the Venice is delivering a massive 590 SM2 points and 310 SM3 points.

Jawed

But isn't there a cache difference on these two CPU's?

Also SLi-7800 would give alot more horsepower, so far we have only seen in benchmarks, single cards. Anyone have links to benchmarks on Sli'ed 6800gs's perform against a single 7800

Hubert

19-Jan-2006, 19:06

So basically, I really did just spend several hundred dollars on a total dud that was marketed as a Ferrari. :cry:

This strikes me as bordering on unethical behavior by ATi.

I might as well have set my money on fire or purchased a Geforce FX or whatever that was.

This is about as bad as I have felt in a long time. Worst case of Buyers remorse ever.

Stop whining. :) I still have my order. I won't cancel it. You'll be happy with that card. It's better than a 7800 GTX 256 Mb IMO. And it costs the same.

Jawed

19-Jan-2006, 19:08

The Venice gains 193 points from 2.0 to 2.5GHz, while the FX55 gains 285 points from 2.2 to 2.6GHz.

So the cache is already factored-in to the benchmark results we're examining.

As to my own results, my X800XT is clearly far too GPU-limited to show up the CPU limits in 3DMk06. Not to mention the fact that I can't be arsed.

As I posted earlier, results posted in this thread indicate that the game tests are CPU-limited.

Jawed

mrcorbo

19-Jan-2006, 19:12

Well, at first, I was pretty preplexed by why the CPU tests were factored into the final 3dMark score. Now it makes sense to me, though. Futuremark have taken past criticisms to heart WRT their benchmark not simulating a game properly by not doing any AI or physics calculations. Now the 3DMark score is much more of a measure of system performance instead of just GPU performance. Those that want to measure just the GPU performance are still free to just use the scores from the GPU tests. It's not like it's impossible now.

And as far as relative CPU performance between different types of CPUs goes:

A64/PIV w/o HT = 1 thread at a time
A64X2/PIV w/HT = 2 threads
PIV-D = 4 threads
XBOX360 = 6 threads

Notice the trend. And now that developers have seen how easy and beficial to performance it is to take advantage of the ability to execute multiple threads using an existing middleware tool, I think this is definately the way they are going to head.

dizietsma

19-Jan-2006, 19:15

I think you're mistaking the graphics-limited SLI-ed 6800GS for the apparently not graphics-limited SLI-ed 7800GTX.

Jawed

What graphics limitated SLI GS ?

This is the overclocked 485/1100 XFX version which gives the 512MB GTX a good run for it's money

http://www.motherboards.org/reviews/hardware/1564_6.html

so I don't think the GS is limited.

Ragemare

19-Jan-2006, 19:15

Can't reviewers manually work out what score the 7800 would get with AA on? The 3dMark score equation was posted earlier on in this thread unless I'm imageinging things.

dizietsma

19-Jan-2006, 19:18

As I posted earlier, results posted in this thread indicate that the game tests are CPU-limited.

Jawed

Except mine of course :D

Lets all do the tests and come to some concensus. Just run 06 at 50%, 75% and 100% of your default cpu speed and see how the results vary. Should make a nice little database at the least.

Joe DeFuria

19-Jan-2006, 19:22

Can't reviewers manually work out what score the 7800 would get with AA on?

Yes, they can. Do you think they will?

I think there's more chance of them not even RUNNING AA tests since a score isn't generated for nVidia cards (thus, no interesting and pretty bar charts to show) , than calculating what the score "would have been".

Which, if I were a cynical type, I would say is exactly what nvidia would prefer.

Joe DeFuria

19-Jan-2006, 19:23

Boy, do we ever need a B3D write up on this....

Razor1

19-Jan-2006, 19:28

Hehe definitly Joe.

In the meantime, lets look at this.

http://www.pcper.com/article.php?aid=199&type=expert&pid=6

Seems like its not CPU limited here at all.

Looking at the individual tests, sm 3.0 ATi cards seem to do better respectively, possible due to the increased branch performance.

Jawed

19-Jan-2006, 19:29

What graphics limitated SLI GS ?
Compared with 7800GTX SLI in 3DMk06, obviously, yes.

Jawed

dizietsma

19-Jan-2006, 19:31

Boy, do we ever need a B3D write up on this....

Very true Joe. If nothing else the release of this 3dmark has made people think about things.

Myself I think the cpu score is a Godsend. I don't think it is over estimated because Intel are promising quad cores and what are you going to do with 4 cores ? One for the operating system background tasks and 3 for the game is my answer !

At present we have only 2 physcial cores so I do not mind 06 being weighted this way.

Moloch

19-Jan-2006, 19:34

So basically, I really did just spend several hundred dollars on a total dud that was marketed as a Ferrari. :cry:

This strikes me as bordering on unethical behavior by ATi.

I might as well have set my money on fire or purchased a Geforce FX or whatever that was.

This is about as bad as I have felt in a long time. Worst case of Buyers remorse ever.
easy killer, it's only a benchmark.
It isn't a game.
Even if your card supported everything the GF7 does it still would run this benchmark like a turd.

Razor1

19-Jan-2006, 19:34

Very true Joe. If nothing else the release of this 3dmark has made people think about things.

Myself I think the cpu score is a Godsend. I don't think it is over estimated because Intel are promising quad cores and what are you going to do with 4 cores ? One for the operating system background tasks and 3 for the game is my answer !

At present we have only 2 physcial cores so I do not mind 06 being weighted this way.

What would be even nicer is if windows vista will allow us to configure which programs use which cpus :)

Joe DeFuria

19-Jan-2006, 19:36

Myself I think the cpu score is a Godsend.

I have no problem with the test itself, nor do I have a problem with the CPU test figuring into the final score. The only real issue I have, is that IMO, they put far too much weight into CPU test when calculating the overall score.

No matter how many cores the CPU manufacturers offer us, that doesn't mean games will be able to make almost linear usagae out of them.

dizietsma

19-Jan-2006, 19:48

Compared with 7800GTX SLI in 3DMk06, obviously, yes.

Jawed

I just tested 515/1100 on the card instead of 485/1100 and get

FX-55 at 2400Mhz

4569 => 2358/2035/940

Which is higher than my score for FX-55 at 2600Mhz at 485/1100. So on my system at least this bench is overwhelmingly gpu limited ie 40Mhz gpu gain is worth more than 200Mhz cpu gain and that is not including bandwidth which this test seems to like.

Indeed the top score for a GS is

http://service.futuremark.com/compare?3dm06=8451

and even taking away 1000 points for his dual A64 he is still ahead of my card at lower clocks showing there is still some headroom to go.

boltneck

19-Jan-2006, 19:49

Didnt you guys see where i took that post back just a post later? :???:

I vented, i got over it, i am happy again :)

dizietsma

19-Jan-2006, 19:52

The only real issue I have, is that IMO, they put far too much weight into CPU test when calculating the overall score.

No matter how many cores the CPU manufacturers offer us, that doesn't mean games will be able to make almost linear usagae out of them.

That's where we'll have to see. Already cpu's are just starting to make fairly big differences for games with multithread tweaks, bigger gains than whether the IHV's AF algorithm is set to high quality or not and we make a lot of noise about that do we not ?

Who's to know what gains we can have when both IHV driver teams and game developers start working on offloading to the multicores ? Futuremark's "guess" might be a massive underestimation.

I can just see nvidia marketing SuperNitroTurbocache2 in 2007 where they are referring to the cpu and not the memory :)

mrcorbo

19-Jan-2006, 19:57

Unknown Soldier

19-Jan-2006, 19:58

I am not even mentioning the .2 fps or so that my 3200+ at 2503 Mhz gets in the CPU test .

My 3200+ Barton hit 1, 5 times out of 100 frames ;)

US

Hanners

19-Jan-2006, 19:59

With regard to the SM2.0 and HDR/SM3.0 results being affected by the CPU, here are my results on an Athlon64 3500+ versus an Athlon64 X2 4200+ (Both running with a NVIDIA GeForce 7800GT):

Athlon64 3500+
SM2.0 Score 1518
HDR/SM3.0 Score 1525
CPU Score 866

Athlon64 X2 4200+
SM2.0 Score 1493
HDR/SM3.0 Score 1512
CPU Score 1655

dizietsma

19-Jan-2006, 20:02

Re: CPU scaling

Keep in mind that if you change your CPU by O/C the HT bus that you are not only increasing your CPU perfomance, but also the memory and of course the HT performance, as well. The proper way to test this is as suggested. Underclock your CPU by changing the multiplier down.

That's a good point. The other person Jawed quoted was running at 250Mhz HTT and so the Pcie speed might have been out as well. Pcie speed is yet another kettle of fish, normally it does not make much difference but who knows yet, I certainly have not tested it.

My figures with my unlocked Fx-55 were done at 11x200, 12x200 and 13x200 to be uniform so I do not have to worry about changes in HTT and Pcie speeds.

SugarCoat

19-Jan-2006, 20:20

wont someone with a gtx pretty please force HQ AF 16 and list scores, pretty please?

Pete

19-Jan-2006, 20:36

question:
Why FM decided that no current generation card will be able to make 25 fps !? [...] personally i'd prefer NOT to be hit so hard in face "upgrade, upgrade, UPGRADE, UPGRADE"I think previous 3DMs debuted similarly, with high-end cards scoring around 5k, but I wonder if this "low" performance is a clue as to how long until we can expect a DX10 3DM?

I wish Nick would have completely clarified FM's decision to exclude NV scores when AA is applied. Graphix and Mariner made the best posts most succinctly, so I'll just (re-)repeat for emphasis:3DM06 AA Test:
X850 does run SM2.0 tests with AA -> SM2.0 score counting (because it cannot run SM3.0/HDR tests)
7800 does run SM2.0 Tests with AA -> No score counting (because it cannot run SM3.0/HDR tests)
1800 XT does run SM2.0 Tests with AA -> SM3.0 score counting (because it can run all tests)
It strikes me that Futuremark's design decisions, however honestly conceived, penalise ATI's current high-end chip for not supporting Fetch4. On the other hand, the R5X0 series of chips are able to support AA + HDR, something no NVidia chip is able to do, yet these NVidia chips are not penalised in the same way.

We know that chips which don't support the required depth textures for the PS2.0 shadowing are forced into a relatively expensive shader workaround which is fine by me as Futuremark have decided 24-bit accuracy is required. On the other hand, if this is acceptable, why aren't chips which are not able to support AA + HDR also forced into a shader workaround?

[...]

This being the case, surely it would have been logical for Futuremark to include a shader workaround for SM3.0 cards unable to support AA + HDR which generates the AA in-engine - the technique recommended by NVidia's Chief Scientist?
Ok about CPU score. I understand CPU testing is essential. But I don't understand why CPU scores can influence the final 3DMark score in such a degree.

Can a dual-core CPU gives you better shader performance?
Can a dual-core CPU gives you HDR+AA?
Can a dual-core CPU gives you SM3.0?

While dual-core CPU IS influential in real game situations, but it won't be much influential as GPU does.
So just imagine one with fast CPU+6600GT get a 3DM06 scores higher than one with slower CPU+6800GT, I really can't and won't find a game is CPU limited in this way, now plus future.I dunno, a slow CPU can probably hamper gameplay as much as a slow GPU. I'm of the opinion that framerate comes first, everything else second. One could argue whether DC will show as much of an improvement as SC, of course, but I think 360 and PS3 should make exploiting DC quite common--and possibly similar to fancy effects, if the majority of the PC market is SC and so the second core is just used for neat tricks like more boxes or boulders.

I'd like to see more testing done on Q4, considering how much of an improvement it sees with DCs. Specifically, we'd want to test without AA and preferably at 12x10, to more closely mimic 3DM's vision of future games.

As for the arguments over uniquely accelerated features, I think they should be held in check until we see more benches (http://www.elitebastards.com/page.php?pageid=13398) using 3DM06's thoughtfully-included switches to disable "hardware" shadow mapping and FP filtering.

Hanner's limited (NV-only) testing does seem to suggest that ATI wasn't too silly to skip "fixed" FP16 filtering (Nick wasn't kidding when he said their SW fallback was "highly efficient"), though I'd like to see SS compares to examine IQ differences, if any. OTOH, NV's huge hits w/o HSM (25% on a 6800GT, 17% on a 7800GT) beg the question why FM couldn't have implemented a SW-based HDR AA workaround and considered it an equivalent situation?

Ah, FM says, but AA isn't part of their standard suite, just an option. Well, is HDR isn't part of SM3's standard suite? HSM? FP filtering? If the answer is that 3DM isn't a D3D test, but a gamer's test, then surely gamers use AA as (much as) they would fancy shadows, as an IQ enhancer?

I'm cool with most of the test. I think it's eminently fair to take advantage of HW features, as surely game devs would do the same. Only the (GF 6's and 7's lack of) AA score reporting puzzles me. Though we can calculate it by hand, we shouldn't have to, and forcing us to do so only diminishes the holistic "3DMarks" relevance.

Unknown Soldier

19-Jan-2006, 20:49

Pete

19-Jan-2006, 20:54

A64 X2@2100 9800pro 370core/340 mem
3D Mark Score= 606
SM2.0 = 281
Cpu Score = 1589.
3DMark Score: 996
SM2.0 Score: 480
CPU Score: 1006

GFX: RADEON 9800 XT @434MHz/370MHz
CPU Intel Pentium M 1.60GHz @ 2.4GHzThought I'd kick in my 9800P scores, just to see how much my AXP 2400+ (@2000MHz, 256kB, single-channel DDR266) is crippling me:

545 3DMarks
SM2.0: 246
CPU: 654

Honestly, not too bad compared to a dual 2.1GHz A64 w/ practically four times the RAM bandwidth (64b DDR266 -> 128b DDR400), and merely iffy compared to a 2.4GHz P-M. Too bad the difference isn't as small in games, where I'm sure faster RAM or more L2 would greatly benefit me.

Nocturn's XT's extra VRAM likely contributes to his much higher SM2 score.

Pete

19-Jan-2006, 20:57

I think that's pretty interesting. A single core 3500+ that beats a dual core X2 4200+"Beats" is a pretty strong word. The difference in the 3D tests is a mere 1-2%, practically margin of error.

But I'd be interested to see 3DM06 performance with both pre- and DC-optimized drivers on DC systems, to see if there are any gains (or even losses, if the drivers and 3DM conflict, as with Q4).

trinibwoy

19-Jan-2006, 20:58

I think that's pretty interesting. A single core 3500+ that beats a dual core X2 4200+

Looks like the 4200+ completely demolishes the 3500+, 1655 to 866. If you're referring to the few points higher on the GPU tests, that is probably well within the margin for error.

Hanners

19-Jan-2006, 20:59

"Beats" is a pretty strong word. The difference in the 3D tests is a mere 1-2%, practically margin of error.

Indeed, and I should mention those scores were on different motherboards (Both A8N-SLI based, but still), so there are likely different memory timings in play there for starters. It was just a quick and dirty comparison really.

MY80S

19-Jan-2006, 21:06

Has anyone tried to bench nvidia hardware on the sm2 codepath yet ? I get a small, consistent fps increase on both nv40 and g70 in GT1 with it. Of course, it can't run the SM3.0 tests on it...

As for the arguments over uniquely accelerated features, I think they should be held in check until we see more benches using 3DM06's thoughtfully-included switches to disable "hardware" shadow mapping and FP filtering.

My results are more or less in line with Hanners'. Disabling HSM hurts the 7800gtx by ~24.5% in gt1 and ~18% in gt2. Forcing shader FP filtering induces a 6% performance hit in gt1 and an 8% one in gt2 on the same card.

ANova

19-Jan-2006, 21:39

X1800 is the odd one out of the X1000 line - it, and only it, does not support Fetch4.

You're right, my mistake. Six pages later.

So we know that the X1900 supports fetch4, but does it support D24X8?

.Melchiah.

19-Jan-2006, 21:41

Thought I'd kick in my 9800P scores, just to see how much my AXP 2400+ (@2000MHz, 256kB, single-channel DDR266) is crippling me:

545 3DMarks
SM2.0: 246
CPU: 654

Weird. My score is slightly better, eventhough you have a better videocard. Did you put "mipmap detail level" to "performance" instead of "quality" in the Catalyst control center? That could explain it.

System: XP2800+ (Barton 2.08GHz), 9700 Pro, 1024MB PC2700 (2x 512MB).

3DMark06 score: 652

SM2.0 score: 312
CPU score: 730

I decided to try the older 3DMarks with Catalyst 6.1 drivers as well;
3DMark05 score: from 2274 to 2382
3DMark03 score: from 5062 to 5059
3DMark01 score: from 14492 to 13099 =O !

Cowboy X

19-Jan-2006, 21:44

My 3200+ Barton hit 1, 5 times out of 100 frames ;)

US

Ok I'll be thankful , lol .

N00b

19-Jan-2006, 22:18

7800 GTX 512 (580/1730), AMD X2 4800+, 2 GB, ASUS A8N-SLI Premium, Driver Version 81.98, High Quality

5479 3DMark06 Score
2242 SM 2.0
2288 HDR/SM 3.0
1852 CPU

:grin:

trinibwoy

19-Jan-2006, 22:28

Looks like somebody just grew a third leg....:eek:

Freak'n Big Panda

19-Jan-2006, 22:42

wont someone with a gtx pretty please force HQ AF 16 and list scores, pretty please?

Here u go:

Without 16xAF
3430 total
SM 2: 1715
SM 3: 1608
CPU: 654

With 16xAF:
3069 total
SM 2: 1347
SM 3: 1562
CPU: 652

The scores are probably worse then what I should be getting with a 3000+ and a 256MB 7800GTX (not oced at all of course :p) but I didn't bother to reebot and I had tons of apps running the background (WMP, folding, azureus ect.).

I dunno, a slow CPU can probably hamper gameplay as much as a slow GPU. I'm of the opinion that framerate comes first, everything else second. One could argue whether DC will show as much of an improvement as SC, of course, but I think 360 and PS3 should make exploiting DC quite common--and possibly similar to fancy effects, if the majority of the PC market is SC and so the second core is just used for neat tricks like more boxes or boulders.

I'd like to see more testing done on Q4, considering how much of an improvement it sees with DCs. Specifically, we'd want to test without AA and preferably at 12x10, to more closely mimic 3DM's vision of future games.

As for the arguments over uniquely accelerated features, I think they should be held in check until we see more benches using 3DM06's thoughtfully-included switches to disable "hardware" shadow mapping and FP filtering.

Hanner's limited (NV-only) testing does seem to suggest that ATI wasn't too silly to skip "fixed" FP16 filtering (Nick wasn't kidding when he said their SW fallback was "highly efficient"), though I'd like to see SS compares to examine IQ differences, if any. OTOH, NV's huge hits w/o HSM (25% on a 6800GT, 17% on a 7800GT) beg the question why FM couldn't have implemented a SW-based HDR AA workaround and considered it an equivalent situation?

Ah, FM says, but AA isn't part of their standard suite, just an option. Well, is HDR isn't part of SM3's standard suite? HSM? FP filtering? If the answer is that 3DM isn't a D3D test, but a gamer's test, then surely gamers use AA as (much as) they would fancy shadows, as an IQ enhancer?

I'm cool with most of the test. I think it's eminently fair to take advantage of HW features, as surely game devs would do the same. Only the (GF 6's and 7's lack of) AA score reporting puzzles me. Though we can calculate it by hand, we shouldn't have to, and forcing us to do so only diminishes the holistic "3DMarks" relevance.

Yay a logical post for once!

Chalnoth

19-Jan-2006, 22:51

OTOH, NV's huge hits w/o HSM (25% on a 6800GT, 17% on a 7800GT) beg the question why FM couldn't have implemented a SW-based HDR AA workaround and considered it an equivalent situation?

Ah, FM says, but AA isn't part of their standard suite, just an option. Well, is HDR isn't part of SM3's standard suite? HSM? FP filtering? If the answer is that 3DM isn't a D3D test, but a gamer's test, then surely gamers use AA as (much as) they would fancy shadows, as an IQ enhancer?
Er, it's very easy to implement FP filtering in the shader for simple situations (and since FP filtering is probably only used for tonemapping, this should be extremely easy). It's impossible to implement multisampling AA in the shader.

One could obviously implement supersampling AA in software, but this would hardly be equivalent either in performance or in visual quality.

Razor1

19-Jan-2006, 22:57

Looks like somebody just grew a third leg....:eek:

:lol:

Jawed

19-Jan-2006, 23:00

Richteralan

19-Jan-2006, 23:02

Chalnoth

19-Jan-2006, 23:16

I believe that 6 and 7 series can generate MSAA samples for an FP16 backbuffer - it's simply that the samples cannot be resolved in hardware, nor can a backbuffer blend be done in hardware.
If you can generate the samples, then clearly it would be possible to resolve them, provided you can access the framebuffer as just a buffer that is higher in resolution than the desired resolution (i.e. a 1280x960 4xAA framebuffer would be stored as 2560x1920).

The question is, why do you think that the NV4x can generate MSAA samples for an FP16 backbuffer?

N00b

19-Jan-2006, 23:21

Looks like somebody just grew a third leg....:eek:It's been there before. I swear!

Ok, I know. Bragging about 3DMark scores is sooo vain. :oops: But this is actually the first time a new 3DMark comes out that I don't feel like upgrading instantly.

IgnorancePersonified

19-Jan-2006, 23:29

Thought I'd kick in my 9800P scores, just to see how much my AXP 2400+ (@2000MHz, 256kB, single-channel DDR266) is crippling me:

545 3DMarks
SM2.0: 246
CPU: 654

Honestly, not too bad compared to a dual 2.1GHz A64 w/ practically four times the RAM bandwidth (64b DDR266 -> 128b DDR400), and merely iffy compared to a 2.4GHz P-M. Too bad the difference isn't as small in games, where I'm sure faster RAM or more L2 would greatly benefit me.

Nocturn's XT's extra VRAM likely contributes to his much higher SM2 score.

What's the vid card clocks Pete? When I get home I will rerun the tests with my vid card at the fastest it can go - (I think 375/355) - its summer here so I down clock most things unless needed. It's really a 9800SE on a 9700 board softmodded. I did that test whilst grabbing dinner.

But the CPU doesn't seem to be weighing in as much as what people are saying. NocturnDragon's 3Dmarks score smashes mine yet the cpu score for yours is 1/3 mine.

SugarCoat

19-Jan-2006, 23:32

Here u go:

Without 16xAF
3430 total
SM 2: 1715
SM 3: 1608
CPU: 654

With 16xAF:
SM 2: 1347
SM 3: 1562

The scores are probably worse then what I should be getting with a 3000+ and a 256MB 7800GTX (not oced at all of course :p) but I didn't bother to reebot and I had tons of apps running the background (WMP, folding, azureus ect.).

256 GTX Without 16xAF
SM 2: 1715
SM 3: 1608

vs http://www.anandtech.com/video/showdoc.aspx?i=2675&p=3 numbers

1747 in SM2.0
1729 in SM3.0/HDR

With 16xAF:
SM 2: 1347
SM 3: 1562

vs X1800XT HQ16AF Forced:

1615 SM2.0 Score
1749 HDR/SM3.0 score

your 3dmark score is not off by much, as the default settings doesnt appear overly effected per other results.

I find the difference between 16AF on ATI vs 16 AF on Nvidia very interesting if not a bit misleading compared to default scoring. On one hand we have the 256mb GTX actually leading the X1800, as firingsquad pointed out going against the grain, and once you enable HQ you have the X1800 taking a rather substantial lead. Feel free to re-bench if you like, but i dont think what you had running in the background effected your graphic scores much. The HDR tests seem the most CPU limited, so i'm sure you are losing some points, but i think i got what i was looking for. My CPU is a 2.0 and the one used on Anand was an FX55.

I think Nvidia has done a bit more homework in optimizations is all.

if anyone else would care to do the same, i wouldnt mind seeing if this is a constant drop the G70 cores are taking.

Jawed

19-Jan-2006, 23:57

If you can generate the samples, then clearly it would be possible to resolve them, provided you can access the framebuffer as just a buffer that is higher in resolution than the desired resolution (i.e. a 1280x960 4xAA framebuffer would be stored as 2560x1920).
Theoretically, yes - but I don't think reading the samples from Z while rendering is a trivial matter (in order to decide from the samples' Z values which samples to overwrite with the newly generated pixel's samples). As a non-dev this is simply beyond my ken :oops: It might, for example, only start to be possible by doing a z-prepass to build Z which is dumped into a texture, in order that it can be queried while doing the colour pass... erm...

The question is, why do you think that the NV4x can generate MSAA samples for an FP16 backbuffer?
Apparently G70 can, as this was a topic that came up as part of the discussion of nAo's implementation of HDR in Heavenly Sword (PS3 title). I presume it applies to NV40, too, as their ROPs are generally seen as common in function.

It was news to me, anyway - and I hope I've interpreted it correctly. It would be nice to get more insight because the concept was left hanging in the air.

Jawed

Chalnoth

20-Jan-2006, 00:01

Theoretically, yes - but I don't think reading the samples from Z while rendering is a trivial matter
Well, if you can't do that, then you can't generate the multisample buffer in the first place.

trinibwoy

20-Jan-2006, 00:35

Control Panel set to HQ. Forceware 82.12. 7800GTX @ 490/1300.

Setting, SM2.0, SM3.0/HDR

Default 1781, 1767
Software FP16 filtering 1779 0%, 1648 6.7%
Disabled Hardware Shadow Mapping 1463 17.9%, 1769 0%
16x AF 1394 21.7%, 1476 16.5%

Percentages calculated using default scores as base hence they represent the performance hit in each situation.

mrcorbo

20-Jan-2006, 00:53

HT has nothing to do with memory bus performance in A64 architecture.

This is not correct. Your memory frequency mirrors your HT freq by default, or you can set a divider.

Unknown Soldier

20-Jan-2006, 00:59

7800 GTX 512 (580/1730), AMD X2 4800+, 2 GB, ASUS A8N-SLI Premium, Driver Version 81.98, High Quality

5479 3DMark06 Score
2242 SM 2.0
2288 HDR/SM 3.0
1852 CPU

:grin:

SLI or Single card?

cho

20-Jan-2006, 02:19

http://www.gzeasy.com/Articlephoto/futuremark_3dmark06/results/image010.gif

the vertex shader simple results seems very strange (the CPU is FX55), RADEON X1300@550MHz higher than the RADEON X1800XL, the later have 4 times VS units.

Chalnoth

20-Jan-2006, 02:22

SLI or Single card?
That had better be single, because my 7800 GT SLI scores a little bit higher, on a slower CPU :)

http://service.futuremark.com/compare?3dm06=27577

Geo

20-Jan-2006, 02:46

Everyone, stay tuned. I've been talking with Futuremark and ATI and have cleared up a couple of things on the whole 24-bit depth stencil texture issue. There should be an update to the article (second page) soon.

I'm still a little concerned that the benchmark doesn't use some form of parallax mapping technique - it's already in FEAR, it's a major feature of UE3, it's going to be in a lot of SM3.0 games, and it's not in 3DMark06. It seems like one of the defining shader effects of 2006-era 3D engines to me.

From my "better late than never" file, thanks for a nice read, Jason. :smile:

We have since learned from ATI. . . Radeon X1300 and X1600 cards, as well as upcoming ATI hardware, support the 24-bit depth stencil textures and Fetch4,

The above lines (hopefully I didn't do violence to the sense with that ellipsis), from your Update, really made me wonder why FM didn't wait another week to release this, as doing so would have removed at least one cause of grief for them the last two days.

Richteralan

20-Jan-2006, 02:59

This is not correct. Your memory frequency mirrors your HT freq by default, or you can set a divider.

That's what I said, the performance of memory bus is judged by memory divider. And A64 memory controller has auto memory frequency adjustment.

Unknown Soldier

20-Jan-2006, 03:08

That had better be single, because my 7800 GT SLI scores a little bit higher, on a slower CPU :)

http://service.futuremark.com/compare?3dm06=27577

That's weird, cause this bloke is only getting 5057 with SLI GTX

http://service.futuremark.com/compare?3dm06=20068

His SM2.0 is lower but his SM3.0 score is higher and the CPU score is lower.

US

Pete

20-Jan-2006, 04:12

Weird. My score is slightly better, eventhough you have a better videocard. Did you put "mipmap detail level" to "performance" instead of "quality" in the Catalyst control center? That could explain it.It just might. I stick with Balanced, or whatever the default is, b/c I don't appreciate MIP-map bands.

I decided to try the older 3DMarks with Catalyst 6.1 drivers as well;
3DMark05 score: from 2274 to 2382
3DMark03 score: from 5062 to 5059
3DMark01 score: from 14492 to 13099 =O !I haven't run 03 or 05 in a while, but I just ran 01 for kicks, and my CPU/RAM are holding me to 10900 (Cat 5.13 Balanced).

What's the vid card clocks Pete?Stock speeds: 378/338, according to ATT. I'll try OCing too, eventually. I think I can go north of 420/380, from the half hour I spent using ATT's OCing tool, and then testing in CSS.
But the CPU doesn't seem to be weighing in as much as what people are saying. NocturnDragon's 3Dmarks score smashes mine yet the cpu score for yours is 1/3 mine.True, at least on our piddling 128MB cards. :)

Er, it's very easy to implement FP filtering in the shader for simple situations (and since FP filtering is probably only used for tonemapping, this should be extremely easy). It's impossible to implement multisampling AA in the shader.Thanks. I figured as much, but had to ask. So, SS is the only option, and that means a huge hit and possibly not-so-great IQ. (Still doesn't explain the NA score, though.)
One could obviously implement supersampling AA in software, but this would hardly be equivalent either in performance or in visual quality.Well, does it have to be "SW-based?" NV does SSAA in "hardware," no? That won't work with 3DM06's HDR?

really made me wonder why FM didn't wait another week to release this, as doing so would have removed at least one cause of grief for them the last two days.Well, two reasons spring to mind: to avoid X1900's shadow (and associated 7900 "leaks" ;)), and to avoid being cast as ATI's plaything (a la Valve).

It's a tough, cynical, HDR world, this PC gaming.

Reverend

20-Jan-2006, 05:56

OTOH, NV's huge hits w/o HSM (25% on a 6800GT, 17% on a 7800GT) beg the question why FM couldn't have implemented a SW-based HDR AA workaround and considered it an equivalent situation?

Ah, FM says, but AA isn't part of their standard suite, just an option. Well, is HDR isn't part of SM3's standard suite? HSM? FP filtering? If the answer is that 3DM isn't a D3D test, but a gamer's test, then surely gamers use AA as (much as) they would fancy shadows, as an IQ enhancer?
A big "Ditto" from me. Haven't checked out all the comments here but I'd appreciate Nick addressing this (if he hasn't already).

dizietsma

20-Jan-2006, 07:37

That's what I said, the performance of memory bus is judged by memory divider. And A64 memory controller has auto memory frequency adjustment.

What mrcorbo said originally was that putting up the HT bus speed also increases the memory bus speed which is true ( this is eaxily shown by cpu-z etc). It also puts up the Pcie speed if this is not locked. Both effect your performance as well as the increased speed of the cpu.

N00b

20-Jan-2006, 07:37

SLI or Single card?Single card.

dizietsma

20-Jan-2006, 08:02

Ah, FM says, but AA isn't part of their standard suite, just an option. Well, is HDR isn't part of SM3's standard suite? HSM? FP filtering? If the answer is that 3DM isn't a D3D test, but a gamer's test, then surely gamers use AA as (much as) they would fancy shadows, as an IQ enhancer?

I'd agree with this Pete except that gamers do have the option of putting up the screen resolution instead ( I am assuming they have a good enough monitor ). Indeed, Futuremark themselves have put the default screen resolution up and yet again left out AA in the standard test.

In the none standard tests then this would be a problem if you could not run tests individually and so could not judge how one cards performance compares to another because one card says N/A, but you can run tests where it is supported, the issue here is that people think the scoring is not fair for none standard tests.

To me this is a bit strange because for the last few months/years this forum has tended to pour scorn on Futuremarks bench, it's scoring and the use the IHV's use the scores to sell cards and that anybody who buys a card based on this is tending towards being a bit daft. But now it seems this is the upmost importance.

| I put on my green tinged pro futuremark hat |

And why is this ? because ATi is not favoured. Whenever Ati is not favoured then we get the most long winded threads where a court is summarily set up and the "injustice" to Ati is gone over in such minute detail that only paranoia can be thought to be in the heads of adjudicators. In swoops thw cardinals in their red ( how appropriate ) gowns, " Everybody expects the B3D inquisition " Fear and surprise is our .....

Remember that poor bloke from Anandtech that came over here and went grey haired before he had to leave saying he had better things to do ( as they all do, like nick[FM] ) apart from Kyle[H] who was rude enough to refuse to leave and eventually the thread had to be locked before veryone lost their marbles completely.

You wait, you lot just wait until nvidia do not have the perfect rub of the green ( pun intended ). i am going to start my very own thread in like manner, even if I have to write all the 400 posts myself.. :D

|rant over, green tinged pro futuremark hat back off|

Phew I feel better now :) Just out of interest, and in no connection to the above, where is WaltC nowadays ?

Mendel

20-Jan-2006, 08:47

I'm getting a whole lot of pauses due to what I think is texture swapping. So maybe 256MB of video memory isn't enough anymore, or maybe its just because AGP doesn't have enough bandwidth for this?

Hubert

20-Jan-2006, 09:04

dizietsma: Do you admit that Nvidia is perfroming better ? I guess you do.

What bothers most people that Nvidia's slight lead without AA is well known, but things are just the opposite when AA enabled, based on real life tests. (games)
But 3DMark2006 simply can't compare competing IHV's cards' with AA enabled. This totally nullifies Ati's effort put in optimising bandwith usage. So, whatever the reason are, this synthetic benchmark simply can't test competing products. Because few people will bother doing separate game tests (mostly reviewers) most people will just run it, got their score and an idea about their systems capabilities.

I think there are several questionable things regarding 3Dmark 2006 as a benchmark, but the main problem is the overall score can be misleading and unrepresentative. Some cards can't run the SM 3.0 tests, some cards don't get scored under certain circumstances, the CPU score weights maybe too heavily into the final score.

Also the fixed frame rate of the CPU test is very disappointing for the uninnicitaes, who don't know why it is so slow. Anyway, why even send pictures to your display at that framrate ? Usually, the CPU's activity is hidden, why have it a slideshow, why not just a blue bar with percentage done, than the final score ? If I got it right, the CPU tests won't ever reach the 20 fps range. Than what's the reason for showing anything ?