Futuremark: 3DMark06

Sunday said:
Since Nick has stated that there is no wrong in dong multi vendor path (even if that meant hurting one side implementing something that it doesn’t support, but relying on it heavily in some crucial tests), ‘cos hell “game developers are doing it†does it mean that is OK now for the vendors to do shader replacement trough drivers, or to do self made patches for 3DMark06 to improve performance (we’ve all prized Humus for his work on DOOM3)?
I have said that doing multivendor paths (as in totally different paths) in games is sure ok, but in 3DMark06 (and all previous) we use only one path with certain fallbacks to enable more hardware to be able to run the tests. We will not allow shader replacements or such. The driver optimization guidelines we set a couple of years ago still are in full effect.

Sunday said:
You’re so full of pride and joy Nick do you honestly cannot see what have you done to the 3DMark? You say there was no artistic reason to put Parallax Occlusion? Well god damn you’ve should create different art! There is a bunch of games that are designed right now in such way that this type of mapping is crucial for immersion, and what you giving us is some un-usable fireflies (this time two), one big dragon fish, skinned silly, AGAIN and one non playable Antarctic scene (good for some in game cinematic, and nothing more).
You think we should create our scenes just to promote one single effect that has been used in one (are there more released games (not engines/tech demos!) which use POM?) game? :???: Certainly not! Personal likings is one thing I won't go into any more than this. If you don't like the artwork, it is ok, but forcing some POM into them wouldn't have changed them visually much at all. It is down to the fact that not all effects are feasible to use just because they exist.

Sunday said:
You wore either lazy, to create something truly new and usable, or you just don’t know how! Anyway you turn it up you’ve failed with 06!
If that's your personal view on how things work and are, then so be it. I can't convince you to like what we have created if you simply don't like it.

Anyhow, thanks for the feedback!

Cheers,

Nick
 
Ratchet said:
I'm not so sure how far ahead it's looking. It doesn't include parallax mapping (which has already been included in games like FEAR) or a decent level of dynamic flow control (which is one of the more important SM3.0 features). If I would say anything, 3Dmark06 seems more like a modern benchmark, not a future one. In that case (and at the risk of soundling like I just graduated from the [H] school of thought) modern games would serve as a better guide to graphics performance.
Tell me what makes Parallax Mapping such an effect that it must be used in all games/apps? I fail to understand why this one effect is such a big issue. You name one game which supports it (FEAR), but I don't see people bashing other game benchmarks or games for not haing POM. Just doesn't make sense.

Sure, future engines may support POM (even we in the future if we find use for it), but we didn't find use for it in 3DMark. As I said, it is simply one effect amongst many others.

Cheers,

Nick
 
Guys,

while I'd love to hang in here 24/7, I need to continue to work. We will release a Technical FAQ soon where we have answered the most frequent asked questions about 3DMark06. It seems that too many simply load up the last page of this thread and I keep repeating myself when answering questions.

I'll keep you guys posted as soon as we have the FAQ ready and released!

If anyone has anything important to report (something that I haven't answered here, or available in the 3DMark06 Whitepaper) feel free to email me at, nick at futuremark.com. Please keep in mind that though we are very open to discuss our products and are willing to help everyone understand how the benchmarks work, there are certain limitations to what info we can share. I hope you respect that since we don't want to step on anyone's toes.

Cheers,

Nick
 
Bouncing Zabaglione Bros. said:
Hang on, wasn't dynamic branching demoed by Nvidia with their launch of NV40? Why is it that R520's late arrival stopped devs using dynamic branching, when Nvidia have been offering it for two product generations?
In the same way that vertex texturing is not a core function within 3DMk06 - it performs like a dog on NVidia hardware - dynamic branching is a complete waste of time on NVidia hardware in any form other than as a shortcut to static branching (or in other words it's prolly being used instead of having multiple, similar, shaders).

3DMk06 clearly doesn't use per-pixel dynamic branching in any meaningful fashion, because if it did the NVidia hardware would be down the toilet.

This is why I had quite a lot of confidence that we wouldn't see any significant use of dynamic branching - because FM prefers to "even things up" (though in the 05 and 06 that process favours NVidia heavily, hmm, strange huh?).

Dynamic branching should be in any forward looking benchmark, and would be a big advantage for ATI as they have spent a lot of transistors on it in R520/R580 - but for some reason Futuremark decided not to test this important aspect of newer SM3.0 hardware.
I agree - which is why I think this is the final nail in the coffin for FM, they've proven once and for all that the 3DMk series is irrelevant as a benchmark. Hoisted themselves by their own petard. And very prettily done, it has to be said.

Jawed
 
Nick[FM] said:
...

Sure, future engines may support POM (even we in the future if we find use for it), but we didn't find use for it in 3DMark. As I said, it is simply one effect amongst many others.

Cheers,

Nick
Well what is 3DMark06 then, is it future looking or not? You say in one post that it is, but in another that it won't sport features that aren't in (enough) current games?
 
Last edited by a moderator:
Jawed said:
3DMk06 clearly doesn't use per-pixel dynamic branching in any meaningful fashion, because if it did the NVidia hardware would be down the toilet.
Surely a game's performance shouldn't hinge on the use of dynamic branching though? What dev house would use it in such a manner to totally bork a huge amount of the user hardware base?

Wavey said:
AFAIK there are IP and usage issues related to PCF, though.
Oh I wasn't suggesting that ATI should be using PCF but rather that they implement FETCH4 across either the entire X1000 series or, at the very least, on their top-end model rather than just the mid- or low-end.
 
Jawed said:
In the same way that vertex texturing is not a core function within 3DMk06 - it performs like a dog on NVidia hardware - dynamic branching is a complete waste of time on NVidia hardware in any form other than as a shortcut to static branching (or in other words it's prolly being used instead of having multiple, similar, shaders).

With that in mind though, is dynamic branching likely to be used to any great extent in any future titles currently in development? In a sense, it's a bigger disappointment (to me anyway) that dynamic branching wasn't included in some way as an additional feature test (in the same sort of way vertex texture fetch was utilised in the shader particles test), rather than as part of the main graphics tests.
 
Ratchet said:
Well what is 3DMark then, is it future looking or not?
Can you tell me where in your opinion in 3DMark06 POM would improve image quality noticable? Which effect should have been dropped in favour of POM? (I assume that futuremark does not have unlimited resources and fixed shedule, so they probably would have had to drop something else in order to support POM)
 
Hanners said:
With that in mind though, is dynamic branching likely to be used to any great extent in any future titles currently in development? In a sense, it's a bigger disappointment (to me anyway) that dynamic branching wasn't included in some way as an additional feature test (in the same sort of way vertex texture fetch was utilised in the shader particles test), rather than as part of the main graphics tests.

Exactly...even if it didn't make it into the game tests in a meaningful way, there should at LEAST be a feature test that is basically a "branching test", similar to the vertex fetch test. I mean really...BOTH vendors support branching...and it is the single stand-out feature of PS 3....
 
N00b said:
Can you tell me where in your opinion in 3DMark06 POM would improve image quality noticable?

That is a separate question actually. It seems to me that both SM3.0 tests are very similar in terms of quality and features that they stress...why is this? Why, if you're going to have two test, would you not make them very different in terms of what they are stressing?
 
The theoretical/feature tests are always very low down, if not the last, on the list of FM's test priorities for 3DMark. God only knows how many times Worm has heard me twine about it...
 
N00b said:
Can you tell me where in your opinion in 3DMark06 POM would improve image quality noticable? Which effect should have been dropped in favour of POM? (I assume that futuremark does not have unlimited resources and fixed shedule, so they probably would have had to drop something else in order to support POM)
It isn't about improving image quality, this isn't a game or a eye-candy demo, it's supposed to be a benchmark. If one of your "tests" doesn't have a suitable place for POM, then make a test that does. Even if was a simple feature test, one that doesn't factor into the final score (like the perlin noise or shader particles tests) which stresses nothing but the parallax mapping feature would have been fine. Futuremark chose instead to completely ignore it even though it's probably going to be an often used feature in many upcoming games.
 
Last edited by a moderator:
Jawed said:
3DMk06 clearly doesn't use per-pixel dynamic branching in any meaningful fashion, because if it did the NVidia hardware would be down the toilet.

This is why I had quite a lot of confidence that we wouldn't see any significant use of dynamic branching - because FM prefers to "even things up" (though in the 05 and 06 that process favours NVidia heavily, hmm, strange huh?).


I agree - which is why I think this is the final nail in the coffin for FM, they've proven once and for all that the 3DMk series is irrelevant as a benchmark. Hoisted themselves by their own petard. And very prettily done, it has to be said.

Jawed
Which is exactly what I am alluding to. Futuremark made some questionable choices in 3D Mark 06, ones that benefit one IHV and actively hamstring the other. Strange eh? I guess they are still worried about what would happen if one company took their green-backs away, started "leaking" internal documents about how terrible Futuremark are, and threatened them with a bunch of lawyers.

Edit: I just think they gone to far in placating Nvidia by not including one of the most forward looking and lauded SM3.0 techniques (ie dynamic branching), even though Nvidia is the company that has been pushing this the longest of all! Leaves a bad taste in the mouth and asking what value is 3DMark06 if it doesn't even include dynamic branching, one of the keystones of SM3.0?
 
Last edited by a moderator:
Hanners said:
it's a bigger disappointment (to me anyway) that dynamic branching wasn't included in some way as an additional feature test (in the same sort of way vertex texture fetch was utilised in the shader particles test), rather than as part of the main graphics tests.

I agree with this statement, perhaps the single feature tests for next time need to be padded out so that as many techniques and effects can be measured, this at least will take away the claim that some features of a card are favoured or not favoured.

I would imagine ( perhaps naively) that lots of small tests are easier to chunk out than those impressive games so you can cover more effects. Futuremark is not the worlds biggest company of course so obviously they do have a limited amount of developement resources and manpower.

You still need a scored "game" though at least to get the crowds in .
 
Back
Top