FM puts up their FAQ on 3dm2k5

[trinibwoy]

That would be... Interesting

I don't think Dave or Rev would have made such a fuss if both major IHV's supported the feature. They definitely would not have used the exclusion of 3Dc to support their argument.

 

[Tim Murray]

You're missing one.

Oh. S3.

 

[Diplo]

I actually feel very sorry for the guys over at Future Mark because whatever they do they will be damned by a large section of the gaming public simply because there are so many IHV zealots around who take things so very personally.

To me there are two distinct purposes of a benchmarking tool:

One purpose is to provide synthetic tests for specific functions, such as fill-rate or how many triangles it can pump out. These are the theoretical performance and are useful for gaging raw performance and for 'apples to apples' comparisons.

The other purpose is to provide people with an idea of how future games will perform on their particular graphics card. Now, given that nearly every 3D engine is going to be optimised to utilise specific features of an IHVs 'feature set' it's impossible to try and provide this info and simultaneously be 'fair'. You have to be pragmatic in order to represent what will happen in real life. If vendors are likely to utilise IHV specific features then any hypothetical 3D engine should simulate that demand (or at least have the option to). Remember, the purpose of benchmarks are not to prove that your cock is bigger than someone else's, but to provide useful information to the public to enable them to make informed decisions.

 

[carpediem]

That would be... Interesting

I don't think Dave or Rev would have made such a fuss if both major IHV's supported the feature. They definitely would not have used the exclusion of 3Dc to support their argument.

I don't even think that will happen myself, but it would be rather interesting to see.

 

[Dave Baumann]

Actually, its not a political one, because going through DirectX guarantee's you two things: a specification and the ability to implement the hardware.

Here there is no specification for it, hence there is no set method to define how these should be implemented. If another vendor were to implemnt the fast PCF with merely a point filter, what would be the recourse for arguing against it - its not to the spec? No, there isn't one (other than what an IHV has implemented), it doesn't look the same? No, the DST and non-DST paths already don't look the same. There are also parallels here as to their reasoning being not using AA/AF as the default.

Secondly, if MS licenses a feature for us in DirectX then every hardware vendor is able to implement that hardware for use within DirectX under the terms of their DirectX hardware license - the classic example is DXTC, which is the same as S3TC; the vendors can freely implement it in hardware and are able to use it in DX, however they have to pay a license fee to S3 (or get it via some other means) in order to use it in OpenGL. Here DST is patented via SGI, but is not licensed by MS so vendors would need to get a license from SGI in order to implement it (if they were indeed aware that it was licensed by SGI).

 

[trinibwoy]

Secondly, if MS licenses a feature for us in DirectX then every hardware vendor is able to implement that hardware for use within DirectX under the terms of their DirectX hardware license - the classic example is DXTC, which is the same as S3TC; the vendors can freely implement it in hardware and are able to use it in DX, however they have to pay a license fee to S3 (or get it via some other means) in order to use it in OpenGL. Here DST is patented via SGI, but is not licensed by MS so vendors would need to get a license from SGI in order to implement it (if they were indeed aware that it was licensed by SGI).

Understood. But if in reality, as Futuremark suggests, many developers are going to pay out of pocket for a DST license anyway then isn't the above a moot point. The focus should be on the features that game engines will implement in the future and based on that knowledge we should define our expectations of synthetic benchmarks not the other way around. I hope this is what Futuremark tried to do with 3dmark05.

 

[Scali]

Here there is no specification for it, hence there is no set method to define how these should be implemented. If another vendor were to implemnt the fast PCF with merely a point filter, what would be the recourse for arguing against it - its not to the spec? No, there isn't one (other than what an IHV has implemented), it doesn't look the same?

The exact same arguments go for trilinear/anisotropic filtering.
If an IHV feels like implementing it with pointfilters, then so be it. It's not in the spec. Even so, the whole spec-argument is strictly political. The practical thing is that the image quality is reduced, and spec or no spec, I want the highest image quality possible.

Secondly, if MS licenses a feature for us in DirectX then every hardware vendor is able to implement that hardware for use within DirectX under the terms of their DirectX hardware license - the classic example is DXTC, which is the same as S3TC; the vendors can freely implement it in hardware and are able to use it in DX, however they have to pay a license fee to S3 (or get it via some other means) in order to use it in OpenGL. Here DST is patented via SGI, but is not licensed by MS so vendors would need to get a license from SGI in order to implement it (if they were indeed aware that it was licensed by SGI).

I don't see how this has anything to do with game developers or Futuremark. If IHVs choose to implement the feature, then software can use it. The rest is the problem of IHVs.

 

[Dave Baumann]

Well, the problems you have there is that they have implemented one shadowing mechanism for the entire benchmark suite - not every game will use the same shadowing mechanism, nor are they likely to implement dynamic shadows where they aren't needed.

 

[Dave Baumann]

The exact same arguments go for trilinear/anisotropic filtering.

There are however quality checks within DirectX DCT and the implementations of Trilinear presently fall within those. You'll note that Ansiotropic filtering is not part of the default benchmark.

I don't see how this has anything to do with game developers or Futuremark. If IHVs choose to implement the feature, then software can use it. The rest is the problem of IHVs.

Previously this was a DirectX benchmark - now they are asking IHV's to go outside of the licensing of DirectX in order to "score highest" on the benchmark. (edit - should say "score higher" I guess)

 

[digitalwanderer]

Presviously this was a DirectX benchmark - now they are asking IHV's to go outside of the licensing of DirectX in order to "score highest" on the benchmark.

No, no Dave...it's so it better simulates "real-world gaming".

 

[Scali]

Well, the problems you have there is that they have implemented one shadowing mechanism for the entire benchmark suite - not every game will use the same shadowing mechanism, nor are they likely to implement dynamic shadows where they aren't needed.

3DMark2001 covered projected shadows, 3DMark03 covered stencil shadows, 3DMark05 covered depth shadowmaps. What else is there?
And where exactly weren't the dynamic shadows needed in 3DMark05?

 

[Scali]

Previously this was a DirectX benchmark - now they are asking IHV's to go outside of the licensing of DirectX in order to "score highest" on the benchmark. (edit - should say "score higher" I guess)

Even if IHVs would feel moved to implement features just for 3DMark05's sake, which I sincerely doubt, this is not a bad thing, since all games with DST-support (quite a number of them, and still growing) will immediately benefit aswell.

 

[trinibwoy]

Previously this was a DirectX benchmark - now they are asking IHV's to go outside of the licensing of DirectX in order to "score highest" on the benchmark. (edit - should say "score higher" I guess)

Cheap shot. It depends on how you look at it. Maybe they are just trying to emulate how developers will be using the hardware features that are available. If games are going to employ them then why shouldn't the benchmark? You're right that it was previously a purely DirectX benchmark and now it's not. It's evolved - maybe for the better. Can't remember the last time I played DirectX to tell you the truth

 

[PatrickL]

About same time you played 3dmark i bet

 

[digitalwanderer]

Even if IHVs would feel moved to implement features just for 3DMark05's sake, which I sincerely doubt, this is not a bad thing, since all games with DST-support (quite a number of them, and still growing) will immediately benefit aswell.

Swell, then FM should quit calling it a DirectX9 benchmark and then they can just live up to their motto of "The Gamer's Benchmark".

 

[Dave Baumann]

3DMark2001 covered projected shadows, 3DMark03 covered stencil shadows, 3DMark05 covered depth shadowmaps. What else is there?

Well, there is a mix of both for a start, which is where the big guys sound like they are going to go, and there are smoothies and horizon maps for environment shadowing.

And where exactly weren't the dynamic shadows needed in 3DMark05?

The sun in the third test appears to be a the only source - if the sun moves then you can use horizon maps on the static geometry, if it doesn't then there is no point in wasting any time on the static world geoetry calcuating dynamic shadows (certainly game developers are unlikely to do this).

 

[Scali]

Even if IHVs would feel moved to implement features just for 3DMark05's sake, which I sincerely doubt, this is not a bad thing, since all games with DST-support (quite a number of them, and still growing) will immediately benefit aswell.

Swell, then FM should quit calling it a DirectX9 benchmark and then they can just live up to their motto of "The Gamer's Benchmark".

Why? It's still capable of benchmarking DirectX9 hardware.
Can we not call ATi and NVIDIA hardware DirectX9 hardware because they support non-standard features, even though they support DirectX9?

 

[Infinisearch]

You're missing one.

RENDITION?

 

[Scali]

Well, there is a mix of both for a start, which is where the big guys sound like they are going to go, and there are smoothies and horizon maps for environment shadowing.

I believe smoothies are patented.

The sun in the third test appears to be a the only source - if the sun moves then you can use horizon maps on the static geometry, if it doesn't then there is no point in wasting any time on the static world geoetry calcuating dynamic shadows (certainly game developers are unlikely to do this).

Given the amount of detail in the ship and the seamonster, and their prominent places in view, I don't think the static world geometry is all that relevant to performance anyway. Besides, with this approach you can make the dynamic objects cast shadows on the world with just a single shadowmap, no extra textures or lighting required.
Also, aren't horizon maps specifically for selfshadowing in bumpmaps? Or is there another technique that also goes by the same name?

 

[Dave Baumann]

I believe smoothies are patented.

This indicates their work is "related" to a patent:

http://graphics.csail.mit.edu/~ericchan/papers/smoothie/

Given the amount of detail in the ship and the seamonster, and their prominent places in view, I don't think the static world geometry is all that relevant to performance anyway.

It is if you are needlessly wasting cycles for no good reason. Probably is if you want to reflect "games" as well.