3DMark05 and certain websites

[mrbill]

Scali's preference for "bilinear weighted" is a personal preference. It's not universally recognized to be higher quality.

So some folks prefer the "bilinear weighted" implementation.
Other folks prefer the original PCF algorithm as specified by Reeves et al.

See:
[Reeves1987] William T. Reeves, David H. Salesin, and Robert L. Cook. Rendering Antialiased Shadows with Depth Maps. SIGGRAPH 1987
pp. 283–291.

I'm a purist who argues that there is no meaning to how far you are from a passed or failed sample, therefore the unadulterated PCF is more correct. (For example, if one out of four samples passes, with unadulterated PCF the answer is 0.25, with bilinear weights the answer varies between 0.0-1.0.)

Either way, four samples will be poor quality. (Reeves noted that they normally used at least 16 samples.) So both will look not so good. But four samples is probably a good quality/performance tradeoff right now.

-mr. bill

 

[Spaceman-Spiff]

3DMark05 Review: http://www.hexus.net/content/reviews/review.php?dXJsX3Jldmlld19JRD04Nzg=

 

[Scali]

The Hexus screenshot takes a situation where there is a large sampling problem anyway. But what I dislike about the regular method is the noisy, grainy edges, which are clearly visible in the Hexus screenshot aswell.
The DST edges are smooth. In this case the sampling problem is more or less disguised by the noise, which Hexus classifies as better.
But in cases where there is not such an obvious sampling problem, I suppose most people will prefer a smooth edge to the noisy grainy stuff. I know I do anyway. I consider noise to be bad quality.
I suppose in the case of Hexus' screenshots, I'm more worried about the sampling problem itself, than about the smoothness of the edges.

So now the 'better' is more qualified, I suppose

 

[Rys]

The Hexus screenshot takes a situation where there is a large sampling problem anyway. But what I dislike about the regular method is the noisy, grainy edges, which are clearly visible in the Hexus screenshot aswell.
The DST edges are smooth. In this case the sampling problem is more or less disguised by the noise, which Hexus classifies as better.
But in cases where there is not such an obvious sampling problem, I suppose most people will prefer a smooth edge to the noisy grainy stuff. I know I do anyway. I consider noise to be bad quality.
I suppose in the case of Hexus' screenshots, I'm more worried about the sampling problem itself, than about the smoothness of the edges.

So now the 'better' is more qualified, I suppose

I did wonder about that while sitting in the cinema this afternoon, after I'd written the article! I didn't think about whether it was a worst case scenario in GT3 in 05, which is where the edge stepping is most visible, while I was writing it.

Now that I think about it, I can't really say PCF = worse quality in all cases (it's obviously not), but it's worse (for me anyway) in GT3 in 05, and of course there's the issues about its use when comparing to cards that don't support it.

Rys

 

[Neeyik]

You're right, Rev. You can guess PCF's function by its name and FM's description of its use, but no real explanation of either feature in the whitepaper.

Well, FM can't possibly explain every 3D tech in a 3DMarkXX whitepaper. That should be in a separate document ("3D Dictionary"). A few days ago, after I read the whitepaper, I suggested how it can be improved for better clarity and understanding. Maybe we'll have an updated 3DM05 whitepaper in the future, shrug.

I'm working on getting the entries into the next update for the vocab (on FM's site) but it's well down on a long list of things I've got to do right now. Plus I get to moan about the help file too, since I didn't write this one!

 

[GraphixViolence]

The shadow aliasing in those screenshots is pretty nasty. As Scali said, it's tough to argue the quality of different PCF implementations when it's overshadowed (no pun intended) by such obvious artifacts. I hope this isn't representative of what we will see in future games, because it's a step backward from Doom 3's stencil shadows if it is.

 

[tEd]

The shadow aliasing in those screenshots is pretty nasty. As Scali said, it's tough to argue the quality of different PCF implementations when it's overshadowed (no pun intended) by such obvious artifacts. I hope this isn't representative of what we will see in future games, because it's a step backward from Doom 3's stencil shadows if it is.

there are sure solutions for the aliasing around the shadows

 

[ERP]

The shadow aliasing in those screenshots is pretty nasty. As Scali said, it's tough to argue the quality of different PCF implementations when it's overshadowed (no pun intended) by such obvious artifacts. I hope this isn't representative of what we will see in future games, because it's a step backward from Doom 3's stencil shadows if it is.

there are sure solutions for the aliasing around the shadows

Really? Just wondering what solutions you've seen that don't resort to stencil volumes or very large numbers of shadow maps.

I think you can address some of the aliasing issues, but I'm still not convinced you can resolve all of them in the general case.

 

[tEd]

The shadow aliasing in those screenshots is pretty nasty. As Scali said, it's tough to argue the quality of different PCF implementations when it's overshadowed (no pun intended) by such obvious artifacts. I hope this isn't representative of what we will see in future games, because it's a step backward from Doom 3's stencil shadows if it is.

there are sure solutions for the aliasing around the shadows

Really? Just wondering what solutions you've seen that don't resort to stencil volumes or very large numbers of shadow maps.

I think you can address some of the aliasing issues, but I'm still not convinced you can resolve all of them in the general case.

using 16x filtering samples or higher(nobody said anything about performance )

 

[ERP]

The shadow aliasing in those screenshots is pretty nasty. As Scali said, it's tough to argue the quality of different PCF implementations when it's overshadowed (no pun intended) by such obvious artifacts. I hope this isn't representative of what we will see in future games, because it's a step backward from Doom 3's stencil shadows if it is.

there are sure solutions for the aliasing around the shadows

Really? Just wondering what solutions you've seen that don't resort to stencil volumes or very large numbers of shadow maps.

I think you can address some of the aliasing issues, but I'm still not convinced you can resolve all of them in the general case.

using 16x filtering samples or higher(nobody said anything about performance )

That still wouldn't resolve the extreme undersampling that occurs at surface angle close to the light angle and covers a large portion of the screen. In these cases literally 100's of pixels in screen space map to a single depth map texel.

 

[Smurfie]

My favorite game currently is City of Heroes, which the 6800s seem to do a lot better in.

Me too, but.... the 6800 has real texture shimmering problems in COH, the IQ of the X800 just blows the 6800 away - and I own both cards!

Hmm, my 9600 has texture flashing that gets really bad in Founder's Falls and Brickstown. The texture shimmering seems to occur in both my 6800 and 9600 though. My 6800 should be back soon though.

I am still running the Catalyst 4.8 though, so if 4.9 or 4.10 does resolve that problem, let me know, I will go update my drivers.

 

[tEd]

The shadow aliasing in those screenshots is pretty nasty. As Scali said, it's tough to argue the quality of different PCF implementations when it's overshadowed (no pun intended) by such obvious artifacts. I hope this isn't representative of what we will see in future games, because it's a step backward from Doom 3's stencil shadows if it is.

there are sure solutions for the aliasing around the shadows

Really? Just wondering what solutions you've seen that don't resort to stencil volumes or very large numbers of shadow maps.

I think you can address some of the aliasing issues, but I'm still not convinced you can resolve all of them in the general case.

using 16x filtering samples or higher(nobody said anything about performance )

That still wouldn't resolve the extreme undersampling that occurs at surface angle close to the light angle and covers a large portion of the screen. In these cases literally 100's of pixels in screen space map to a single depth map texel.

I'm no expert on the subject just what i read from experiences from people currently using shadow maps , that 16x sample should already give you much much better quality than what we see in 3dmark05 now. I guess there won't be a perfect solution anytime soon. There are always new optimized shadow map algorithm to improve the situation. PSM vs TSM?

 

[WaltC]

...

A review/preview of a graphics card should ideally have a batter of tests that expose the test product to an increasingly wider range of conditions and scenarios - starting with the totally "theoretical" such as Marko's fill rate tester, to restricted scenarios provided by RightMark and ShaderMark, to full gaming environments found in the likes of Far Cry and yes, 3DMark.

Yes, exactly, and this is the fact that undermines in my view any argument which attempts to rationalize why 3dMK should not be used in a hardware review--because the proper way to use 3dMK in a review is among a wide range of tests, including shipping 3d game-engine performance tests.

The argument used by the "no 3dMk in hardware reviews" crowd is to me essentially dishonest as it argues only why 3dMK shouldn't be used exclusively at the expense of the other kinds of testing that should be done at the same time, such as "real game" testing. But to my knowledge no one's ever recommended or suggested that 3dMk be used exclusively in a hardware review, to the exclusion of everything else, or to the exclusion of "real games" testing. Hence the entire "argument" as to why 3dMk shouldn't be used in hardware reviews is actually a straw man and always has been.

The question that pleads an answer from the "no 3dMK in reviews" crowd is this: What, precisely, is the point to excluding 3dMK testing from 3d-hardware reviews since using 3dMk in no way prohibits the reviewer from running any other testing software he may wish to use and using those results in his review along with the 3dMk results?

It's the one question I've often directly posed to anti-3dMK positions that to my recollection has never been answered when I've asked it. (Though it is certainly easy to see why questions that have no reasonable answers are ignored...)

Interesting to note as well that some hardware sites which exclude 3dMk testing from their reviews on the purported basis that "it's not a real game" often have no qualms at all about including items like the UT2kx fly-by benchmark scores in their reviews although it is not remotely indicative of actual UT2kx gameplay (and was never intended to be.) Then there's the "ShaderMark is A-OK for reviews but not 3dMk" crowd, etc.

Since there's no reason to use 3dMk in a review while excluding ShaderMark and real game testing, or any other testing software, it would appear that the actual objection to the use of 3dMk in reviews is simply that a site simply doesn't like the results the benchmark generates, and the rest of their arguments are essentially spurious fluff designed to obscure the real reason they exclude 3dMk completely, which is that they object to its conclusions.

 

[Scali]

I'm no expert on the subject just what i read from experiences from people currently using shadow maps , that 16x sample should already give you much much better quality than what we see in 3dmark05 now. I guess there won't be a perfect solution anytime soon. There are always new optimized shadow map algorithm to improve the situation. PSM vs TSM?

Futuremark didn't use TSM because it is patented. It remains to be seen if there will be games using TSM. Carmack is not using it anyway. Each solution has its own problems though, it seems.

 

[martrox]

Hmm, my 9600 has texture flashing that gets really bad in Founder's Falls and Brickstown. The texture shimmering seems to occur in both my 6800 and 9600 though. My 6800 should be back soon though.

I am still running the Catalyst 4.8 though, so if 4.9 or 4.10 does resolve that problem, let me know, I will go update my drivers.

ATI knows about the texture problem - it's still there - and says they will be fixing it in a future update. For me, though, the ailising textures are far more annoying and widespread.....

 

[Randell]

For me, though, the ailising textures are far more annoying and widespread.....

and for me too in certain games

 

[martrox]

It's bad enough that a X800 has replaced the 6800 in my main machine......