3DMark Vantage numbers - do they hold up

[Valzic]

I have been having a lot of fun reading the speculation threads on both GT200 and R700. The leaks on performance have been 2 kinds.

1. Looking at best guesses on the architecture of new products and combining them of what we know on current architecture, then saying "wow this will be monster or will be fast etc". A bit risky if you ask me as little tweaks may screw performance or "fix' performance on new hardware. Unless AMD and nVIDIA have left the architecture 100% alone, which we know they have no, performance may or may match what we know.

2. Posting new Vantage numbers - especially the "X' numbers. These are supposed to be more 'objective' but may or may not be accurate versus game play.

We all know that winning 3DMark06 does not equate to actual superiority of game play. We have evidence of that and the comments indicate that it is pretty universal held opinion.

My question is how the new 3D Mark numbers reflect on actual game play? Do the numbers "Better" reflect what game play might be or are they just as bad? Saying "You don't play Vantage" seems to skirt around the question on if Futuremark is closer to better reflecting gameplay that the old numbers.

Are the Vantage numbers on current hardware indicative of gaming prowess compared to the old ones of 3DMark06. If so it gives us a peek at a "glimmer" of what the performance should be.

What is everyone's experience?

 

[Freak'n Big Panda]

Vantage clearly stresses the video card so no doubt it's a good indication of performance. All of this 3DMarkXX doesn't represent real game play is mostly BS IMO. Generally a card that excels in 3DMark will excel in game play. Of course it's not 100% accurate (and it never will be as it's impossible) but it's always been close enough to give a rough idea of relative performance which is all that is needed. Anyways the whole argument that benchmarking games is better is quite ridiculous as game benchmarks aren't any more relevant if you're not actually going to be playing the benchmarked game. Each engine has it's own set of bottlenecks and 3DMark is no exception, you just need to know what the bottlenecks are and take them into consideration when evaluating the overall performance of a particular part.

 

[ShaidarHaran]

FBP: R6xx would certainly take issue with the "if a card performs well in 3dmark it will perform well in games" statement.

 

[Valzic]

R6xx would certainly take issue with the "if a card performs well in 3dmark it will perform well in games" statement.

I am aware of those comments and where they come from. But the question is whether the Vantage scores do a better job of identifying products that are like that?

I don't have a 2900 but would running Vantage extreme indicate much better that that card does not perform as well in real games as the old scores showed.

The primary reason I am asking is that we are all being bombarded by raw X9999 scores because frankly we do not have the products to do the actual game play. So either we just say that the Vantage scores mean nothing and ignore the fact that GT280 scores much better than a 4850 in Vantage so saying it is faster and rely on gut feeling or we get a comfort level and say yes the GT280 is faster. And maybe Vantage is better indicator of game play than the old products.

It would seem to me that Futuremark would have taken heart the criticism on net regarding the mismatch of their testing tool to actual game play and attempted to adjust Vantage. So maybe Vantage is closer to game play now..

 

[Freak'n Big Panda]

It's impossible to do that! Unless they license a game engine or something but then it wouldn't be as forward looking as 3DMark needs to be.

FBP: R6xx would certainly take issue with the "if a card performs well in 3dmark it will perform well in games" statement.

I said generally Clearly in the case of r600 it does not fit so well. But given the theoretical abilities of R600 it is scoring exactly what it should be so you can draw some useful conclusions from that. This is a bit of a trend with ATI hardware btw. I remember when the 8500 came out it beat the GF3 in 3dmark 2001 by a healthy margin but lost in the vast majority of games, indeed 8500 was the more advanced architecture and after the drivers matured it was beating the gf3 across the board. I think the reason for this is that 3DMark is a fixed target which makes it easy to optimize for and it's generally forward looking which tends to more closely match ATI's hardware design choices.

Anyways I think Vantage is just as valid as any previous 3DMark and as such shouldn't be ignored just because it doesn't represent actual game play. It's a valid metric for comparing performance.

 

[BRiT]

Anyways I think Vantage is just as valid as any previous 3DMark and as such shouldn't be ignored just because it doesn't represent actual game play. It's a valid metric for comparing performance.

Agreed. It's only a valid metric for comparing performance in 3dMark Vantage.

 

[AlphaWolf]

Agreed. It's only a valid metric for comparing performance in 3dMark Vantage.

Right, just like any game benchmark.

 

[ChrisRay]

I just find it odd that its always 3dmark scores that get leaked.

 

[Valzic]

I suspect access to pre-releases hardware is much more controlled - so running a single test can sometimes be run secretly or the number be leaked. As soon as someone says --- xx runs this fps we get --- well was that with this resolution - what was the AA / AF etc etc,, Stuff best left for proper reviews.

So it seems like these scores like Vantage are just made for leaks