The truth isn't very useful for measuring NVIDIA cards-BJORN

[Vortigern_red]

oh my, what a tangled web you weave....
Benching a game provides more than just information regarding how your systemw ill run that game.
If you know the game is bottlenecked by shader calcs, for instance, then you can easily exprapolate which card has better shader performance....

Ah, A tangled web indeed

If the game/benchmark has App specific "optimisations" it is quite difficult to use that data for reference to other games/Apps, No? Unless of course you assume that all games/apps have the same level of "optimisation".

 

[digitalwanderer]

oh my, what a tangled web you weave....
Benching a game provides more than just information regarding how your systemw ill run that game.
If you know the game is bottlenecked by shader calcs, for instance, then you can easily exprapolate which card has better shader performance....

Ah, A tangled web indeed

If the game/benchmark has App specific "optimisations" it is quite difficult to use that data for reference to other games/Apps, No? Unless of course you assume that all games/apps have the same level of "optimisation".

You mean that the nVidia cards run TR:AOD so crappy WITH cheats included?!!?

 

[ByteMe]

(Ok, my real thoughts on it can be found here...but I didn't think them apprapo for B3D.)

You shouldn't hold back so much. What is the worst that could happen?

 

[Althornin]

oh my, what a tangled web you weave....
Benching a game provides more than just information regarding how your systemw ill run that game.
If you know the game is bottlenecked by shader calcs, for instance, then you can easily exprapolate which card has better shader performance....

Ah, A tangled web indeed

If the game/benchmark has App specific "optimisations" it is quite difficult to use that data for reference to other games/Apps, No? Unless of course you assume that all games/apps have the same level of "optimisation".

well, first off, it wouldnt be the "game/benchmark" that has the application specific optimizations, it would be nVidias drivers.
Secondly, when said performance lines up almost exactly with what is predicted by synthetic benchmarks....

 

[see colon]

Benching a game provides more than just information regarding how your systemw ill run that game.
If you know the game is bottlenecked by shader calcs, for instance, then you can easily exprapolate which card has better shader performance...

isn't that what i just said? it's worthless as a game benchmark and usefull as a synthetic benchmark.
c:

 

[PeterAce]

oh my, what a tangled web you weave....
Benching a game provides more than just information regarding how your systemw ill run that game.
If you know the game is bottlenecked by shader calcs, for instance, then you can easily exprapolate which card has better shader performance....

Ah, A tangled web indeed

If the game/benchmark has App specific "optimisations" it is quite difficult to use that data for reference to other games/Apps, No? Unless of course you assume that all games/apps have the same level of "optimisation".

You mean that the nVidia cards run TR:AOD so crappy WITH cheats included?!!?

The TR:AOD shaders have been posted to this forum before - they had a very high number of '_pp' (FP16) hints.

This really shows the weakness of the NV3X shader performance (much lower than R3X0 even with '_pp' hints!

 

[Vortigern_red]

well, first off, it wouldnt be the "game/benchmark" that has the application specific optimizations,

Yes sorry, I meant if the game/app is subject to app specific optimisations by the drivers. I will try and be more careful with my wording in future.

it would be nVidias drivers.

Or anybody elses drivers

I was talking in a more general sense, about the problem of taking game benchmarks and assuming that the data represents anything other than performance in that one game, if that game is subject to app specific optimisations because you don't know whether any other game has the same (or any) optimisations.

Secondly, when said performance lines up almost exactly with what is predicted by synthetic benchmarks....

Does it? for all optimised software?

I see you are talking about TRAOD but I was taking your comment:

Benching a game provides more than just information regarding how your systemw ill run that game.

To be more general than just TRAOD, sorry for any confusion.

 

[Althornin]

I see you are talking about TRAOD but I was taking your comment:

Benching a game provides more than just information regarding how your systemw ill run that game.

To be more general than just TRAOD, sorry for any confusion.

no problem, you are right.
If there are cheats, then the benchmark/game tells us only how it performs in that game.

 

[cthellis42]

Secondly, when said performance lines up almost exactly with what is predicted by synthetic benchmarks....

Does it? for all optimised software?

I see you are talking about TRAOD but I was taking your comment:

Pretty much all that was coming out, to different degrees. But yes, it showed for TR:AoD, with first glimpses at Halo, with the first HL2 benchmarks, from developer comments coming out perviously, or when they would comment on the situations that got enough attention... Things don't line up exactly, but you're never going to find ANY benchmark that will line things up exactly between different programs (synthetic, canned game, FRAPSed game, or what have you), but the connections certainly lined up. The objectionable part with Futuremark has really been regarding their handling of the whole mess, not the performance of the benchmark itself; most people who complain about it bitterly from that angle are rarely looking at the results but rather repeating whatever phrases they've heard elsewhere that they think applies.

Meanwhile, has anyone tried FRAPSing TR:AoD lately to see how playthroughs compare now with their most up-to-date patches, rather than just relying on the benchmark?

 

[digitalwanderer]

Meanwhile, has anyone tried FRAPSing TR:AoD lately to see how playthroughs compare now with their most up-to-date patches, rather than just relying on the benchmark?

No, no one will actually PLAY that pos game....I don't think they could pay me enough to play it!

 

[cthellis42]

C'mon, it'd just be an interactive benchmark!

 

[digitalwanderer]

C'mon, it'd just be an interactive benchmark!

Pffft! It kills the reliability of the benchmark to allow free-will, they shouldn't have marketed TR:AOD as a game.

 

[cthellis42]

If you test a method enough times and get minimal sway, it's certainly good enough. I'd trust it more than random flybys and canned developer benchmarks that may use the same engine, but not stress a card quite the same way PLAYING the game does. (Yet another thing that makes me laugh when people snipe blindly at synthetic benchmarks, but trust any game bench implicitly.)

In the end, it's all just more information. More information = good! ^_^

 

[Pete]

Seriously, how precise do we need benchmarks, anyway? Sure, FRAPS is less precise than 3DM, but any framerate differences below 5fps are probably unnoticable, and any benchmark where a 5fps difference is considered great is probably too slow to be veyr playable (IMO). I'd rather have FRAPS numbers that are +/- a few fps, than synthetic numbers that are +/- a few tenths of a frame yet under suspicion of cheating.

OTOH, custom demos used with a benchmarking framework seem fine. Unless IHVs can detect a benchmark mode and reduce IQ to gain speed....

 

[Tim Murray]

OTOH, custom demos used with a benchmarking framework seem fine. Unless IHVs can detect a benchmark mode and reduce IQ to gain speed....

Unless they optimize for specific demos, I don't see how they could do this... but if they do, just keep demos private, problem solved (at least in theory). But yeah, if there's a game you like that doesn't have demo recording or benchmark modes, start sending email to the developers. It benefits everybody.

 

[digitalwanderer]

<sigh>

If all games would just ship with benchmarking features that included the ability to make/record custom demos for benching the graphics world would be a muchly better place.

</sigh>

 

[see colon]

But yeah, if there's a game you like that doesn't have demo recording or benchmark modes, start sending email to the developers. It benefits everybody.

i'd rather they spent more time fixing bugs and tweaking games for performance than have them add a benchmarking mode.
c:

 

[cthellis42]

Because inserting the ability to record one's active gameplay would be an utter chore? Heck, they don't even have to create their own benchmarking utilities, just allow for recording and run-throughs--FRAPS can do the rest. Anything that want to add after that would just be icing, but "the cake" in this situation would solve OH SO MANY of the problems we have now.

 

[Quitch]

i was just comenting on the relevance of the benchmark itself. sure, benching tr:aod is a fine way to tell how fast your card/system will run tr:aod, but how many people are actualy playing the game? i could bench the hell out of chasm:the rift all day but it deosn't make the game relevant to any decent amout of gamers out there, who will not be playing the game anyway.
c:

Uh, it's a top selling PC game. A *lot* of people are playing it.

Mind you, I fail to see wha the popularity of a game has to do with its use a benchmarking tool. Does a game being unpopular mean its shaders are less PS 2.0? Does popularity directly corrolate with the quality of the shaders or number?

I doubt it.

 

[digitalwanderer]

Uh, it's a top selling PC game. A *lot* of people are playing it.

I dunno. I can't argue that it's a top selling PC game, I just have a feeling that it's the most bought game that people never play.

Seriously, I haven't read a whole lot of people talking about this game on ANY forums...at least not talking about playing it, just benching.