How benchmark 3D Cards, today

[Altcon]

Well it is in fact much faster than the R520 in a few games like AOE3 for example.
Not wanting to see the theoretical shader performance of a part with over double the alus is just plain lazy imo.
Shader benchmarks also show why the NV3X sucks in DX9 games.
Using those shaderbenchmark you can predict how the NV3X plays DX9 games... very badly

It's a pity you don't read what I write, you might have something to respond to.
As I said (and you even quoted me) I don't think the technical benchmarks should be discarded, I think a moe personal subjective aspect should be added to the review process.
I also don't believ the timedemo's provide much accurate infromation.

Lastly I don't understand why this has turned into and ATi - Nvidia debate (piss war). I think the method should be applied to BOTH parties regeardless and that the reall winner would be the consumer.

Notice in the end the examples you cite are GAMES, because you too know- it's the only thing that REALLY counts.

 

[Altcon]

There are up sides and down sides to all of these methods, but taken as a triumverate they cover pretty much every aspect of a board and architecture. No one site has time to cover all of these aspects in the real world, thus read a review from each of these three 'disciplines' and I would say you're likely to end up with a pretty good grounding in what's good and what's bad about a particular board.

I think you're right, but imagine what happened if you had a review at B3d that covered the subjective aspect, wouldn't that add to your experience?
I must say i haven't read many of your reviews, I keep forgetting EB ...
I do remember they were always pretty good (though they always smelled a bit RED) .

Also, I think everyone has sort of ignored IQ lately, anyone whos says 6xxx-7xxx is fine at driver defaults is blind. But, I haven't seen any reviews trully bashing nVidia for it past 77.77 drivers, I find that odd.

 

[jb]

I think the out of the box experince has some merits to reviews. I know a lot of my friends have no idea that they can change things like AA/AF in their driver options...doh!!!

 

[Moloch]

It's a pity you don't read what I write, you might have something to respond to.
As I said (and you even quoted me) I don't think the technical benchmarks should be discarded, I think a moe personal subjective aspect should be added to the review process.
I also don't believ the timedemo's provide much accurate infromation.

Lastly I don't understand why this has turned into and ATi - Nvidia debate (piss war). I think the method should be applied to BOTH parties regeardless and that the reall winner would be the consumer.

Notice in the end the examples you cite are GAMES, because you too know- it's the only thing that REALLY counts.

Apparently I overestimated your ability to see the point I was making.
Theoretical benchmarks can easily predict future performance- point I was making, in plain no read between the lines bullshit.
And everyone knows (or should) that timedemos aren't a propery way to benchmark, fraps is the only away to benchmark games and get real results- omg something the [h] does OH NOES

The R580's theretical advantage over the R520 is showed both in shadermark (although not 3x as fast) and in games which rely heavily on pixelshaders.
Therefore I think both games and theoretical performance should be given equal weight, not just games, or not just synthetic benchmarks.
I did read your post, I'm sorry if I didn't give the response you had hoped for.
This isn't n ati vs nvidia war either, if I so firmily believed ati had better products I wouldn't have gotten nvidia for the last two upgrades.
The fact of the matter is that the 6600GT and 7800GT offer the best performance for the price at the times i purchased them.
I mean I paid 309 bucks (plus tax) for a graphics card that is well over three times faster than my previous card which I paid 100 less.
For my 6600GT the only ati option in the 200~ price bracket was the 9800 which is slower in every modern game, and for the 7800gt a 1800XL was much more expensive.

 

[bigz]

I think that generally, all websites that review video cards complement each other in some way or another, and it's always pleasing to read that my subjective analysis falls inline with the views of editors that use more scientific methods of testing.

Also, FWIW, I noticed a comment about 'an unnamed site that doesn't post frame rates anymore' - we didn't for a while, but then we decided that they were of some use and we've used them in our reviews since the start of December. We've looked at several ways of presenting those results, but felt that there were some issues with using the line graphs that Kyle/Brent use (they're quite hard to understand in my personal opinion). That's why we've ended up with a table of settings and a table displaying minimum & average frame rates.

As always though, we're still trying to improve our own methods of testing, analysis and evaluation.

The major krux for us is always the time it physically takes to benchmark/evaluate a graphics card, especially when there's a tight time schedule around a new product launch - I've worked something like 21 consecutive days and my next day off is a week today (yesterday in the UK). It's just that time of the year at the moment, because things are getting pretty hot at the moment. There have been times during the last few weeks where we've had discussions about moving to a fully automated timedemo routine, because it'd free up a lot of time for me to be getting on with other things. However, I've stood firm by our testing methods, despite my boss wanting me to 'save time' using more conventional and scientific benchmarking techniques. I'm prepared to do the time, as such. I wouldn't do that if I didn't think there was at least some value in what we were doing.

 

[Moloch]

I think that generally, all websites that review video cards complement each other in some way or another, and it's always pleasing to read that my subjective analysis falls inline with the views of editors that use more scientific methods of testing.

Also, FWIW, I noticed a comment about 'an unnamed site that doesn't post frame rates anymore' - we didn't for a while, but then we decided that they were of some use and we've used them in our reviews since the start of December. We've looked at several ways of presenting those results, but felt that there were some issues with using the line graphs that Kyle/Brent use (they're quite hard to understand in my personal opinion). That's why we've ended up with a table of settings and a table displaying minimum & average frame rates.

As always though, we're still trying to improve our own methods of testing, analysis and evaluation.

The major krux for us is always the time it physically takes to benchmark/evaluate a graphics card, especially when there's a tight time schedule around a new product launch - I've worked something like 21 consecutive days and my next day off is a week today (yesterday in the UK). It's just that time of the year at the moment, because things are getting pretty hot at the moment. There have been times during the last few weeks where we've had discussions about moving to a fully automated timedemo routine, because it'd free up a lot of time for me to be getting on with other things. However, I've stood firm by our testing methods, despite my boss wanting me to 'save time' using more conventional and scientific benchmarking techniques. I'm prepared to do the time, as such. I wouldn't do that if I didn't think there was at least some value in what we were doing.

Sorry, haven't read your site since then
the [T] *cough* excuse me, [H] method of tasting is indeed a poor one since you have cards running at different settings on the same graph.

 

[Reverend]

Can't believe I missed a thread like this that's close to my heart!

To address the first/original poster :

I'm very bored to see false, incorrect benchmark

If a reviewer doesn't make any mistakes then the numbers/fps depicted in his review can't lie; doesn't matter if his numbers are based on a demo run that doesn't represent 90% of the game's performance or not.

BTW, I didn't read the rest of your post beyond what I quoted above... too damned lazy

 

[Reverend]

Nope, I think I'd really love to try and do a review sometime that had absolutely no benchmarks whatsoever. Purely subjective from start to finish and aimed soley at describing the experience with the card in a way that requires almost zero technical knowledge to understand yet at the same time shows the strengths/weaknesses of the part.

No matter how you choose to do your theoretical review, "experience with the card" must come down to performance. You may say something like "When I turn to shoot that monster at the higher rez/AA/AF/wtf setting(s), I noticed no perceptible slowdown that caused my health to go down significantly that tiny bit if I were at a lower graphics settings" but you can never guarantee that such a subjective observation/experience is The Rule simply because a lot of folks have different thresholds, not to mention how subjective the word "significantly" is, and not just in terms of depleted health bars (something a lot of gamers just cannot accept because they think they are the perfect gamer).

IOW, I'd just as soon read a review of a card that has no technical explanations for whatever benchmarks offered (but as long as benchmarks are provided) than a review thatis a game review posing as a hardware review.

There is only one real reason why folks upgrade their card and your idea of such a review surely cannot help them.

Unless, of course, you have been going at it for a long time and a lot of folks have come to trust and believe that your "experience of the card" will mirror theirs 90% of the time in 90% of the games in 90% of the systems out there.

Anyway, it's not terribly surprising to read how you wish to write reviews.

 

[digitalwanderer]

No matter how you choose to do your theoretical review, "experience with the card" must come down to performance. You may say something like "When I turn to shoot that monster at the higher rez/AA/AF/wtf setting(s), I noticed no perceptible slowdown that caused my health to go down significantly that tiny bit if I were at a lower graphics settings" but you can never guarantee that such a subjective observation/experience is The Rule simply because a lot of folks have different thresholds, not to mention how subjective the word "significantly" is, and not just in terms of depleted health bars (something a lot of gamers just cannot accept because they think they are the perfect gamer).

IOW, I'd just as soon read a review of a card that has no technical explanations for whatever benchmarks offered (but as long as benchmarks are provided) than a review thatis a game review posing as a hardware review.

There is only one real reason why folks upgrade their card and your idea of such a review surely cannot help them.

Unless, of course, you have been going at it for a long time and a lot of folks have come to trust and believe that your "experience of the card" will mirror theirs 90% of the time in 90% of the games in 90% of the systems out there.

Anyway, it's not terribly surprising to read how you wish to write reviews.

Aw, thanks Rev!

 

[Reverend]

HardOCP now contracts 20 independent editors and folks in various other positions in three countries to help in delivering a variety of opinions.

I admire the fact that you chosed the last word in the quoted sentence above.

To address your thoughts on "too much technical things," Brent Justice might disagree with you on that when it comes to coverage. I think Brent does a great job covering the technical merits of new video card technologies. Without a doubt though, gameplay experience is our focus when it comes to video cards as we think that is what most people want to know. Please do note that we link most all Beyond3D technical write-ups so our readers are aware where they might find more in-depth reading on the topic GPUs.

Your last sentence is not required since you already have a target audience for your reviews. To be honest, I rarely read Brent's "technical pages" on video cards because they are rarely anything other than what the IHVs provide to media outlets.

I have always practised writing reviews within my limits [edit] and have never saw it fit to think my target audience should require more materials to read than what's available at my own site. My apologies for being an extremely single-minded individual in what I set out to do! [/edit]

 

[digitalwanderer]

I have always practised writing reviews within my limits.

"practiced"

 

[Moloch]

Corrected by a stoner... wow..

 

[Reverend]

Aw, thanks Rev!

Unfortunately and contrary to your apparent appreciation of my post, my comments weren't meant the way you apparently took them to be.

You will never make a good or relevant video card reviewer. Sorry but that's my opinion. I'm sure many will disagree but that's what forums are for, yes?

 

[Geo]

Whupsie, how impolite of us! One should always provide a reciprocal to one's traffic partners!

http://www.hardforum.com/showthread.php?t=1027523&page=1&pp=20

 

[Moloch]

Whupsie, how impolite of us! One should always provide a reciprocal to one's traffic partners!

http://www.hardforum.com/showthread.php?t=1027523&page=1&pp=20

Um when was it deleted and put back?

 

[Geo]

Wavey mentions it in that thread over there somewhere! One of the mods here felt it was headed south, and OT, so pulled the plug, not realizing it was linked from [H] front page. Then when Wavey w/o/k/e//u/p/ discovered it, he brought it back.

 

[digitalwanderer]

Unfortunately and contrary to your apparent appreciation of my post, my comments weren't meant the way you apparently took them to be.

You will never make a good or relevant video card reviewer. Sorry but that's my opinion. I'm sure many will disagree but that's what forums are for, yes?

Yup.

For a second there I was worried you didn't like me or something.

 

[panzeramd]

I don't have anything add really, But i do think that pictures showing image quality of a video card ( and any difference with it's competitor ) is the most important part of a review.

Their will always be problems with that i guess, since games like HL2 or Quake4 have a limted play area, AF ( and mip-map ) differences won't correspond to games with a bigger play area like for an example Morrowind ( i still play it ), so testing some games that aren't part of the list of the most popular games at the moment is needed, of course the games tested would have to other differences ( play area, terrain type, game type ) to be broad enough to represent what most gamers play. What i mean is that i have Quake4, farcry, HL2 but they are not the only types of games i play ( hope that makes sense ).

You will never make a good or relevant video card reviewer. Sorry but that's my opinion. I'm sure many will disagree but that's what forums are for, yes?

That's a bit harsh Rev. One thing a Digi review would be though, it would be funny ( just going by his posts ).

I managed to post and be nice in a thread that a certain web master had also posted in, where do i get my mickey mouse badge

Just to add : i have to recharge the batterys in my mouse and KB so sorry if this post doesn't make any sense.

 

[Kombatant]

Digi is better when he interviews himself

 

[Mendel]

How make thread titles, today