How benchmark 3D Cards, today

[FrgMstr]

2. Subjective analysis (a la [H]ard|OCP)

Now hang on, you shifted gears on me. Last October you said, and I quote:

[H]'s current vendetta against ATI has completely overshadowed their ability to report objectively.

And then you said:

[H] are now completely out of touch with the community and market they purport to represent.

Do you still stand by these statements? I have to ask as it fully impacts whether I derive ANY value from your opinion or not. It is my opinion the above statements are ill-informed and irresponsible at best.

 

[Hanners]

Wow, someone had certainly pissed in my cornflakes that morning! In retrospect, that piece reads as being overly harsh to me, although you were way, way off base with your comments about the X800 GTO that I was referring to at that time.

So, as a snapshot of my thoughts regarding your website's dealings towards ATI at that time, do I retract what I said? No, absolutely not, although I shouldn't have worded them so strongly, that was admittedly a bit of a knee-jerk reaction on my part. Would I say those things about your site today, or even a month or so after I made those statements? No, I wouldn't.

I'm kind of baffled as to why a tonne of bricks has come down on me from your direction regarding an ancient news post out of blue, but oh well... I'll change the post that inflamed your ire to mention bit-tech as an example of subjective benchmarking instead, which will hopefully cause less offence.

 

[Kanyamagufa]

Setting all the drama about who favors who aside for the moment, I really do appreciate the approach [H] takes when reviewing GPUs. It's refreshing when compared to how so many other sites deal with the matter.

 

[Geo]

I am not overly fond of how [H] bench but I am glad they are doing it because they add variety to the type of reviews out there; no doubt somebody hates what I like in benches and they might be better catered for looking at the [H] review.

I would agree with that, actually. I think the community mosaic is rich enuf to support some variation in method. And, naturally, there will be proponents and opponents of this methodology or that. Goes with the territory and all the alpha males in da house.

 

[Geo]

Now hang on, you shifted gears on me. Last October you said, and I quote:



And then you said:



Do you still stand by these statements? I have to ask as it fully impacts whether I derive ANY value from your opinion or not. It is my opinion the above statements are ill-informed and irresponsible at best.

What's the relevance of the GTO imbroglio to your testing methodology tho?

It certainly would have been kinder for Hanners to have admitted the possibility of a #3 --a momentary bout of head-up-assness. It happens to everyone, with the speed with which things move in this industry/community as a major contributor. Tho I suppose if you're asking that be granted you, then you ought to offer it up elsewhere as well.

 

[kinab]

Optimizations

Apparently you're either blind or don't own an nvidia card.
Since [h] is supposed to be about gaming, how many people with nvidia cards game with them on quality setttings?
The first thing I did when I got my 6600GT was turn the opts off, same with my 7800GT, except the 7800GT has worse filtering :smile:
I can't stand the IQ with the opts on and even off they could very well could stand to improve.

The difference between using SSE is that using SSE doesn't degrade the quality, it speeds it up with no loss in "IQ" so why not use SSE1/2/3?
It speeds things up with no downsides if coded properly.
Nvidias opts are very visable.

Btw that's a bad first post since you revealed you either have no experience with graphics card or need glasses.

Radeonic - I have personally owned the following 3d cards in my system, rendition verite v2100 (as in vquake - before glquake), 3dfx voodoo 1, 2x3dfx voodoo2x12mb, 3dfx voodoo3, nvidia tnt2, nvidia geforce 2mx, radeon 8500, radeon 9600xt, nvidia geforce 5900xt, ati x800 pro. I'm as old-school in consumer 3d graphics as you can be. I stand by my statements, optimizations are fine IF they don't change the way the game looks. Try reading my post before flaming me. I know most nvidia optimizations and default settings are bad. If you know that, and it bothers you, then buy ati next time.

----------------------

There really is a place for both types of benchmarks though. Numbers benchmarks let you get a feel for the relative performance between cards, hardocp's reviews give you an idea of what to expect if you actually bought the card. Personally I like more variety in reviews.

 

[Kombatant]

Now where's that popcorn smiley when you need it...

 

[digitalwanderer]

 

[QuakerOatz]

Not much to see here but a bit of a pissing match:
More detail is better! Less detail is better!

Where it takes a turn is when some decide that a lack of detail indicates a lack of intelligence or the significance of a review.
It's called engineers block, where those so familiar with technical data they become removed from a users experience (joe gamer).

Ex. You shouldn't benchmark on popular games you should use games few people play and this information will be useful to the 95% of people who play the mainstream games you didn't benchmark.

Remember, while you look down upon brief, less technical reviews. The most powerful CEOs, Chairmen, CTOs in the world are reading brief non-technical reports of similar nature. There's a reason!

And it's not because you're smarter than them!

 

[FrgMstr]

What's the relevance of the GTO imbroglio to your testing methodology tho?

It certainly would have been kinder for Hanners to have admitted the possibility of a #3 --a momentary bout of head-up-assness. It happens to everyone, with the speed with which things move in this industry/community as a major contributor. Tho I suppose if you're asking that be granted you, then you ought to offer it up elsewhere as well.

I have no idea what the GTO has to do with this. (And as a note, to this date, we have NEVER been briefed on the GTO being an official ATI skew. At the time that I made my statements about the GTO, ATI had never informed us of its existence. Brent and I thought that it was simply another AIC partner assigned part designation. Please feel free to verify that with ATI.) I felt as though that Hanners' past statements are brought up in a very valid reference to HIS objective stance about HardOCP practices. I like to listen to opinions of lots of folks, but when the person making the statements has in the past referred to us having a "vendetta against ATI" and also being "completely out of touch with the community," it is tough for me to assign ANY value to that person's opinion. Hanners' past statements are broad and sweeping and hardly confined to the GTO part as he tried to make it seem.

I did explain myself in my line of questioning. Here is my quote from above.

Do you still stand by these statements? I have to ask as it fully impacts whether I derive ANY value from your opinion or not. It is my opinion the above statements are ill-informed and irresponsible at best.

I think after making statements such as that, they should be called into reference in this conversation as it shows to Hanners’ true character and his underlying thoughts of HardOCP's value and direction. His past statements have been egregious at best. I know he is a respected forum member here, but I think it is worth bringing up as it may show some true colors. And yes we linked this on the HardOCP portal page, so I want HardOCP to get a fair shake here.

 

[Hanners]

Hanners' past statements are broad and sweeping and hardly confined to the GTO part as he tried to make it seem.

Perhaps my comments at that time may have sounded broader than simply the issue I was reporting on, but that certainly wasn't my intention - It was purely a reflection (albeit, as I've already said, perhaps an overly harsh one) of how I felt you were dealing with a particular situation at that particular time.

Anyhow, now everyone has seen my 'true colours' (apart from eating live babies, they don't know about that part yet ), I dont want to derail what is an extremely useful discussion any further. You don't respect my opinion with regard to the subject at hand, and I can totally accept that. For the sake of this thread though, I think any more exposure of said 'true colours' would be best taken elsewhere.

 

[Kombatant]

Kyle, with all due respect mate, he did say (bold by me):

This news post is proof of one of two things:

1. [H]'s current vendetta against ATI has completely overshadowed their ability to report objectively.

2. [H] are now completely out of touch with the community and market they purport to represent.

so it's pretty obvious he was talking about the said GTO piece. Was he too broad and harsh? Yes. But I believe he was clear about why he made that conclusion.

 

[digitalwanderer]

I like to listen to opinions of lots of folks, but when the person making the statements has in the past referred to us having a "vendetta against ATI" and also being "completely out of touch with the community," it is tough for me to assign ANY value to that person's opinion.

Well except for your having had a vendetta against ATi and for being pretty much completely out of touch with the community in the past I'd understand where you're coming from...

 

[Geo]

I have no idea what the GTO has to do with this. (And as a note, to this date, we have NEVER been briefed on the GTO being an official ATI skew. At the time that I made my statements about the GTO, ATI had never informed us of its existence. Brent and I thought that it was simply another AIC partner assigned part designation. Please feel free to verify that with ATI.)

Well, the post of Hanners you are pointing at was clearly in reference to an [H] news piece re GTO. Now, in my experience the littler fish tend to have an exalted opinion of the resources of the bigger fish. And sometimes they might not even recognize that a given post might have other fish to fry (to change fish context in mid post!) as well.

In retrospect, I read the [H] post as at least as much about rapping ATI PR knuckles for not keeping [H] better informed as anything else.

 

[Kombatant]

Oh, and Kyle? Just because ATI hadn't informed you of the GTO's existence, it doesn't mean it would never exist. There were many online publications that were briefed about it back then, perhaps you could have simply sent an email to them and asked before coming out with a piece like that.

 

[John Reynolds]

And yes we linked this on the HardOCP portal page, so I want HardOCP to get a fair shake here.

Translation: I want my site's f@nboys to flood this message board.

And your diatribe on outing Hanners' "true colors" is odd. I've both praised and harshly criticized your actions in the past, Kyle, depending on the circumstances. Have my true colors been revealed? Does someone really have to have an agenda (true colors) to dare criticize Kyle/[H]? Is it conceivable that a person might have an opinion without some nefarious agenda at work?

 

[Dave Baumann]

Lets keep the thread on the topic at hand please.

 

[Moloch]

Radeonic - I have personally owned the following 3d cards in my system, rendition verite v2100 (as in vquake - before glquake), 3dfx voodoo 1, 2x3dfx voodoo2x12mb, 3dfx voodoo3, nvidia tnt2, nvidia geforce 2mx, radeon 8500, radeon 9600xt, nvidia geforce 5900xt, ati x800 pro. I'm as old-school in consumer 3d graphics as you can be. I stand by my statements, optimizations are fine IF they don't change the way the game looks. Try reading my post before flaming me. I know most nvidia optimizations and default settings are bad. If you know that, and it bothers you, then buy ati next time.

----------------------

There really is a place for both types of benchmarks though. Numbers benchmarks let you get a feel for the relative performance between cards, hardocp's reviews give you an idea of what to expect if you actually bought the card. Personally I like more variety in reviews.

You're missing or ignoring my point.
The optimizations DO affect IQ and there is no way you can tell you can't tell the difference unless you need glasses.
So why would you want to see what FPS a card offers when it's giving subpar IQ that people don't game with?
It would be nice if a site would benchmark a bunch of games with various optimization settings.

As for buying ati next, I just may because I'm on my third nvidia card and its an odd number
I went GF2mx-8500-6600GT,7800GT.
I've wanted to go ati since the 6600GT but ati didn't offer a car that was comparable for the price.
As for older cards (for the whole e-penis old schooler thing you've pulled) My first 3d card was a lil old voodoo2
voodoo2-voodoo3-GF2mx... you know the rest.
Guess I'm not old school enough to have my opinion mean anything

 

[Sandwich]

You got me wrong about it. I said that the best games for benchmarks are less known games. There are many (3D) games out there that support SM 3 and DX9 but they are not that famous like COD II or FarCry.

Excellent idea.
Wouldn't have to be new titles either. Benchmarking games that never got benched previously on major sites, could be a good way to measure real-world performance without specific driver optimizations.
This can also be very helpful to gamers who don't play just the major shooters and a handful of other top10 titles, but favor more obscure RTS, simulation or crpg titles.

Maybe this would be more meaningful for mainstream and budget cards, that may still struggle with games that don't do cutting-edge gfx.
I think dropping older benchmarks for any card performing less than a 6800 is a bad idea in any case. IMO if you can't run a fast-paced game above 100fps avg at the desired IQ settings, performance differences matter.

 

[Druga Runda]

what a short thread (I expected 100 posts at least thus far )

-----------

I really appreciate the way [H] at the moment creates the reviews, + this newly found will to listen

. It is good that their reviews are different from most other sites out there, and give you a nice feeling of what you can get when you can play particular games that they test on. I like the idea of "max playable settings" too as you can assume that lower resolution will play fair bit better for you and the amount of testing is in all likeleyhood lessend for the reviewer so he can perhaps focus to get more games into review/use the time differently to add more info.

My only question is about IQ, as some people are more sensitive to this, so ideally the same settings should be used which might not correspond to actual names on settings between different IHV's, that might be the only quirk, but again most of the readers cannot see the difference anyway so if the reviewer is happy with what he sees that is kind of "fair game". I guess it will be up to other sites/reviwers who are not happy with what they see to show it and bring up such quality differences or otherwise risk the wrath of dissapointed readers who bought the stuff. Whether you need glasses or not if you don't see the difference is another matter ... still tolerating the optimizations is in a way slippery slope leading to all sort of stuff from IHV's to make sure the FPS on the graph is the highest resulting in correspondingly higher sales for the company.