[H] Benchmarking Future Ed.

[Yonex]

Thank you Mariner.

Very good posts from BrainPimp over on that forum. It looks as though he must have been a poster there for quite some time and he doesn't appear to have any axe to grind.

I'll keep an eye on that particular thread to see if Kyle does respond to any of his points.

The problem is that Kyle is not just "defending the indefensible", he's "not even mentioning the indefensible!" - especially on his own web site.

Hello everyone at B3d. Very astute observation Mariner. I am, in fact, a long time reader of [H] . I also have no axe to grind. I own equipment from both companies. This will probably get me banned over there in spite of the fact this is all totally correct and factual and on topic.

I do not understand nor condone the apparent blatant onesided "lack" of reporting going on. I like Kyle very much. I have admired and responded to his style of the tell it like it is in the hardware industry for several years. I read multiple sites and have never failed to get a straight answer for any question that has arisen. That makes this incident all the more confusing. This is my opinion and mine alone. The actions of the ownership of [H] are bordering on ridiculous. The blatant refusal to address direct specific questions is most telling in my mind. On several threads the "gauntlet was thrown down to just ask, nothing is being avoided." Nothing was addressed. So I finally just asked. I was told I was free to read other sites. The total and complete avoidance of any direct specific relevant questions was avoided. I tried to be open minded and posted new specific questions relating directly to a post by Kyle himself.

My response................................................................silence.

The total, utter, and complete avoidance of any of the pointed direct questions.

My post: with quotes from his post:
Kyle ..Why?
quote:

Originally posted by FrgMstr
I have said it before and I will state it again, this situation is not as simple as Quack was a couple years ago. The game is totally different and quite honestly, all the bitching and complaining in the world is not going to change NVIDIA's actions. It is a DONE DEAL.

Why is it a done deal? What is it about nvidia that makes them unable to be approached with this information?

Why would bringing this information public not have an impact?

When did you know this information?

What do you know Nvidia is doing?

These are some of the questions I would liketo see addressed by [H].
quote:

Originally posted by FrgMstr
Exposing Quack changed things, I knew months ago exposing what NVIDIA is doing would not change the way they are doing anything. At that point, I decided to move forward and not waste our energy spinning our wheels. I can only wish, as much energy would have been put into solving the problem as there has been about griping about it and exposing it.



How could readers spend the same amount of energy solving the problem?

This is the part that blows me away. Here you state you knew what Nvidia was doing months ago. What? You knew about his months ago and said nothing to us? If this is true I am stunned. How could you possibly know this and not post it on your front page and seriously look into re-evaluate at the very least the most recent reviews?

You continued to post flawed benchmarks?

I want to make sure I got this right. That is huge news.
quote:

Originally posted by FrgMstr
Exposing Quack changed things, I knew months ago exposing what NVIDIA is doing would not change the way they are doing anything. At that point, I decided to move forward and not waste our energy spinning our wheels.



How could you know about this for months, Kyle and not tell us? Whether it would change a damn thing with Nvidia is irrelevant. We your readers would have known.

I am very interested in this story. I think what s good for the goose is good for the gander. I agree ATI should have been called on the carpet. I also think Nvidai should be called ont he carpet just as harshly. For whatever reason [H] does not want to do it. Kyle's excuse of it will not do any good so I won't say anything is one of the weirdest excuses I have ever heard for the owner/author of a hardware review site.

Honestly, I am confused. Unless something is going on that we don't know about the blatant actions defy logic and explanation and the silence leads one to some very unpleasant conclusions.

Sorry to bomb your thread but I just don't feel like this is going to get discussed over there.

 

[digitalwanderer]

Re: Thank you Mariner.

Very good posts from BrainPimp over on that forum. It looks as though he must have been a poster there for quite some time and he doesn't appear to have any axe to grind.

I'll keep an eye on that particular thread to see if Kyle does respond to any of his points.

The problem is that Kyle is not just "defending the indefensible", he's "not even mentioning the indefensible!" - especially on his own web site.

Hello everyone at B3d. Very astute observation Mariner. I am, in fact, a long time reader of [H] . I also have no axe to grind. I own equipment from both companies. This will probably get me banned over there in spite of the fact this is all totally correct and factual and on topic.

I do not understand nor condone the apparent blatant onesided "lack" of reporting going on. I like Kyle very much. I have admired and responded to his style of the tell it like it is in the hardware industry for several years. I read multiple sites and have never failed to get a straight answer for any question that has arisen. That makes this incident all the more confusing. This is my opinion and mine alone. The actions of the ownership of [H] are bordering on ridiculous. The blatant refusal to address direct specific questions is most telling in my mind. On several threads the "gauntlet was thrown down to just ask, nothing is being avoided." Nothing was addressed. So I finally just asked. I was told I was free to read other sites. The total and complete avoidance of any direct specific relevant questions was avoided. I tried to be open minded and posted new specific questions relating directly to a post by Kyle himself.

My response................................................................silence.

The total, utter, and complete avoidance of any of the pointed direct questions.

My post: with quotes from his post:
Kyle ..Why?
quote:

Originally posted by FrgMstr
I have said it before and I will state it again, this situation is not as simple as Quack was a couple years ago. The game is totally different and quite honestly, all the bitching and complaining in the world is not going to change NVIDIA's actions. It is a DONE DEAL.

Why is it a done deal? What is it about nvidia that makes them unable to be approached with this information?

Why would bringing this information public not have an impact?

When did you know this information?

What do you know Nvidia is doing?

These are some of the questions I would liketo see addressed by [H].
quote:

Originally posted by FrgMstr
Exposing Quack changed things, I knew months ago exposing what NVIDIA is doing would not change the way they are doing anything. At that point, I decided to move forward and not waste our energy spinning our wheels. I can only wish, as much energy would have been put into solving the problem as there has been about griping about it and exposing it.



How could readers spend the same amount of energy solving the problem?

This is the part that blows me away. Here you state you knew what Nvidia was doing months ago. What? You knew about his months ago and said nothing to us? If this is true I am stunned. How could you possibly know this and not post it on your front page and seriously look into re-evaluate at the very least the most recent reviews?

You continued to post flawed benchmarks?

I want to make sure I got this right. That is huge news.
quote:

Originally posted by FrgMstr
Exposing Quack changed things, I knew months ago exposing what NVIDIA is doing would not change the way they are doing anything. At that point, I decided to move forward and not waste our energy spinning our wheels.



How could you know about this for months, Kyle and not tell us? Whether it would change a damn thing with Nvidia is irrelevant. We your readers would have known.

I am very interested in this story. I think what s good for the goose is good for the gander. I agree ATI should have been called on the carpet. I also think Nvidai should be called ont he carpet just as harshly. For whatever reason [H] does not want to do it. Kyle's excuse of it will not do any good so I won't say anything is one of the weirdest excuses I have ever heard for the owner/author of a hardware review site.

Honestly, I am confused. Unless something is going on that we don't know about the blatant actions defy logic and explanation and the silence leads one to some very unpleasant conclusions.

Sorry to bomb your thread but I just don't feel like this is going to get discussed over there.

You're not bombing this thread, I found your post most informative. The bit about "Kyle's excuse of it will not do any good so I won't say anything is one of the weirdest excuses I have ever heard for the owner/author of a hardware review site." sort of sums up my whole problem with taking any of his other stances on benchmarking very seriously.

I truly don't get it, how do they expect this to end?

 

[nelg]

Hi Yonex. Has anyone at [H] asked Kyle what has changed.....

have said it before and I will state it again, this situation is not as simple as Quack was a couple years ago. The game is totally different and quite honestly

? BTW it's better over here . Dave B and Reverend do not shy away from questions.

 

[K.I.L.E.R]

have said it before and I will state it again, this situation is not as simple as Quack was a couple years ago. The game is totally different and quite honestly

It's not as simple because Kyle refuses to see it as simple as the Quack situation.

Let me solve the little mind boggling problem for Kyle.

nVIDIA = cheats; as it lowers IQ on the output.

Ati = optimises; as it doesn't change the output IQ.

Both companies cheated in 3dmark03.

See? Hard or easy to understand?

Ati's optimsiation is acceptable in games while nVIDIA's cheat isn't acceptable in games while niether cheats or specific optimsiations are acceptable in 3dmark03 or any synthetic benchmark.

There is no 2 ways about this.

 

[micron]

[
Ati = optimises; as it doesn't change the output IQ.

Dont jump the gun kiler, somethings about to happen again

 

[jjayb]

Dont jump the gun kiler, somethings about to happen again

Any bets that if ATI gets busted lowering quality Kyle will be all over it? A nice big front page headline?

 

[digitalwanderer]

[
Ati = optimises; as it doesn't change the output IQ.

Dont jump the gun kiler, somethings about to happen again

No hints? What you're not just gonna leave us hanging like that, are you?

Does it have to do with Aquamark3, the article about Unwinder's anti-cheat script, or something else entirely?

 

[2senile]

It's unwinders script. Check the end of "My thread @ Futuremark(Re =Waite for Unwinder)" thread.

 

[K.I.L.E.R]

I'm out to get both companies.

I wish IHVs would be honest with consumers.

 

[nggalai]

Hi WaltC,

I generally agree with your posting, but there's one bit:

Why? Because despite what Kyle believes, the vast majority of 3D games sold do not offer "optimized code paths" for anybody's hardware.

This may change, in the not-too-distant future. It's only guesswork whether the actual majority of games will support different code paths for different IHV in the future (I personally don't think so), but the number of such games will increase drastically. Looking at M$'s latest concepts (and betas) for the DX9 HLSL, I wouldn't be surprised to see many if not most DX9 games in the future offering optimised code paths--at least those games that make use of HLSL.

This, of course, doesn't mean I agree with the [H] blurb. Synthetic benchmarks are important. As far as I am concerned, though, so are as many different game benchmarks as possible, mind. A sensible balance should be strived for, booing either doesn't achieve anything.

93,
-Sascha.rb

 

[Yonex]

Yes they have

Hi Yonex. Has anyone at [H] asked Kyle what has changed.....

have said it before and I will state it again, this situation is not as simple as Quack was a couple years ago. The game is totally different and quite honestly

? BTW it's better over here . Dave B and Reverend do not shy away from questions.

Kyle has been asked several questions.

His responses have been:

1. You can't cheat in a benchmark that doesn't matter.
2. If you can't see, it it isn't a cheat.
3. It wouldn't do any good so why bother.

The part that stills stuns me is his comment that he knew about this months ago and decided not to waste his time because Nvidia wasn't going to change. The jail is full and the new one isn't built yet so we aren't going to enforce any laws until we get our new jail. You cannot ignore the present because we aren't in the future yet.

The consequences from that statement are:
1. He did know Nvidia was cheating and held this incredible information back which is a biblical level violation of reader trust.
2. He didn't know and his most recent comment is not truthful, not accurate and not factual. This would be Clintonese for I didn't have sex.

Those are the only 2 ways that comment can go. 1 or the other is true.

It is very disappointing. I like [H]. I have been there for a long time. I have never seen this kind of behavior. I truly don't understand it. The implications are he is either an Nvidia shill or he is so focused on his view 3dmark is bad that he will embrace damn near anything that will help him acheive his desire of removing them. There are other alternatives I am sure but those are the most obvious and the most likely. The simplest solution to a question is usually correct.

 

[Doomtrooper]

Very interesting article by Lars, and one of the best reads I had at THG since... well, a damn long time. No name-calling, and very deep and thought-provoking. Thumbs up on this one !

hy·poc·ri·sy


A person given to hypocrisy.

-The practice of professing beliefs, feelings, or virtues that one does not hold or possess; falseness.
-An act or instance of such falseness

[Middle English ipocrite, from Old French, from Late Latin hypocrita, from Greek hupocrits, actor, from hupokrnesthai, to play a part, pretend ; see hypocrisy.]

I see some people got a laugh out of the mocking of Fanboys and the Jester in this article, now hypocrisy is riding high here.

The author of this article is Lars Weinand not Tom Pabst but the Nvidia fansite webmaster from http://www.rivastation.com/about_e.htm..
This is the problem with the industry, hypocrisy is the best term to comment on this article, especially mocking supposed fanboys when he is by far Nvidia slanted (or wouldn't be running a Nvidia fansite).
Since he started working at Tomshardware they have tried to change the image of the site and copies his reviews from Tomshardware onto his own site, but at the end of day that site always was a Nvidia Fan Site, so I hope Borsti likes his new hat

Not alot of Diversity here a couple years ago:

http://www.rivastation.com/news-alt.htm

I refuse to read reviews done there anymore, the 5800 Ultra was the icing on the cake..not only were his results laughable and lost all the HQ benchmarks even with running no Trilinear AF, he claimed them as the "new King".
That kind of journalism is beyond excusable.

 

[K.I.L.E.R]

DT, I already posted the meaning of it.

Lets sum this all up.

Kyle = hypocrite
Lars = hypocrite

 

[micron]

Borsti's article was pretty good. He also post's/hangs out here sometimes.
When I was over at the Tom's forums, I asked him if he could answer a few questions over there, because he was answering them here at the time. He apologized to me and said he would, and that the reason he was discussing things here @ B3D was because there was alot of intelligent people here that he could get information from. Maybe we might not want to call him names. To me he seemed rather pleasant, in the messages he sent to me, and I dont think that him owning RivaStation is grounds for dismissing his article as not having any merit.
I do agree though that giving NV30 the speed crown in the past was rather silly of him....

 

[WaltC]

This may change, in the not-too-distant future. It's only guesswork whether the actual majority of games will support different code paths for different IHV in the future (I personally don't think so), but the number of such games will increase drastically. Looking at M$'s latest concepts (and betas) for the DX9 HLSL, I wouldn't be surprised to see many if not most DX9 games in the future offering optimised code paths--at least those games that make use of HLSL.

This, of course, doesn't mean I agree with the [H] blurb. Synthetic benchmarks are important. As far as I am concerned, though, so are as many different game benchmarks as possible, mind. A sensible balance should be strived for, booing either doesn't achieve anything.

93,
-Sascha.rb

You are forgetting something critical to the software developer--he wants to write software that runs well in any API-compliant environment. Software developers are interested in selling their software to the largest possible market--they are not interested in helping IHV's sell their latest hardware. So many times people think the goals of developers and IHV's are identical--when the truth is they couldn't be further apart. Game developers want to sell their software to everybody with a 3D card environment compliant with the specific versions of the APIs the developers target. The way the developer looks at it--and rightfully so--is that it's up to the IHV to create the proper environment to host his software. Again, developers are not interested in taking extra time to support one IHV's product at the expense of another. While that might help a particular IHV it does not help the developer at all--it just locks his software out of a certain percentage of sales. Such a policy is suicide for a developer.

The problem with whole concept of "vendor-specific" code paths is that they work to undermine and defeat the basic API. And make no mistake, the basic API is the developer's one true friend. API's were developed themselves for the *benefit* of software developers--not for the benefit of IHV's. But IHV's who adhere to the basic API development will certainly benefit because it is to the basic API that 99% of all software developers will write--and following the API allows the IHV to greatly expand the market for his products because of the abundance of software created which can run on his product. This is basic 3D-API 101. Following the API is of mutual benefit to the game developer and the IHV--not to mention the consumer since he has a wide selection of hardware from which he can choose, all of which will run 99.9% of the software available.

Likewise, the IHV who seeks to thwart the API paradigm will suffer in many ways, as will the software developer who follows him. Attempting to jerry-rig non-API compliant vendor-specific code paths into game software is a giant step backwards, IMO. For that reason you will see very few, if any, software developers choosing that route. (Aside from the very, very few whom the IHV's intentionally finance to write in code paths which are not representative of 99% of the 3D software available, which the IHV can then use in a marketing campaign to pimp his products.)

I thought Tim Sweeny said it well when he spelled out recently that no one should ever confuse Atari's participation in nVidia's TWMTBP marketing campaign with anything relevant to Epic's software development. IE, one may never assume vendor-specific code paths in TWMTBP-marked software--because in 99% of cases it won't exist. (As it does not exist in UT2K3.)

 

[nggalai]

Hi WaltC,

agreed. But I was speaking of HLSL applications. I'm not too much up-to-date with what's been made public and what hasn't, but I am pretty sure I've seen the different HLSL target profiles introduced with one of the last betas of the next DX9 SDK mentioned on these boards before. Devs don't need to make any special provisions for supporting different targets, HLSL will take care of it automatically, either at run time or during compilation (as long as you specify the target(s), of course).

93,
-Sascha.rb

 

[Borsti]

The author of this article is Lars Weinand not Tom Pabst but the Nvidia fansite webmaster from http://www.rivastation.com/about_e.htm..
This is the problem with the industry, hypocrisy is the best term to comment on this article, especially mocking supposed fanboys when he is by far Nvidia slanted (or wouldn't be running a Nvidia fansite).
Since he started working at Tomshardware they have tried to change the image of the site and copies his reviews from Tomshardware onto his own site, but at the end of day that site always was a Nvidia Fan Site, so I hope Borsti likes his new hat

@Doomtrooper

I see you know nothing about the history of RIVA Station. Is there any word on the site that it is HIV related like nvnews or rage3d?

I started the site with a german language FAQ for the ELSA Victory Erazor (RIVA 128) next to my personal homepage. Not because I was a Fan of ELSA or NVIDIA but because of many things simply did´nt work with that card. I added some news and then wanted so separate it from my personal HP. I did´nt have any money to buy other cards at that time and coded websites in my free time to get money to pay the bandwith for a own domain. I got almost NO support from any HIVs at that time aside of from german PR people at ASUS and ELSA. At that time german PR people did not know anything about hardware websites so it was very tough to run a site in germany at that time.

After a while i was able to make contacts to 3dfx, ATI, NVIDIA, Matrox and others and descited to post news and reviews on general VGA hardware. NVIDIA changed their product name so I saw no reason to change the name of the site. I never saw it as an HIV related site. I made unbiased reviews about all kinds of VGA cards over the years at RS. Do you really think that ATI, 3dfx or Matrox would have given me samples if they thought that RS is a NVIDIA related side!?!?!?!?!

To make it clear: I never got any special support from NVIDIA at all. I also never cared about. My first contact with NVIDIA was 2 years after I started the site when I posted benchmark numbers of leaked Detonator drivers and offered the drivers for download. The first card sample from NVIDIA I received for a review was a GeForce 256 DDR!

I´m still running the site as a hobby. To support the community which grew in the Forums over the years and to stay in close contact with the readers. I don´t see any reason to change the name only because it has a name of a way outdated product from NVIDIA in it. Do you call B3D´s Reverend biased against ATI and NVIDIA because he ran a site with 3dfx related content years ago???

Your imputation that I tried to change the image of the site when I started to work for Tom is wrong as well as the other imputations in your post. You may not like me and/or THG. That´s your problem. But please stop posting any imputations about things you don´t seem to know.

Lars (Tom´s Hardware Guide)

 

[WaltC]

Hi WaltC,

agreed. But I was speaking of HLSL applications. I'm not too much up-to-date with what's been made public and what hasn't, but I am pretty sure I've seen the different HLSL target profiles introduced with one of the last betas of the next DX9 SDK mentioned on these boards before. Devs don't need to make any special provisions for supporting different targets, HLSL will take care of it automatically, either at run time or during compilation (as long as you specify the target(s), of course).

93,
-Sascha.rb

OK, sorry nggalai, I didn't quite catch your drift from the first post--sorry for the elementaries lesson... Yes, nothing wrong with API-compliant HLSL, of course. I wouldn't call that "vendor-specific", though, at least from the developer's position, as you point out--unless you want to do something in there that departs from the API and goes toward custom hardware support applicable to a particular IHV.

 

[Pete]

Lars, HIV is not the same as IHV (which is what you meant) in English.

 

[micron]