TechReport chimes in on the HL2 benchmark

[Joe DeFuria]

Re: Toaster

I recently bought a toaster; the man at the corner of the street from whom I purchased the toaster promised me that the toaster would not only allow me to play Half-Life 2 at 2048 x 1536 at a constant 120 FPS, but it would also double the length of my penis. I trust the man at the corner of the street because I have no proof that he is lying. Why would he lie to me?

So...how'd the penis thing work out? 8)

 

[kyleb]

lol Joe, well if that doesn't work out i have a few emails i could forward you.

 

[WaltC]

Re: Toaster

...This is clearly an anti-toaster conspiracy.

Heh...Could you run that by me again...?

 

[Dave H]

The mixed mode does just refer to some of it being forced to PF16, but also because some sahders are dropped down to PS1.4 rather than PS2.0. Also some operations like vector normalisation will be done via cubemaps on the NV30 path (burning texture read/writes and bandwidth) whereas the HLSL path will use maths within the PS (burning ALU cycles).

Just to clarify:

Some site (can't recall which one ATM) claimed that there are actually two NV3x paths in HL2. The first--which is what Valve labeled "mixed" in their presentation--differs from the full DX9 path by more liberal use of the _pp hint (does the full DX9 path use _pp at all??), replacing computation with lookups to precomputed textures, and by rewriting shaders in more NV3x-friendly ways (is the output of the new versions identical, either by mathematical proof or subjective IQ comparison?). This version would seem to be directed primarily if not exclusively at NV35, as NV34 got a much smaller performance boost, and NV31 actually lost performance running the "mixed" path. (And, FWIW, we can guess that NV30 would also do poorly as a result of trading computation for bandwidth, as this path does.)

The second--dubbed "dx82"--may make use of some or all of the techniques of the "mixed" path, but what differentiates it is that some (not all) of the PS 2.0 shaders in the full and "mixed" DX9 paths are replaced with PS 1.1 or PS 1.4 equivalents. The implication is that the "mixed" path does not replace any PS 2.0 shaders with DX8 versions, although I'm not sure that was explicitly stated by Valve. This path, then, is intended to be run on NV31 and NV34.

Can you confirm or clarify if this is accurate? On the one hand, I can only recall seeing mention of this new "dx82" path at one site; on the other, it does seem odd that Valve wouldn't have a second NV3x path considering how the "mixed" path leads to lower performance on NV31 and still-unplayable performance on NV34. Of course it could be that they decided nothing short of dropping NV31 and NV34 to the DX8/DX8.1 path would allow playable framerates, and thus didn't bother...

 

[Vince]

Charge more to nVidia card owners specifically? First: are you serious about that being feasible, or are you just avoiding recognizing a problem with your analogy?

For myself, I still don't think Valve has the option of charging more to nVidia customers specifically, and your proposition otherwise continues to seem nonsensical to me. This still leaves your analogy "all kinds of broken" AFAICS.

Valve can do what they want via Steam, which was the point of the post. If they wish to target N consumers that have purchased i hardware - then they will have to put in the manpower/monetary/temporal resources that will allow it run acceptable. This is an obvious prerequisite to make a given sale in any marketplace. Thus, as per the origional post - Gabe knew the repercussions of supporting nVidia's hardware and still went down that path. His comments as bdmosky stated, only created a more polar enviroment.

They did. It just isn't compatible with full feature exposure, because of the hardware's capabilities. This point is still lost on you, I see?

I already stated:

It was lost due to the fact that Gabe went out of his way to criticize an IHV for making him do his job wrt making a product that is usable for consumers. He is the conformer, not the consumer. He is targeting the consumer, and he has the choice of who or what to target. If you do not like these choices, then don't and take the repercussions. But, if you do take the path of conforming, don't criticize

So, they didn't make it playable within the limits of the hardware? Or does the hardware not have any limits?

You're missing what I'm saying. Of course there are upper bounds on absolute preformance - yet, you can offset this to a large extent by investing more time to run a given application more effeciently. Which is something Gabe stated they did, but used it in a negative context.

Which comes back to my main point, which Joe agreed to, that the consumers bought that partticular hardware. Thus, if you're targetting and chasing a given group of consumers with i hardware, then be prepared to put front-end work and money into creating an application that runs acceptably. Don't go out of your way to critisize a company for making you do your job as a developer.

"making him do his job"? OK, so it really does boil down to simply considering hardware limitations as an irrelevant factor, and ignoring that they specifically did make the product usable for consumers, because they couldn't make the hardware usable while doing full features. What about limitations in hardware capabilities not lining up with what consumers think, and dealing with that eventuality?

Missed the point, already covered.

Heaven forbid consumers buy hardware because it is actually better. No, they shouldn't conform to performance capability recognitions, simply their brand loyalty and ignorance of such information. Shame on a developer for challenging that, and seeking to inform contrary to consumer ignorance.
In short: hardware can't actually be better, people should be able to pick the brand name they want?

This is absurd. As a supplier, your goal is to sell a given product which has a set-piece value to a given group. To stick with the earlier automobile example, if I was producing a given brakepad designed with a Dodge Viper in mind: Should I tell the other 99.99% of the marketplace who own POS cars: "Heaven forbid consumers buy hardware because it is actually better"?

Common' here. The seller conforms to the prevailing consumer and marketplace - the marketplace doesn't adapt to what I see. The rest of your post about informing the consumer is irrelevent to a discussion on producing a product that has a wide market appeal from a preexisting userbase.

 

[Vince]

No, not "fallicious" at all. Preexisiting opinion? Let me see why. Caught red-handed in public cheating in 3D Mark 2003 with several driver revs. Caught red-handed in public cheating in UT2003. Why might I have an opinion that they cheat? Oh yeah, and continued cheats in 3D Mark 2003 (texture compression is easy to see, you know as are low quality shaders). None of this is new

So, if a man was convicted of previous felonies but then *appeared* and *commented* on his intent to change - any charge made against him - even those with no empirical evidence that proves guilt - after that point should be considered true?

So you're telling me they didn't cover-up the cheating? Prove it. My position is much easier because the cheating never stopped.

I don't care. Show me proof and then I'll join you. Without unbiased, reproducable, and empirical proof - you've comments have NO validity and can be made by an invidual against any Company, Institution, and Product. How hard is this to comprehend?

I choose to apply Occam's Razor: The simple answer is the correct answer, and this corresponds with reality in this case.

First of all, Ockham's statement is often misused by people (eg. as you did here) and strengthened beyond it's origional intention. Someone once commented that, "This is an interesting example of the fact that even scholars of audacius spirit and fine instinct can be obstructed in the interpretation of fact by philosophical prejudices." Which would seem to hold true here.

You obviously don't know what Ockham stated or intended which translates roughly to: Plurality shouldn't be posited sine a necessity.

So, lets think about this. What's more likely in this singular case: That nVidia has a massive and widespread platform in place to support cheating that's not observable and verifiable due to a plurality of intricate detection routines or that they optimized it?

But, I say forget Billy. If you can prove (eg. something reproducable) that they're cheating then all of us who are trying to be reasonable will join you. But, to just jump into the pool on blind-faith and your internal bias is wrong.

P.S. I also like how you resort to personal attacks to support your position. My degrees are in mathematics, not law.
P.P.S. Since you brought it up: Many cases are won on circumstantial evidence alone. You don't always need habeus corpus, sometimes means, motive and opportunity are enough.

(a) Great, congradulations! It's a noble achievement, but it hardly chnages the fact that I want proof before jumpin on a boat filled with people who've made a decision based on circumstantial evidence.
(b) They also have massive instances of guilty men in jail and on death row. I like reproducable proof.

 

[Vince]

True. His customers bought nVidia hardware for a variety of reasons. His points (as they pertain to "nVidia owners"), since you haven't figured it out:

Evidently I have yet to, but thanks for explaining it.

1) First and foremost, unless you are of the very small portion of the "100 million" nVidia owners who own a DX9 "featured" card, the DX9 paths have no relevance to you. You jsut use the DX8 path like everyone else, and we have no issues with nVidia hardware on the DX8 path.

True. Dave expanded on this quite well in fact and I agree. Although, the case can be made that with nVidia's absolute DX9 marketshare higher than ATI's (yes, I know that ATI holds like >95% of the $300 PP) that if you want to target this group, then putting the costs in to provide the consumer with a workable product is a a given. Otherwise, just don't target them - and the rest of the argument follows.

2)...if his customers bought nVidia hardware to play Half-Life, that's not a particularly good choice.

Most likely correct. But then just don't target nVidia's [DX9] hardware at all.

3)...If his customers are buying nVidia cards because of supposed DX9 performance and features touted by nVidia, and bluffed via cheating benchmarks....they bought it for the wrong reason.

Again, this is most likely correct and I'm not arguing this. But it still doesn't change the Valve <-> nVidia userbase dynamic.

4) If his nVidia FX customers are happy with the DX8 gaming performance and quality on every other game its offered to date, Half-Life2 will be just fine.

Utterly irrelevent, but a damn slick jab at nVidia if I may say so.

 

[nelg]

This is absurd. As a supplier, your goal is to sell a given product which has a set-piece value to a given group

The problem, Vince, is that when the seller is nVidia this "set-piece value" has been misrepresented due to nVidia's cheating.

 

[Vince]

The problem, Vince, is that when the seller is nVidia this "set-piece value" has been misrepresented due to nVidia's cheating.

I understand completely and the whole paradigm sucks quite frankly. But, then don't support that product, don't put the energy into it as it will bring nothing but bad karma.

Yet, if you do support the entity [in whatever manifestation it is - even a mouse and wheel] then be prepared to do what it takes to get your product into a state that is satasfactory to the consumer. And nobody wants a 1:1 comperason, just reasonable. And, frankly, I don't want to hear a developer whose going to make enormous profit off the [FX] line bitch about how he has to do his job.

 

[Dio]

I don't want to hear a developer whose going to make enormous profit off the [FX] line bitch about how he has to do his job.

Was it fair for nvidia to bitch about 3dmark?

 

[Dave Baumann]

They appear to be trying to make it clear that there is a gulf of difference between perception and reality. From the perception NVIDIA have given developers and users the performance of the NV3x series is right up there - by utilising HLSL and DX9 and getting more than acceptible performance from one DX9 architecture I think they are trying to make it clear that they have to go above and beyond "their job" to make the performance comparable (and outside the bounds of HLSL) on the FX series as other applications haven't made this terribly apparent (although they have to many here).

And yes, Valve pretty much realised what you are suggesting a little while into the development - having spent the time optimising for "DX9" on NVIDIA's boards they realised that they should have just treated them as DX8 boards. Valve actually stated that this is what they should have done (and indeed have done for 5200/5600).

 

[Joe DeFuria]

Which comes back to my main point, which Joe agreed to, that the consumers bought that partticular hardware.

That's about all I agreed with you on, btw,. which I didn't think was in dispute by anyone.

Thus, if you're targetting and chasing a given group of consumers with i hardware, then be prepared to put front-end work and money into creating an application that runs acceptably.

Um, that's what Valve did. And they weren't complaining about it either...until the results came back: horrid performance.

Don't go out of your way to critisize a company for making you do your job as a developer.

Please do go out of your way to criticize a company that's trying to put forth unrealistic expectations of it's hardware. Valve's job as a developer is not to turn a turd into gold. That's impossible.

It's to get it running as good as can be expected on that turd. And again, the problem is: nVidia is setting those expectations to high, via several dubious methods like driver cheats.

This is absurd. As a supplier, your goal is to sell a given product which has a set-piece value to a given group. To stick with the earlier automobile example, if I was producing a given brakepad designed with a Dodge Viper in mind: Should I tell the other 99.99% of the marketplace who own POS cars: "Heaven forbid consumers buy hardware because it is actually better"?

Talk about absurd. Why stick with the automobile analogy after it's been exposed to be worthless? Valve did not just make a brakepad with a Dodge Viper in mind. They made a series of brake pads for the following: Viper (DX9), Porshe (DX9 mixed), Nissan Maxima (DX 8.1), Toyota Corolla (DX 8.0), and Yugo (DX6).

All Valve is doing is saying: nVidia wants you to believe that your 5600 is a Viper or a Porsche. It's not. It's more like a Maxima. If you try the Viper pads on your 5600, don't expect your card to brake like a Viper.

Common' here. The seller conforms to the prevailing consumer and marketplace - the marketplace doesn't adapt to what I see. The rest of your post about informing the consumer is irrelevent to a discussion on producing a product that has a wide market appeal from a preexisting userbase.

Let's try a Vince tactic here.

This is a PC graphics forum, not a console forum. Try going back to the console forum, because obviously you have no idea what that PC business is like.



(Yes, that sentiment is ridiculous, I hope you get the point.)

Vince, valve's product DOES have a widest market appeal possible. It scales from DX6 class hardware, all they way up to DX9 class. And it makes GOOD USE of DX9 hardware. This is arguably the first title to do so. In other words, HL2 has extremely wide market appeal: base quality for older hardware users, and it takes advantage of the latest hardware for the hardcore gamers.

What "preexisting userbase" Valve is ignoring or dissing?

 

[Joe DeFuria]

They appear to be trying to make it clear that there is a gulf of difference between perception and reality.

Why is the clear to everyone but Vince?

 

[Joe DeFuria]

Again, this is most likely correct and I'm not arguing this. But it still doesn't change the Valve <-> nVidia userbase dynamic.

What exactly is this Valve <-> nVidia userbase dynamic.?

Dave expanded on this quite well in fact and I agree. Although, the case can be made that with nVidia's absolute DX9 marketshare higher than ATI's (yes, I know that ATI holds like >95% of the $300 PP) that if you want to target this group, then putting the costs in to provide the consumer with a workable product is a a given. Otherwise, just don't target them - and the rest of the argument follows.

I explained this earlier: the problem is, Valve has no idea from the onset of DX9 development, if "targeting 5200 and 5600 for DX9" is viable or not. Valve has to rely on the IHV's assesment and guidance. I'm sure nVidia gave them favorable guidance, so yes, Valve went ahead and expended the effort (as is their job) to accomodate that.

Problem is, nVidia's guidance was wrong.

Now, these things can happen...so that in and of itself is probably not a good reason to slam nVidia. What likely did it, was nVidia's det 50's. Gabe saw them, and realized that nVidia wasn't going to take Valve's assesment of "not really a DX9 card" laying down, and nVidia would do what it took to try and deceive people of the truth.

And that, as Gabe said, would ultimately "make his customers pissed". And unfortuntely, they'll probably be pissed at valve, for something that is not Valve's fault.

 

[Joe DeFuria]

Yet, if you do support the entity [in whatever manifestation it is - even a mouse and wheel] then be prepared to do what it takes to get your product into a state that is satasfactory to the consumer. And nobody wants a 1:1 comperason, just reasonable. And, frankly, I don't want to hear a developer whose going to make enormous profit off the [FX] line bitch about how he has to do his job.

*Cough*

Then take it up with Deano.

http://www.beyond3d.com/forum/viewtopic.php?t=7873

Obviously ATI weren't that upset to see there hardware coming ahead so well, but what wasn't prehaps so expected was how glad everybody else was that HL2 results matched the results most of us had already seen ourselves. Somebody on the forums (sorry can't remember who) asked why developers seemed quite shy in stating our results, I obviously can't talk for everbody but the answer is probably a simple case of somebody had to first and whoever that person/company was, they better be able to handle the heat that it would produce.

 

[Vince]

This is a PC graphics forum, not a console forum. Try going back to the console forum, because obviously you have no idea what that PC business is like.

Actually, I will leave. Because I don't need to waste time posting endless responces against someone who just wants to argue and invokes comments about nVidia's failure at every junction - something which should be irrelevent in the context of Valve providing a quality product at all costs to the consumer, and if thats not possible or worth-while to just not support it.

If you look at what I've stated and what Dave replied to - I think it's clear that there it nothing I've said in principle that's wrong. Nor is there anything I've stated that Valve hasn't recogized and adapted to.

You need to get off the high-horse and stop this endless bashing of nVidia's products based on these futile arguing-points and support of everything that has a direct or indirect influence on ATI. How you can so polarize a conversation about Valve's support of a population (regardless of the IHV that's used by that population) to between nVidia' lies and ATI's ulitimate justice just amazes me.

And with that I shall leave for good. Post a reply in which you argue against non-issues again, go critisize me, I won't see it nor do I really care what you have to say.

 

[Joe DeFuria]

Actually, I will leave. Because I don't need to waste time posting endless responces against someone who just wants to argue and invokes comments about nVidia's failure at every junction - something which should be irrelevent in the context of Valve providing a quality product at all costs to the consumer, and if thats not possible or worth-while to just not support it.

nVidia's failure...and nVidia's inability to just accept it, is exactly why Valve is being active in protecting the integrity and quality of their product.

I think it's clear that there it nothing I've said in principle that's wrong.

Of course there is. One example, you said: "I understand completely and the whole paradigm sucks quite frankly. But, then don't support that product, don't put the energy into it as it will bring nothing but bad karma. "

And here's what's wrong with that: Say Valve follows your advice, and based on preliminary tests just decides to not code and special support for NV3xat all. They release the game, it runs that absolute crap on any nVidia card (NV35 included) in DX9 mode compared to ATI.

Valve gets HAMMERED for "playing favorites." nVidia goes on the rampage: "They got paid millions by ATI to purposely make us look bad." You can imagine the conspiracy theories.

You and other nVidia faithful come in and argue "Valve should have coded a special path...like Carmack..he's doing his job, why don't you do yours?" Valve responds and says "it wouldn't have done any good". You say "you didn't even try! (Insert nvidia cheat benchmark here) shows good DX9 performance...you must SUCK as a developer!"

This is the FIRST significant title to use DX9 to an appreicable extent. Valve had little choice but to give it a more than good faith effort.

You need to get off the high-horse and stop this endless bashing of nVidia's products based on these futile arguing-points and support of everything that has a direct or indirect influence on ATI.

I'm not doing the nVidia product bashing. Valve is. The arguments here are about Valve's justifications for it. I'm bashing you...there's a difference, though I can understand why you might see it the same.

And with that I shall leave for good. Post a reply in which you argue against non-issues again, go critisize me, I won't see it nor do I really care what you have to say.

Damn, and I was so looking forward to you confronting Deano about his "bitching about having to do his job."

 

[WaltC]

So, if a man was convicted of previous felonies but then *appeared* and *commented* on his intent to change - any charge made against him - even those with no empirical evidence that proves guilt - after that point should be considered true?

Logical contradiction. If there is no empirical evidence of guilt, why would the man "appear" and "comment on an intent to change"? In order to change, one must first acknowledge one's present course is in error. Surely you are not characterizing nVidia here, as I've seen plenty of empirical evidence, but no "comment" on the part of nVidia which specifies an "intent to change." You therefore must mean some other entity.

I don't care. Show me proof and then I'll join you. Without unbiased, reproducable, and empirical proof - you've comments have NO validity and can be made by an invidual against any Company, Institution, and Product. How hard is this to comprehend?

Illogical contradiction. Proof has abounded for months from a multiplicity of sources. You have not acknowledged that proof--therefore, whatever proof he could show you would also be ignored. You ask him to show you proof, and then you declare that he cannot do so. You have decided--before seeing it--that such proof does not exist.

First of all, Ockham's statement is often misused by people (eg. as you did here) and strengthened beyond it's origional intention. Someone once commented that, "This is an interesting example of the fact that even scholars of audacius spirit and fine instinct can be obstructed in the interpretation of fact by philosophical prejudices." Which would seem to hold true here.

Assuming, however, that Occam's Razor means what he said it means, or whether it does or not, you have not explained why the simplest answer is not, in fact, usually the correct answer.

So, lets think about this. What's more likely in this singular case: That nVidia has a massive and widespread platform in place to support cheating that's not observable and verifiable due to a plurality of intricate detection routines or that they optimized it?

The problem with this statement is the falsity that such cheating is not observable--which illustrates the illogic of the statement. Valve recognized it as such. If the cheating was not "observable and verifiable" then no one would know the cheating is taking place and the accusations would never have been made. Since they have been made and observed and duplicated, they are therefore quite observable and verifiable. Hence, the accusations exist.

There is no need to confuse optimizations with cheats.

But, I say forget Billy. If you can prove (eg. something reproducable) that they're cheating then all of us who are trying to be reasonable will join you. But, to just jump into the pool on blind-faith and your internal bias is wrong.

Your data set is incomplete, and again you are illogical. You say that such cheating has never been observed and is not reproducable--so whence come the charges? It is not logical to assume that accusations of cheating occur without observable, empirical, verifiable evidence to sustain them--else no one would accept them. Futuremark's original audit report, for instance, provided empirical, verifiable, repeatable evidence that was indeed repeated on several websites by differing individuals. Hence, what is lacking empirical evidence is the suggestion that no such cheating has taken place. That is the fantasy which lacks empirical evidence. If the testimony of Valve, a developer which has worked months with nVidia to develop an nV3x code path, is not empirical, then what is?

(a) Great, congradulations! It's a noble achievement, but it hardly chnages the fact that I want proof before jumpin on a boat filled with people who've made a decision based on circumstantial evidence.
(b) They also have massive instances of guilty men in jail and on death row. I like reproducable proof.

Again, the evidence is plentiful, empirical, and has been repeated often. You cannot expect him to do your research for you, and you cannot expect him to be able to correct your inability to perceive the evidence when it has been presented to you. IE, it's not his fault you don't get it.

 

[StealthHawk]

The mixed mode does just refer to some of it being forced to PF16, but also because some sahders are dropped down to PS1.4 rather than PS2.0. Also some operations like vector normalisation will be done via cubemaps on the NV30 path (burning texture read/writes and bandwidth) whereas the HLSL path will use maths within the PS (burning ALU cycles).

Just to clarify:

Some site (can't recall which one ATM) claimed that there are actually two NV3x paths in HL2. The first--which is what Valve labeled "mixed" in their presentation--differs from the full DX9 path by more liberal use of the _pp hint (does the full DX9 path use _pp at all??), replacing computation with lookups to precomputed textures, and by rewriting shaders in more NV3x-friendly ways (is the output of the new versions identical, either by mathematical proof or subjective IQ comparison?). This version would seem to be directed primarily if not exclusively at NV35, as NV34 got a much smaller performance boost, and NV31 actually lost performance running the "mixed" path. (And, FWIW, we can guess that NV30 would also do poorly as a result of trading computation for bandwidth, as this path does.)

The second--dubbed "dx82"--may make use of some or all of the techniques of the "mixed" path, but what differentiates it is that some (not all) of the PS 2.0 shaders in the full and "mixed" DX9 paths are replaced with PS 1.1 or PS 1.4 equivalents. The implication is that the "mixed" path does not replace any PS 2.0 shaders with DX8 versions, although I'm not sure that was explicitly stated by Valve. This path, then, is intended to be run on NV31 and NV34.

Can you confirm or clarify if this is accurate? On the one hand, I can only recall seeing mention of this new "dx82" path at one site; on the other, it does seem odd that Valve wouldn't have a second NV3x path considering how the "mixed" path leads to lower performance on NV31 and still-unplayable performance on NV34. Of course it could be that they decided nothing short of dropping NV31 and NV34 to the DX8/DX8.1 path would allow playable framerates, and thus didn't bother...

That would be Anandtech who said that.

 

[cthellis42]

Interesting post by Deano. This in particular caught my eye, though:

For Valve to do this, shows they were really annoyed, also the fact Microsoft issued a press release stating HL2 was the DirectX 9 benchmark also show how annoyed they were.

What is he referring to specifically here?