Reverend at the Pulpit #4

Reverend

Banned
Reverend at the Pulpit #4

Site Shtuff
First of all, naturally, is I'd just like to express how delighted and pleased about our just-launched Shader competition, sponsored by ATI. The prizes are actually better than what was originally planned (a couple of Radeon 9800 cards) -- everyone say thanks to both Dave and ATI for reaching a better agreement! For about a year now I had suggested to Dave about holding such a contest (it's a result of my experience at Voodooextreme, where they held a UT2003 contest sponsored by ATI) and it is very satisfying to see this come to fruition and as a result expands further on the scope of Beyond3D beyond (no pun intended) "just a review and articles site". And it is especially gratifying to read an email from ATI saying "Beyond3d has a lot of respect within ATI.". I wonder if NVIDIA thinks the same... ;) As it is, at time of posting this, we have had 5 registrations for the competition, from a student to guys who program for a living... not an overly enthusiastic number but not too shabby (IMO) given that this was launched less than 24 hours ago. Hopefully, word will get around and things will pick up speed.

Again, "cheating" and "optimization" seems to be the buzzwords, as a result of Dave being the first to discover the way NVIDIA drivers are detecting UT2003 and performing texture filtering that does not give the user what he specifies nor what NVIDIA claimed to offer. I truly believe that this is an "optimization" but the least NVIDIA could do was inform the public that this is what's happening and they really shouldn't have lied to reviewers when they said that their "Quality" mode means proper trilinear filtering, the latter of which means inaccurate reviews. Granted "apples-to-apples" has almost become nearly impossible when you compare different video cards but reviewers sure wouldn't reject being told the truth about what's actually happening. I still believe that one of the things that should change when it comes to reviews is to not do any comparisons between video cards from competing IHVs because this will (already has!) lend to all the furore this has created. Review VidCard A and say "This is what's happening when it comes to AA and/or texture filtering" and say the same thing when reviewing Vid Card B. When a reviewer tries to "equalize" things, the entire "review" (which is really a shootout) becomes presumptious. I can honestly say that I do appreciate what NVIDIA's drivers are doing in UT2003 but I don't appreciate the fact that we'd been lied to -- it just sours what would have been a creative "optimization" IMO. All of this however comes down to the way reviews are being done by various sites. Yes, NVIDIA drivers filters textures differently compared to ATI drivers in UT2003 -- but then both IHVs execute AA differently anyways. The problem really originates from the fact that websites just wants to do shootouts while trying to "equalize" things. I truly believe that every specific technology implementation by competing IHVs that gives us different results have their place. What a reviewer can't, and shouldn't, do is to pass judgement on the differences because a reviewer can't know if what he perceives as "bad" may or may not be actually seen as a "benefit" in the eyes of others. Put it this way -- if all reviews thus far has never appeared as shootouts, would the gravity of this texture filtering in UT2003 situation be changed in its importance? I believe so. Again, "equalizing" has its good and its bad sides. Most times, however, it means overlooking things that are seen as "benefits" by others. Anyway, in summary, I personally like what NVIDIA has done wrt this issue but I do not like the fact that they lied to reviewers and hence misled the public through reviewers' reviews. I'm sure I haven't expressed my opinion about this whole thing as well as I have it in mind but then all this is just a little convoluted to begin with, with a number of related topics revolving around the whole "cheating/optimization" thing, especially ones concerning how best to write reviews... so don't shoot me if my opinion about this matter looks incomplete... because it very probably isn't complete.

[edit]On the same issue, a few days ago I'd asked Tim Sweeney if Epic approves/agrees to what NVIDIA's drivers are doing in UT2003. He replied. Initially, I did not want to reveal Tim's reply (although I informed Dave and B3D's exclusive collection of individuals from a few websites, and told them not to reveal it either) but after thinking this through a bit, I see no harm in quoting Tim and that it may actually be quite illuminating in terms of how things progress between IHVs and developers in specific matters and what matters are priority (or not) to developers... after all, I specifically asked that whatever his reply may be that he should consider it to be available for public viewing. I'm not going to post what I wrote Tim (it's a mite sensitive and should remain private) but here's Tim's reply to me :

Tim Sweeney said:
Anthony,

We're not in the business of publically critiquing the nuances of drivers or approving/disapproving them. We only get involved in the driver situation publically when there is a major driver bug that has a significant negative impact on people who play our games, and that hasn't happened in years. Things like mipmap bias factors and trilinear optimizations/shortcuts don't fall in this category; they are subjective and it's best for users and web critics to debate what performance/quality tradeoffs are acceptable.

We often have informal conversations with NVidia and ATI about performance/quality tradeoffs and shortcuts, but these are private discussions and not public endorsements or rejection of the tradeoffs they make.

Long term I would like to see the rendering API defined in terms of exactly reproducable results, so that there isn't room for debate on things like texture filtering optimizations or mipmap biasing. Then, a given set of triangles and render state passed down to 3D hardware is guaranteed to give the exact same results on all hardware and any software emulation or fallbacks. There are general and non-controversial definitions of how all such things should work, from IEEE 752 for floating point to Fundamentals of Computer Graphics for the definition of trilinear filtering and mipmap factor calculation.

-Tim

Read into that what you will. Looks like NVIDIA discussed this texture filtering thing with Tim/Epic and Tim knows about it but he didn't say if he/Epic actually approves of it... or whether such an approval is asked for or needed by NVIDIA.[/edit]

For the past one week I have been corresponding with the developer of a hugely popular game franchise in terms of understanding the technologies behind the latest game in this series. The game features a hidden benchmarking feature but it is lacking in terms of benchmarking options that helps automate benchmarking the game, something reviewers truly treasure. So I'm happy to say that the developer listened to my suggestions and has implemented almost all of them (there are two outstanding ones, which I suggested to him only yesterday, one of which is the ability to color MIP levels, much like what we see in UT2003) and I can say that they all work perfectly using a private beta build I was given. My suggestion was the ability to specify various command line options (such as rez, AA, types of effects) that were all lacking in the first place as well as the benchmark outputting a CSV file showing all the results. The only outstanding problem is a bug that exists on Radeon cards at a certain setting, which the developer informed me is hopefully just a driver bug and that the ATI folks visiting him yesterday should help fix it. All of this should result in a patch soonish and once that patch is official I should be able to record a demo that will be used in perhaps all of B3D's future reviews. A demo which the public can't have of course :) Anyway, I spent a considerable amount of time on this... hopefully it will prove to be very useful insofar as my efforts to continually upgrade B3D's range of software used for reviews and articles.

Marco's revolutionary ( ;) ) new engine appears to be progressing along nicely -- hopefully a test run is not too far away. One thing that I have been thinking of for a long time, and of which I have yet to voice to Dave and the gang, is to have a Chinese language version of the site. The People's Republic of China is huge, yes? I have no idea, and I haven't investigated any possible existing surveys on such, what is the size of China's Internet usage but there is no doubt that it can only go up. It's a huge Internet market to tap. Of course, to do this, we either need an engine that automatically translates our main English site to Chinese, or we need a guy that knows Chinese well, which I'm not unfortunately (and embarrasingly since I'm Chinese). Anyway, something for Dave and Marco to think about since I've brought it up here first. You all want B3D to take over the world, right? :)

As for my promised pixel shader-centric demo (as a result of my PS research), well, that's kinda stalled due to the competition and my messing around with the game mentioned two paragraphs above. I'll try to put in more time for it as I believe it will be useful for B3D (and the public).

Also, I'm mulling over the idea of whether I should strangle Scott Sellers until he agrees to an interview with me. I'm sure many of you are wondering just what the hell he's been up to.

Life
My 4-year-old boy's team won the silver in his kindergarten's first ever Sports Day. He thinks he won gold however and I sure ain't gonna tell him the truth! He's growing up fast... time flies... I'm growing older... Me and the missus have just about ended any further contemplation about having another kid... looks like Sean will be The One.

Watched Finding Nemo and T3. Former was great, latter was ho-hum. Anyone knows what James Cameron is currently working on? I have no doubt he would've made a much better T3.

Lastly, to quote Gary Tarolli : The Internet is the ultimate free speech medium. If you own a popular website that has a forum and you start banning folks in your forums and locking up threads on a very consistent basis because valid criticisms (not flames) are provided and you don't like reading such, your forum becomes useless because forums are not meant to be a soapbox for saying "I'm always right and you cannot say anything otherwise" but it is a medium for the owner of that forum to learn and to exchange ideas. Refusal to let threads stand and folks to express their opinions that don't agree with yours in a non-flaming manner is just either not being open-minded enough or proof of an inflated ego... or afraid of being proven wrong and accepting it humbly. The last takes a bigger man than one who continuously fights to stand his ground even in the face of flurries of criticisms.

All of the above in my very humble and personal opinion, and please excuse any signs of incoherence!
 
Congrats to you son!

Congrats on your son's sport day at school...but what kind of kindergarden has class in the summer? My 3 year old daughter is successfully making the jump from diapers to underwear and started pre-school over at Purdue last week so I am sort of jumping up-n-down for joy over the long-dark 5 years of changing the diapers FINALLY being over! :D

And my 6 year old son is going to play on some "bumper boats" in his local summer camp program today and is quite looking forward to it. :)

(Sorry, just in a Daddy/family mood in the morning getting the kids ready for the day...I'll be in viddy mode when I get done. ;) )
 
my kids broke up for summer yesterday. Wife is already stressed after one day. 6 more weeks to go!
 
Rev Wrote:
The People's Republic of China is huge, yes? I have no idea, and I haven't investigated any possible existing surveys on such, what is the size of China's Internet usage but there is no doubt that it can only go up. It's a huge Internet market to tap. Of course, to do this, we either need an engine that automatically translates our main English site to Chinese, or we need a guy that knows Chinese well, which I'm not unfortunately (and embarrasingly since I'm Chinese). Anyway, something for Dave and Marco to think about since I've brought it up here first.

Actually B3D does have many readers in China,but for most chinese,they dont have enough English skills to read B3D;second,B3D targets high end 3d fans,so this hurt your influnce in China.Anyway,People respect B3D as a Professional 3d website,but seldem visit it. :(


hehe,Revit's shame u dont know Chinese,while I know Chinese very well since I m a native Chinese.;)
 
Reverend said:
Reverend at the Pulpit #4


Again, "cheating" and "optimization" seems to be the buzzwords, as a result of Dave being the first to discover the way NVIDIA drivers are detecting UT2003 and performing texture filtering that does not give the user what he specifies nor what NVIDIA claimed to offer. I truly believe that this is an "optimization" but the least NVIDIA could do was inform the public that this is what's happening and they really shouldn't have lied to reviewers when they said that their "Quality" mode means proper trilinear filtering, the latter of which means inaccurate reviews. Granted "apples-to-apples" has almost become nearly impossible when you compare different video cards but reviewers sure wouldn't reject being told the truth about what's actually happening. I still believe that one of the things that should change when it comes to reviews is to not do any comparisons between video cards from competing IHVs because this will (already has!) lend to all the furore this has created. Review VidCard A and say "This is what's happening when it comes to AA and/or texture filtering" and say the same thing when reviewing Vid Card B. When a reviewer tries to "equalize" things, the entire "review" (which is really a shootout) becomes presumptious. I can honestly say that I do appreciate what NVIDIA's drivers are doing in UT2003 but I don't appreciate the fact that we'd been lied to -- it just sours what would have been a creative "optimization" IMO. All of this however comes down to the way reviews are being done by various sites. Yes, NVIDIA drivers filters textures differently compared to ATI drivers in UT2003 -- but then both IHVs execute AA differently anyways. The problem really originates from the fact that websites just wants to do shootouts while trying to "equalize" things. I truly believe that every specific technology implementation by competing IHVs that gives us different results have their place. What a reviewer can't, and shouldn't, do is to pass judgement on the differences because a reviewer can't know if what he perceives as "bad" may or may not be actually seen as a "benefit" in the eyes of others. Put it this way -- if all reviews thus far has never appeared as shootouts, would the gravity of this texture filtering in UT2003 situation be changed in its importance? I believe so. Again, "equalizing" has its good and its bad sides. Most times, however, it means overlooking things that are seen as "benefits" by others. Anyway, in summary, I personally like what NVIDIA has done wrt this issue but I do not like the fact that they lied to reviewers and hence misled the public through reviewers' reviews. I'm sure I haven't expressed my opinion about this whole thing as well as I have it in mind but then all this is just a little convoluted to begin with, with a number of related topics revolving around the whole "cheating/optimization" thing, especially ones concerning how best to write reviews... so don't shoot me if my opinion about this matter looks incomplete... because it very probably isn't complete.


Well, Anthony, I'm not sure I can agree with you that "shoot-outs" (otherwise known as product comparisons) are intrinisically bad. 3D cards, like monitors, automobiles, and dishwashers, are commodity products and there are entire genres of traditional magazines which have been built around the premise that product comparison is a very good thing for consumers, from Consumer Reports to Motor Trend--it's a long list. Such magazines, and such information, is very important to the mass of consumers who do not have the time to personally buy and try every product out there, not to mention not having the money to do it either...;) If consumers were capable of this then these respected sources of information wouldn't exist, would they? Their popularity over decades shows the degree to which this approach to product review is appreciated and desired by a sizable mass of the consuming public.

When I bought my last car, for instance, I spent a couple of months reading paper mags like Car & Driver, Motor Trend, etc., and visiting all kinds of websites from the corporate ad pages to the forums devoted to nothing but people recounting their experiences with a particular automobile. I narrowed the list and went to dealers and drove, and compared automobiles, and made the purchase from my original list. That was five years ago, and the car looks like new and has yet to visit the shop for anything but a tune up. So I was very appreciative of all of the comparative information I was able to view prior to actually making that purchase.

Of course, it wasn't quite the same with a 3D card, because I doubt I'll ever spend $40K on a 3D card....;) But I think the principles are the same. People have a limited amount of money and time and there's a great need for comparative information because of that. And so I don't envision comparative formats as ever becoming optional for most people. If one web site or magazine doesn't provide that kind of information, folks will just migrate to where they can get it, IMO.

One of the strongest reasons for solid comparative journalism I think is that it serves as a counter to the mass-publicity companies generate to sell their products. All of that information is one-sided, of course, and good investigative consumer journalism serves as a needed counter to that. Such journalism has a well-respected and well-deserved positive reputation, IMO. I frequently disagree with the opinions expressed in publications such as Consumer Reports, often because they are written from perspectives and needs which differ from my own. But I still read such material from time to time because it oftens provides an insight that I find valuable later on. And as you know the only thing Consumer Reports does is to compare like products and offer up opinions on them along with product recommendations. Ie, they don't shirk from telling you which products they like, and which they don't, and why. Or they might have several products they like, but only one or two of which they recommend, etc.

Last, there are differences in products of all types, including 3D cards. Good comparative product journalism highlights the differences, explains them, and expounds on them in as much detail as is warranted for the situation, or is possible to know. Seriously, there's a great hunger in the market for quality comparative product information--key word being "quality."

I guess I can understand your feeling that product comparisons sometimes create "furors", but if you step back a bit and look at it such furors are usually short-lived and burn themselves out after running their course. In a sense, they simply provide an outlet, or vent, for people to get their priorities in order. But I really don't think the occasional furor which erupts at the disclosure that this or that product manufacturer, or this or that product journalist, has said or done something offensive, or something dishonest, is anything but a natural reaction. Most consumers take a dim view of having the wool pulled over their eyes regardless of the format in which it occurs. The airlines, the stock market, the government--or 3D card companies--it doesn't matter--if a sufficient number of people become offended by the actions of these entitites a furor will erupt. But it will usually die off as quickly as it started provided people can vent--which is one of the great aspects of a generally free press, I think. It's just the way things are, IMO.

With respect to nVidia and the issue you bring up, I kind of agree.

I wouldn't characterize it as an "optimization" simply because this is not a case of nVidia reordering instructions as to how its hardware will do full trilinear filtering simply to execute it more efficiently. Rather, it's a case of them doing something less than full trilinear while at the same time not bothering to inform anybody of the fact--and indeed, as you've stated, stating they were doing the opposite of what they did.

However, handled differently, they could have received praise for what they did had they offered it as an option and not at the expense of full trilinear support. Just IMO, of course...;)


Tim Sweeney said:
Anthony,

We're not in the business of publically critiquing the nuances of drivers or approving/disapproving them. We only get involved in the driver situation publically when there is a major driver bug that has a significant negative impact on people who play our games, and that hasn't happened in years. Things like mipmap bias factors and trilinear optimizations/shortcuts don't fall in this category; they are subjective and it's best for users and web critics to debate what performance/quality tradeoffs are acceptable.

We often have informal conversations with NVidia and ATI about performance/quality tradeoffs and shortcuts, but these are private discussions and not public endorsements or rejection of the tradeoffs they make.

Long term I would like to see the rendering API defined in terms of exactly reproducable results, so that there isn't room for debate on things like texture filtering optimizations or mipmap biasing. Then, a given set of triangles and render state passed down to 3D hardware is guaranteed to give the exact same results on all hardware and any software emulation or fallbacks. There are general and non-controversial definitions of how all such things should work, from IEEE 752 for floating point to Fundamentals of Computer Graphics for the definition of trilinear filtering and mipmap factor calculation.

-Tim

Although I'm a little puzzled as to why Tim might think that comparing bilinear filtering to trilinear is somehow "subjective"--I wouldn't agree with an opinion postulating it is--I'm really not sure Tim is actually saying that. His final paragraph seems to welcome the use of objective criteria on which all hardware comparisons should be made. And I'd agree with that as I certainly think it's possible. True, we'll never get to a point where all manufacturers implement hardware features exactly the same way--but we wouldn't want to be in that position I don't think since that would mean all hardware would be identical. How incredibly dull that would be, but little things like patents and copyrights ensure that will never be the case. And that's good, because differences drive the market competitively, and competition benefits the consumer.

While I'm really not sure what Tim is attempting to say here, I would like very much to suggest to him that he might prevail upon Atari not to advertise specific IHVs within the context of its games since this creates exactly the kind of false impression relative to "optimizations" that Tim says in the first paragraph Epic doesn't involve itself with from the standpoint of any particular IHV. I certainly think that would be helpful, don't you? Especially since Tim is making it quite clear Epic does not involve itself in such matters.
 
:rolleyes: Beyond3D Chinese version? Rookie have tried to do that, but he have not time to do that before 2 years ago... :rolleyes:
 
Back
Top