Ops,"I" did it again. Why does Futuremark still ke

Discussion in 'Graphics and Semiconductor Industry' started by engall, Aug 9, 2003.

  1. Mariner

    Veteran

    Joined:
    Feb 6, 2002
    Messages:
    1,602
    Likes Received:
    241
    I know what you mean here (spelt kowtow, by the way), but the way you have written it brings a peculiar image to mind! :D
     
  2. Reverend

    Banned

    Joined:
    Jan 31, 2002
    Messages:
    3,266
    Likes Received:
    24
    99% of your posts show that you're in a irritated state. The reason why you're irritated is obvious but I think your health will improve if you calm down a bit when participating in a public forum such this site's (which btw should also help improve this site's forum quality).
     
  3. AzBat

    AzBat Agent of the Bat
    Legend Veteran

    Joined:
    Apr 1, 2002
    Messages:
    5,951
    Likes Received:
    1,745
    Location:
    Alma, AR
    Great posts Walt. I would have to agree with everything you said. In fact, if I were still writing benchmark articles for JPA I would have written the same thing. In fact, why don't you forward your comments to my former boss which still works for Jon at Jon Peddie Research? Her email address is kathleen@jonpeddie.com. Tell her I sent you and maybe she'll include them in next week's TechWatch Report.

    Anyway, FM really needs a reality check. During the time I was doing beta testing for 3DMark since before the creation of Futuremark, there was no cost to IHVs to be part of the program that I was ever aware of. I understand that they need capital to pay for development costs, but they should seriously reconsider their business model. I remember the day that Nathan Harley came to San Francisco to meet with me, my boss and Jon Peddie about their plans for Futuremark. I was excited and genuinely liked the company's new ideas. I don't know when they started making IHVs pay to be part of the program, but I would have to say that probably started their demise. It's pretty bad when you fire your whole Toronto(marketing) office which included a founder, but to also have another founder leave to work on MP2 and yet another to leave after this fiasco. I want everybody to know that I was once approached by Nathan and AJ to work on the 3DMark program. Things didn't work out. I guess it was bad timing, but I'm kinda glad it didn't work out because I don't think I could have stayed with the company knowing what's happened recently. I'm even ashamed that my name is still listed in the 3DMark credits. In my eyes, FM has only 2 things they can do to regain my trust...

    1) Quit accepting money from IHVs to be part of the 3DMark BETA program. Make your money from end-users, OEM bundles, selling of benchmark data, support services and custom demo services. If that's not enough, then think of other ways to make money, but do not continue the conflict of interest by requesting thousands of dollars from IHVs.

    2) Stand-by your original stance posted in the white-paper. This means actively searching for cheats and optimizations. If any are found then they need to be rejected publicly and results removed from the database. In other words, grow a spine and don't let the IHVs push you around. The point is that all hardware should run the exact same code. If you want to provide additional tests that are not included the score, but also allow shader replacements, then that's fine, but DO NOT ALLOW application specific be used solely for increasing scores or performance.

    Tommy McClain
     
  4. Pete

    Pete Moderate Nuisance
    Moderator Veteran

    Joined:
    Feb 7, 2002
    Messages:
    4,944
    Likes Received:
    348
    I don't see the conflict in your first point, Walt. The public could care less about the black box that is game code and hardware features and drivers--they just want a good-looking game at playable framerates. It's the developer's job to deliver that, multiple hardware standards or not. It's an uneven "playing field" to begin with, if you consider differing feature sets or hardware optimizations to be uneven. I consider it a reality that must be dealt with. FM can surely devise benchmarks that reflect well on both ATi's and nV's current hardware, as game developers are forced to do.

    I agree with your second point. It seems antithetical to 3DMark's goal of an impartial benchmark to allow IHVs pay to be Beta members. It's almost like blackmail--FM starts charging after they find themselves the only game in town in terms of one-click 3D evaluation,* and IHVs are forced to pay a relatively small fee in relation to their overall profits (consider it part of their marketing budget) to stay competitive in the eyes of the always-important VARs (like Dell). From this perspective, I can agree with nVidia's dislike of 3DM03, but I still can't excuse the actions they took to achieve their goal of discrediting 3DM.

    In the end, 3DM seems to still be a potentially useful tool if you know how to use and interpret it--much like any other benchmark, and moreso given the ever-widening field of IHV-forced "optimizations." Just as you can't extrapolate Quake 3 performance to Splinter Cell or Tiger Woods or even another FPS based on the Q3 engine, you shouldn't expect a catch-all benchmark written by a single group of developers. If reviewers and companies want to present their less-diligent readers/consumers with an easy-to-digest number, they're better off with an amalgamated benchmark like ExtremeTech's GameGauge that takes into account performance of a variety of game categories coded by a variety of developers. They're still bound by developers including a repeatable, simple benchmark mode, though, and that's one area where FM excels.


    * Quick, someone patent this before Amazon does! ;)
     
  5. Solomon

    Newcomer

    Joined:
    May 14, 2003
    Messages:
    29
    Likes Received:
    0
    Location:
    In a House
    Well Said...
     
  6. WaltC

    Veteran

    Joined:
    Jul 22, 2002
    Messages:
    2,710
    Likes Received:
    8
    Location:
    BelleVue Sanatorium, Billary, NY. Patient privile
    Thank you, Tommy. I found your post fascinating--very interesting, indeed. I'll tell you, since you and I share the same opinions here, but you have some in-depth experience with some of the principals involved which I lack, it might be better for you to forward these sentiments to Kathleen as your being able to flesh out the background adds a worthwhile dimension (or, if you'd rather, feel free to reference me to whatever degree you'd like.) I, of course, agree with your conclusions as well. What puzzles me is that what's so obvious to us apparently isn't to FM. I guess we can only hope they'll do some soul searching about where their present model appears to be taking them.

    And thank you, too, D....;)
     
  7. WaltC

    Veteran

    Joined:
    Jul 22, 2002
    Messages:
    2,710
    Likes Received:
    8
    Location:
    BelleVue Sanatorium, Billary, NY. Patient privile
    Pete, what I was talking about there was the public expectation that running 3dMK03 on competing 3d hardware made by different companies will provide them comparative results which tell them something meaningful about those products in relation to each other (which they assume they can infer by looking at the numerical scores the benchmark puts out--FM's "hall of fame," etc.) But since the IHVs contribute financially to FM they each expect that the resulting benchmark software will produce scores flattering to their products, since otherwise the fees they pay to belong to the program would not be justified from their point of view. Therefore the IHVs have one expectation, the public which runs the benchmark another. Thus, the conflict of interest.
     
  8. BetrayerX

    Newcomer

    Joined:
    Aug 13, 2003
    Messages:
    74
    Likes Received:
    0
    Location:
    PR


    If FM changed their stance about the banchmark. Ain't that a breach of the agreement you supposdely signed (when clicked yes), invalidating it? Thus, can a refund of the money be asked?
     
  9. BetrayerX

    Newcomer

    Joined:
    Aug 13, 2003
    Messages:
    74
    Likes Received:
    0
    Location:
    PR
    Re: Ops,"I" did it again. Why does Futuremark stil

    Yeah, let the cheats begin.

    Ladies and Gentlemen:
    In the left corner we have the 800 pound gorilla...the master of all cheats, the discrediter of benchmarks, the optimizer of useles tools....Ladies and gentleman, let's give a big welcome (right) to NVidiaaaaaaaaa!

    Who can beat that beast? Quackmark03?
    LOL, if ATI comes with something like this (and I hope they do), I dare someone to call them cheaters or something n those lines. I'll go personally to his/her home to kick his/her ass. :lol: :lol: :lol: :lol:
     
  10. Mark

    Mark aka Ratchet
    Regular

    Joined:
    Apr 12, 2002
    Messages:
    604
    Likes Received:
    33
    Location:
    Newfoundland, Canada
    Hah, funny you should mention that. I asked for a refund from FM for exactly that reason, to which they denied. I would post the email exchange I had with them, but I'm not sure they would appreciate it... I'll think about it anyways...
     
  11. BetrayerX

    Newcomer

    Joined:
    Aug 13, 2003
    Messages:
    74
    Likes Received:
    0
    Location:
    PR
    Who cares about what they (FM) appreciate?

    Did they cared when they changed politics like a pair of dirty socks? (actually they took out the clean ones then got the dirty ones...you get the point). :wink:

    I wonder if someone might sue FM for fraud.....thats what they did, basically....the change of politics is so 180 deg that no one can deny it KILLED 3dMark's credibility, and of course, usefulness.
     
  12. BetrayerX

    Newcomer

    Joined:
    Aug 13, 2003
    Messages:
    74
    Likes Received:
    0
    Location:
    PR
    So, NVidia paid only God knows how many hundreds of thousands just to have the logo on their website?

    Cut the crap, we have been swiming in it for more than 6 months now.
     
  13. BetrayerX

    Newcomer

    Joined:
    Aug 13, 2003
    Messages:
    74
    Likes Received:
    0
    Location:
    PR

    I wonder if ATI could potentially sue FM...they could also ask for a refund too (and a biiig one I bet)
     
  14. micron

    micron Diamond Viper 550
    Veteran

    Joined:
    Feb 23, 2003
    Messages:
    1,189
    Likes Received:
    12
    Location:
    U.S.
    BetrayerX = :roll:
     
  15. AzBat

    AzBat Agent of the Bat
    Legend Veteran

    Joined:
    Apr 1, 2002
    Messages:
    5,951
    Likes Received:
    1,745
    Location:
    Alma, AR
    Walt,

    You're welcome. I went ahead and forwarded your comments to Kathleen. I gave you full credit, but if I had your email address I could have sent that along too. Anyway, I'll let you know what I hear.

    As for FM, I did receive a PM from one of my contacts at FM. Evidently he wasn't too pleased with my comments, but like I told him it's a good thing I'm no longer in the business of writing benchmark articles for JPA. ;) Personally I would had hoped that they would understand where I and others were coming from, but I believe the problem they're having is mainly staying alive through all this mess. If I was in charge I think I would rather die fighting and keep my honor and integrity instead of worrying about losing my job. Jobs come and go, but it's hard to get back honor and integrity after you've lost it. If anybody knows about this, it's me as I was "laid off" by JPA. I'm not proud of how I left the company, but it's a good thing I had long and great relationships with them or it could have been a lot worse. Anyway, hopefully some of these comments will sink in their head.

    Tommy McClain
     
  16. WaltC

    Veteran

    Joined:
    Jul 22, 2002
    Messages:
    2,710
    Likes Received:
    8
    Location:
    BelleVue Sanatorium, Billary, NY. Patient privile
    I would hope they'd realize that the integrity of their software is at stake and has been put at risk. If the general perception of FM benchmarks becomes one of skepticism and mistrust, it won't take the IHVs long to figure out that they are not getting their money's worth any longer and they'll drop off anyway. The sum total is that FM's position will actually be much worse at that point than it was when nVidia quit. But that's the long view, and many companies today don't believe they can afford to take a long view about much of anything. I agree--if such interest conflicts are obvious to outsiders, why can't FM see them? Perhaps they just don't want to...
     
  17. BetrayerX

    Newcomer

    Joined:
    Aug 13, 2003
    Messages:
    74
    Likes Received:
    0
    Location:
    PR
    micron = :?:


    What's that supposed to mean dude?

    What would you do when a product doesn't do what it was promised to do?
    Return it and ask for a refund. That's exactly what FM has done and I believe people and companies as clients and partners that have invested some hard earned money must have a say in the matter. This move is not only a simple Hypocritical decision from FM but also one where no one else was asked for even an opinion. If you as a costumer might feel ripped off by it, what can you expect from companies that endorse some huge amounts of money in FM.
     
  18. micron

    micron Diamond Viper 550
    Veteran

    Joined:
    Feb 23, 2003
    Messages:
    1,189
    Likes Received:
    12
    Location:
    U.S.
    You figure it out dude......
     
Loading...

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...