Whatever happened with Futuremark and nVidia with 53.03?

Discussion in 'Graphics and Semiconductor Industry' started by digitalwanderer, Jan 5, 2004.

  1. dan2097

    Regular

    Joined:
    May 23, 2003
    Messages:
    323
    Likes Received:
    0
    Well that the thing isnt it, with all the cheating in 3d mark 03 Nvidia have been doing even with drivers that are possibly legitimate using the newest version of the program reviewers dont want to use it anymore. Looks like Nvidia won :shock:
     
  2. Bouncing Zabaglione Bros.

    Legend

    Joined:
    Jun 24, 2003
    Messages:
    6,363
    Likes Received:
    82
    It's a very phyric victory. Look at how many cards Nvidia sold on the back of 3DMark2001 - how are they going to do that when no one trusts FM or Nvidia anymore?

    In the long run FM may be dead in the water, but in killing 3DMark off, Nvidia have gained the reputation of being untrustworthy cheaters and liars in all forms of benchmarking, as well as destroying one of their (previously) favourite marketing tools.
     
  3. dan2097

    Regular

    Joined:
    May 23, 2003
    Messages:
    323
    Likes Received:
    0
    Futuremark isnt dead yet, they still plan to make 3d mark 04/5/whatever. Im sure nvidia will support it if their current top of the line card is shown in a good light in it :wink:

    EDIT: the general public may believe Nvidia otpimizes specifically/cheats on 3d mark 03 but most do not know of any other cheats. i.e. that they app detect code creatures, that they perform best in popular time demos rather than actual gameplay etc.

    Just as a by the by do most sites use the halo built in benchmark for benchmarking, as most web sites show the fx 5950 and 9800XT equal on halo while hardocp which explicity say they use a custom demo shows a large performance difference? :?
     
  4. Brent

    Regular

    Joined:
    Apr 11, 2002
    Messages:
    584
    Likes Received:
    4
    Location:
    Irving, TX
     
  5. dan2097

    Regular

    Joined:
    May 23, 2003
    Messages:
    323
    Likes Received:
    0
    Ok its not a demo per say, but you say its in game and specific to your site. (well you changed it between reviews but you know what I mean)

    Do you still have the fx 5950 and 9800XT? If so what is the difference in performance on the halo built in benchmarks, on your in game test there seemed to be a large difference
     
  6. digitalwanderer

    digitalwanderer Dangerously Mirthful
    Legend

    Joined:
    Feb 19, 2002
    Messages:
    17,219
    Likes Received:
    1,738
    Location:
    Winfield, IN USA
    "That makes Mr.Panda sad!" :cry:
     
  7. Arun

    Arun Unknown.
    Moderator Legend Veteran

    Joined:
    Aug 28, 2002
    Messages:
    5,023
    Likes Received:
    299
    Location:
    UK
    I'd say FM doing a press release restating only the drivers on their approval page are approved (possibly after ATI releases some new ones to justify it?) would do the trick. You don't have to insist the 53.03 are illicit; you just gotta say anything BUT the 52.16 is. I doubt they'd get in as much trouble for that, too.


    Uttar
     
  8. Hanners

    Regular

    Joined:
    Jul 12, 2002
    Messages:
    816
    Likes Received:
    57
    Location:
    England
    It's worth remembering that the nature of the built-in Halo timedemo never made it a good tool for evaluation of actual in-game performance.
     
  9. digitalwanderer

    digitalwanderer Dangerously Mirthful
    Legend

    Joined:
    Feb 19, 2002
    Messages:
    17,219
    Likes Received:
    1,738
    Location:
    Winfield, IN USA
    Who CARES how much "trouble" FM gets in, they're in the right! I wanna see 'em nail nVidia to a bloody cross finally!!! :twisted: :twisted: :twisted:

    (Don't worry 'bout me, I ain't really all psychotic over it...it's just that winter break ended and the kids went back to school today and I'm all giddy. :D )
     
  10. cthellis42

    cthellis42 Hoopy Frood
    Legend

    Joined:
    Jun 15, 2003
    Messages:
    5,890
    Likes Received:
    33
    Location:
    Out of my gourd
    I was talking with Patric about it, since tests were coming out showing PS2.0 results most people didn't get, but after version release it was still "under investigation." So basically, they didn't "know" but found out afterwards, and took the only step they could at that point. (Recinding 52.16 would end up causing the biggest foofrah yet, while adding a cautionary message concerning the 2.0 test at least puts it in perspective.) It's discouraging to have seen them miss optimizations (I don't think any 2.0 ones had been present before, so it likely wasn't a priority) but on the whole it reinforces how hard it is to spot everything if the weight of a huge company is brought to bear.

    And in the meanwhile, as the PS2.0 test has no effect on the main score or talling on the ORB, it was not mission critical to recind 52.16--for which there would be no turning back, and no ability to keep nVidia REMOTELY on board or the suite on track.
    I "mean" very little. All I know is that the internet community is waiting to jump on any further developments and shout it from the rooftops, and at this point I think that would stop any further relations with nVidia and pit them against each other directly--and who is more capable of surviving THAT encounter? For whatever reason, people eventually just shrug off every optimization nVidia makes with an "oh well, they're just up to their old tricks" and eventually don't care, but with 3DMark it's "well I can't trust THEIR benchmark because nVidia always cheats on it!" and it both lingers and becomes the overriding point. o_O If they go all-out right now, what happens? Who's in a better position to survive the carnage? How quickly will everyone shrug THAT off?

    As I don't know what's going on behind the scenes, I have no idea what is the right maneuver at this point it time. I'm SURE, however, that they are attempting to resolve this in a way that doesn't fragment the industry nor their suites, as an overt stance would. It would reinforce the opinion of those already pissed with nVidia's actions, and it might sway a few more people, but eventually FM (3DMark at least) would be dead or useless--and sadly I can't the built-up momentum and "success" of their actions leading to anything but more of it. And then other IHV's having to do it to compete.

    Who's in line to take over? Aquamark 3? Woo! Let the games begin! :evil: Do we really think the industry's positions on "single number bragging points" is going to change? The mindset would have to change first. Illicit optimizations would have to be seen by the development community AS bad and have a stance against it--and notice just how much nothing everyone has said every time previously that this exact matter has come to light? Mmm-hmm.

    Likely not. In the long run, something else would take over as "favorite marketing tool" and not only would the optimizations begin again, but there would be even LESS resistance to them--and none from the developer itself.
    I'm sure they do, but just what support will it end up getting from industry or the enthusiasts at this point? The instigator has already proven eminantly capable of rising above their actions, have shown they won't support a major benchmark unless it shows them in good light (or can be made to), and the much of the community blames FutureMark for the utterly precarious position the industry and nVidia in particular has put them in.

    But hey, I suppose everyone could eventually take the foolish cop-out of saying "all synthetic benchmarks are useless" or think all game benches show an inexorable link to game performance, or that they too can't be optimized for. We're not yet at that state ANYWHERE, and until we are (if we can be), the same actions will freely move where is deemed "most profitable" and snowball in effect.
     
  11. StealthHawk

    Regular

    Joined:
    May 27, 2003
    Messages:
    459
    Likes Received:
    0
    Location:
    I exist
    Here's an interesting question: When are we going to see some approved XGI drivers?
     
  12. cthellis42

    cthellis42 Hoopy Frood
    Legend

    Joined:
    Jun 15, 2003
    Messages:
    5,890
    Likes Received:
    33
    Location:
    Out of my gourd
    Considering what they look like right now, probably not for a while. Heh...
     
  13. karlotta

    karlotta pifft
    Veteran

    Joined:
    Jun 7, 2003
    Messages:
    1,292
    Likes Received:
    10
    Location:
    oregon
    as long as dell is around mark# 4 and 5 will be around
     
  14. Nick[FM]

    Regular

    Joined:
    Feb 8, 2002
    Messages:
    527
    Likes Received:
    7
    Location:
    Helsinki
    All approved drivers are listed here http://www.futuremark.com/community/drivers/?approved. If a driver does not get our approval to be used with 3DMark03, it simply won't be listed there. We test all new official WHQL drivers as they are released, and we will keep that list of approved drivers updated.

    AFAIK the official & available XGI drivers are not WHQL, and thus aren't approved. We only approve WHQL drivers and when we get WHQL drivers from XGI, we sure will test them.
     
  15. Hanners

    Regular

    Joined:
    Jul 12, 2002
    Messages:
    816
    Likes Received:
    57
    Location:
    England
    Are there no plans to announce drivers that fail the tests publically, and give reasons for their failure? I think most people (quite rightly) assumed that this would be the way FutureMark would deal with things.
     
  16. digitalwanderer

    digitalwanderer Dangerously Mirthful
    Legend

    Joined:
    Feb 19, 2002
    Messages:
    17,219
    Likes Received:
    1,738
    Location:
    Winfield, IN USA
    That's IT?!? You won't put 'em on the approved list? What about when websites start benching 'em for reviews? Will you say anything? Or will you happily assume that since the drivers are not on your approved drivers list that people reading the reviews will check your list and realize that the results shouldn't be taken into consideration?

    Do you really bloody mean that's it?!?!?!
     
  17. jimbob0i0

    Newcomer

    Joined:
    Jul 26, 2003
    Messages:
    115
    Likes Received:
    0
    Location:
    London, UK
    Uh-oh..... digi mate - remember that heart of yours....... remember calm cool chilled digi now......

    /me runs off and and sneaks a look at the old threads and what was said and 'hoped' would happen...
     
  18. Doomtrooper

    Veteran

    Joined:
    Feb 6, 2002
    Messages:
    3,328
    Likes Received:
    0
    Location:
    Ontario, Canada
    Friendly Reminder Time

    LawyerMark 2004
    Code:
                               =
                               =
                               =
       =                       =
       =                       =
    Futuremark              Nvidia
     
  19. digitalwanderer

    digitalwanderer Dangerously Mirthful
    Legend

    Joined:
    Feb 19, 2002
    Messages:
    17,219
    Likes Received:
    1,738
    Location:
    Winfield, IN USA
    Yeah, yeah. :roll: Thanks Jim, I keep forgetting I ain't supposed to get all worked up anymore over silly shit...it's just a hard concept to get used to for me.

    Lemme know what ya find please, I'm gonna go exercise or eat something disgustingly healthy and meditate for a bit. :)
     
  20. Nite_Hawk

    Veteran

    Joined:
    Feb 11, 2002
    Messages:
    1,202
    Likes Received:
    35
    Location:
    Minneapolis, MN
    Hi worm,

    I want to thank all of you at futuremark for being willing to go through and test each driver revision from all these companies to make sure they comply with your guidelines. I'm sure it must be a lot of work, and you guys probably have to walk a bit of a tightrope when it comes to politics in the field.

    I have a question about the policy you quoted above though. Given that I'm sure it takes a fair amount of time to actually test the drivers to make sure there arn't any performance altering bugs in them, how should we as readers distinguish between drivers that you have tested and failed (which do not conform to futuremark's guidelines) versus drivers that you have not yet tested or have not finished testing (which may conform to futuremark's guidelines)? This distinction is something that I'm guessing a lot of your users would be interested in knowing.

    Thanks,
    Nite_Hawk
     
Loading...

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...