How benchmark 3D Cards, today

Discussion in 'Graphics and Semiconductor Industry' started by Griffith, Mar 3, 2006.

  1. FrgMstr

    Newcomer

    Joined:
    Jun 26, 2002
    Messages:
    223
    Likes Received:
    4
    Location:
    Lucas, TX
    Unless you live in the land of the fairies..... The fact of the matter is that time and money resources DO give you the answer in the real world. Theoretically I would have done a lot of things differently over the last 8 years, but the fact is that we simply have not had the resources to accomplish all of those lofty goals. So we work inside of our means. Do we make concessions? Yes we do.

    So you can sit here and theorize just exactly how it SHOULD be done, but that just may not be a solution that is realistic.
     
  2. Geo

    Geo Mostly Harmless
    Legend

    Joined:
    Apr 22, 2002
    Messages:
    9,116
    Likes Received:
    213
    Location:
    Uffda-land
    Well, this is what I was responding to: http://www.beyond3d.com/forum/showpost.php?p=712811&postcount=93

    I already said a periodic visitation of the issue is a reasonable compromise given the resource limitations. I can't look around and find anyone else who's doing what I'd like, so I certainly can't fault [H] for failing to meet a standard.

    But I don't have any problem between knowing what I'd like to see on the one hand and what it is reasonable to expect at any given moment on the other. Indeed, it's kinda hard to move over time to transition the first into the second without that knowledge. . . It's what roadmaps are all about, right?
     
  3. FrgMstr

    Newcomer

    Joined:
    Jun 26, 2002
    Messages:
    223
    Likes Received:
    4
    Location:
    Lucas, TX
    Roger that, fully understood. I think we are on the same page here.
     
  4. GrapeApe

    Newcomer

    Joined:
    Apr 3, 2004
    Messages:
    57
    Likes Received:
    2
    Location:
    Calgary, Canada
    And also keeps the sponsors very happy I'm sure. And considering...

    Ever think of getting ASUS or Gigabyte (both sellers of both ATi and nV) to sponsor? Or maybe to them sponsorship isn't as vital because they know that molding the reviews and the community isn't as important because in the end they have you regardless of which camp people choose.

    While I agree that it's easier to avoid the optimization/IQ debate the way B3D does with their reviews of just ATi and nV cards, but if you're trying to scale the cards and cpu you don't think people would accept a little compromise, especially since there hasn't been much focus on the issue in prior reviews for them to care in the first place?

    I guess page-hits and advertising alone brought from good reviews isn't enough to avoid the need for 'sponsored reviews'. However I'd never take one of those any more seiously than I would the Pepsi challenge. Why should I think you (or anyone else, doesn't matter whom) wouldn't omit a poor result for the whole line if you should come across a performance hole?

    Too bad there's no equivalent Consumer Reports for tech, unfortunately everyone in the hardware industry is worried about all the PR money out there they can get a chunck of and they risk losing.
     
  5. Mendel

    Mendel Mr. Upgrade
    Veteran

    Joined:
    Nov 28, 2003
    Messages:
    1,350
    Likes Received:
    17
    Location:
    Finland
    Eh, what?
     
  6. Geo

    Geo Mostly Harmless
    Legend

    Joined:
    Apr 22, 2002
    Messages:
    9,116
    Likes Received:
    213
    Location:
    Uffda-land
    And thar' she blows!: http://enthusiast.hardocp.com/article.html?art=MTAwMiwxLCxoZW50aHVzaWFzdA==

    Edit: I understand how sponsorship would be helpful (duh), and have no reason to question the statement at the top that BFG had no input or "preview" of the results. But I also can't help but note that there have been suggestions in the past that ATI and NV might have some differences in the degree to which the drivers rely on the cpu in some instances. . .and it would have been nice to have had the opportunity to test that theory. Ah well.

    EditII: Tho having quibbled, still appreciate having the article at all. :)
     
    #106 Geo, Mar 25, 2006
    Last edited by a moderator: Mar 25, 2006
  7. FrgMstr

    Newcomer

    Joined:
    Jun 26, 2002
    Messages:
    223
    Likes Received:
    4
    Location:
    Lucas, TX
    Yes, it would have been nice to have done that, but the fact of the matter is that it took Brent a month to cover what we did.

    Another reason we went with BFG is because they are simply easy to do business with. I think the entire negotiation on the sponsorship negotiation took less than 10 minutes. [H]: "We are doing a CPU scaling article, due to time, we are going to limit video cards used. We need these cards and $X to fund the project and it will make you the only video card builder noted." BFG: "OK"

    I think the article is some of the best work we have done in the last year. That said, almost NO OTHER hardware site on the net linked it (Yes, a couple did but no real mention was made of it much anywhere.) So was it just that bad, or did our real world testing and results fly in the face of their benchmarks that they have based their past opinions on?
     
  8. digitalwanderer

    digitalwanderer Dangerously Mirthful
    Legend

    Joined:
    Feb 19, 2002
    Messages:
    17,331
    Likes Received:
    1,822
    Location:
    Winfield, IN USA
    I think a lot of it probably had to do with the "only one video card" thing. Even if there was no impropriety and the intention was to limit the results to just the CPU it probably gave the impression of bias given some past leanings/tendancies.

    Just my .02 on possibilities, I don't mean it as a dig or a troll. I didn't post it 'cause I haven't been posting news much lately, the family has got me too busy. :roll:
     
  9. Geo

    Geo Mostly Harmless
    Legend

    Joined:
    Apr 22, 2002
    Messages:
    9,116
    Likes Received:
    213
    Location:
    Uffda-land
    Yes, that is a pity. I hope that's an indication of misreading the interest level of their readers in such comparisons. I mean, obviously I hope that as I started the digression upstream.
     
  10. Tim Murray

    Tim Murray the Windom Earle of mobile SOCs
    Veteran

    Joined:
    May 25, 2003
    Messages:
    3,278
    Likes Received:
    66
    Location:
    Mountain View, CA
    I have to question the selection of games, though. 3 FPS games and WoW? Why not games from different genres?
     
  11. FrgMstr

    Newcomer

    Joined:
    Jun 26, 2002
    Messages:
    223
    Likes Received:
    4
    Location:
    Lucas, TX

    WoW because is was the #1 selling PC game in 2005 by a margin of 100%. And the rest because they are still hugely selling games that can heavily stress a video card.

    http://biz.gamedaily.com/charts/?id=182
     
  12. FrgMstr

    Newcomer

    Joined:
    Jun 26, 2002
    Messages:
    223
    Likes Received:
    4
    Location:
    Lucas, TX

    The reader feedback was AMAZINGLY POSITIVE. You can read our forum replies here.

    http://www.hardforum.com/showthread.php?t=1031925
     
  13. Tim Murray

    Tim Murray the Windom Earle of mobile SOCs
    Veteran

    Joined:
    May 25, 2003
    Messages:
    3,278
    Likes Received:
    66
    Location:
    Mountain View, CA
    Well aware of the sales figures for each--I just don't think that three games that stress the video card specifically is necessarily the best idea for a CPU scaling article. Why not Age of Empires 3 or Star Wars: Empire at War? Admittedly I've never played either (although I do remember AoE 3 being considered not too easy on the video card), but I think that one of those would give a more interesting spin on CPU scaling.
     
  14. Geo

    Geo Mostly Harmless
    Legend

    Joined:
    Apr 22, 2002
    Messages:
    9,116
    Likes Received:
    213
    Location:
    Uffda-land
    Well, not amazing to me. :wink: Gratifyingly, that's a good word. Hope you guys are inspired by your reader's reactions to revisit periodically. . .once a year'd be nice.

    Edit: http://www.hardforum.com/showpost.php?p=1029182261&postcount=12 That's a pretty decent thumbnail of what I took out of it at the top, tho a Quake4 would have shown there is some reason to hope on the dual-core front.

    EditII: As an aside, just going thru MS GDC multi-core presentation .ppt. The happy thot I took from that, and the fact that one of the presenters was an XB360 guy, is that the next gen consoles should do a lot to light a fire under this in the medium-to-longer-term.
     
    #114 Geo, Mar 28, 2006
    Last edited by a moderator: Mar 28, 2006
  15. FrgMstr

    Newcomer

    Joined:
    Jun 26, 2002
    Messages:
    223
    Likes Received:
    4
    Location:
    Lucas, TX
    Hehe, when Intel starts doing public contests that are looking for "Best Threaded Game," it is time to start worrying.
    http://www.hardocp.com/news.html?news=MTc3MTYsTWFyY2ggICAgLDIwMDYsaG5ld3M
     
  16. poopypoo

    Veteran

    Joined:
    Jun 14, 2005
    Messages:
    1,026
    Likes Received:
    13
    Location:
    Hong Kong
    I'm as confused by this as i am by your conclusion -- that dual cores make no positive difference to gamers. This is a pretty odd conclusion to draw without having a test, "BF2 with 6 filesharing apps, GAIM, and a virus scan running". Admittedly, we currently all try to avoid such situations, but isnt the whole point of dual core (for gamers) to allow us to play (at least rather dated games like WOW) without tanking if the virus scan kicks in? 9_9
     
  17. FrgMstr

    Newcomer

    Joined:
    Jun 26, 2002
    Messages:
    223
    Likes Received:
    4
    Location:
    Lucas, TX
    I would suggest what you are describing is a difference in multitasking, not gaming. We covered this in 2005 and it is very easy to show.

    [​IMG]

    The page is located here with more data.

    I also added this to the conclusion page.

    Hope this helps.
     
  18. poopypoo

    Veteran

    Joined:
    Jun 14, 2005
    Messages:
    1,026
    Likes Received:
    13
    Location:
    Hong Kong
    Brent's review is ambitious and thorough, and your response to my post was also quite thoughtful. My point, however, was simply that dual core seems like a great idea right now, because even though the performance gains in "pure gaming" are well-known to be negligible, does anyone actually run gaming machines without at least AV, IM, and BT running? Throw in F@H, firewall, fileserving, and (my favorite) scheduled PVR, and you've got quite a case against single-core gaming, IMO. Sorry if my earlier comment was curt -- I have absolutely no interest in dissing Brent's great scaling article (I will probably bump my A64 o/c up a little higher in response to the results) -- I just thought that part of the conclusion seemed a little unwise, given the real-world needs of most gamers. Lol... in this, a thread on GPU review techniques (and not specifically yours), I posit that different methods of testing CPUs could be employed in your review(IMO, with benefits). ^^;
     
  19. FrgMstr

    Newcomer

    Joined:
    Jun 26, 2002
    Messages:
    223
    Likes Received:
    4
    Location:
    Lucas, TX
    Oddly enough, even though there will be little else that multi-core processors in the forseeable future, I am talking to more and more people every day that are using their desktops ONLY for gaming. I am finding that many folks are using their laptops for all their other needs, but going back to the fullsize desktop when they want to get their frag on. :) This is a statement to just how powerful our laptops have gotten.
     
  20. digitalwanderer

    digitalwanderer Dangerously Mirthful
    Legend

    Joined:
    Feb 19, 2002
    Messages:
    17,331
    Likes Received:
    1,822
    Location:
    Winfield, IN USA
    Or how indispensable desktops are to gaming, depending on how you read it. ;)
     
Loading...

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...