Use of Custom Demo's In Reviews

Discussion in 'Graphics and Semiconductor Industry' started by Dave Baumann, Jun 16, 2003.

  1. Doomtrooper

    Veteran

    Joined:
    Feb 6, 2002
    Messages:
    3,328
    Likes Received:
    0
    Location:
    Ontario, Canada
    I agree with this site as credability speaks for itself, while some others I shall not name, it would be hard not question if the demo is truly 'secret'...you know somehow it slips into a email message by mistake :p
     
  2. Hellbinder

    Banned

    Joined:
    Feb 8, 2002
    Messages:
    1,444
    Likes Received:
    12
    all you have to do is do it 3dgameman style.

    Instead of just a chart with the numbers for the Custom demo. make videos of the demo actually being run, and then a screenshot of the monitor with the results shown in detail. So people could see the final scores being posted on the monitor.

    Lots of bandwidth needed for that though ;)
     
  3. McElvis

    Regular

    Joined:
    Apr 15, 2002
    Messages:
    269
    Likes Received:
    3
    Location:
    London
    Brandon has posted the Nascar screenshots. He was forced to take actual shots with a camera.

    Although the AA looks better, it's still a blurred mess...
     
  4. Sabastian

    Regular

    Joined:
    Feb 6, 2002
    Messages:
    991
    Likes Received:
    1
    Location:
    Canada
    Yeah the blur is pretty appearent from what I can see it looks like Quincunx blur filter.
     
  5. micron

    micron Diamond Viper 550
    Veteran

    Joined:
    Feb 23, 2003
    Messages:
    1,189
    Likes Received:
    12
    Location:
    U.S.
    Lol, I cant get enough of game girl.
     
  6. Pete

    Pete Moderate Nuisance
    Moderator Veteran

    Joined:
    Feb 7, 2002
    Messages:
    4,945
    Likes Received:
    350
    So what will it come down to? Will a reviewer have to become intimitely knowledgable about each demo, knowing its quirks on each card (per Sharkfood's Morrowind post), and thus create multiple timedemos to reveal overall performance, with graphs being normalized scores encompassing multiple timedemos, and a table below with individual timedemo scores for reference? That sounds like way too much work.

    On the plus side, perhaps it'll wean websites away from posting rushed and usually incomplete (or limited) reviews the day of a card's "release." They'll be forced to take their time and do it right.
     
  7. K.I.L.E.R

    K.I.L.E.R Retarded moron
    Veteran

    Joined:
    Jun 17, 2002
    Messages:
    2,952
    Likes Received:
    50
    Location:
    Australia, Melbourne
    NO! We are more observant.

    I always found it strange that my Radeon's 2xAF done a better job than NV's 4xAF. I also found myself only using Omega's drivers for my former NV20 as they did a damn good job of increasing IQ by removing all the damn blur which occured regardless if AF was on/off.
     
  8. micron

    micron Diamond Viper 550
    Veteran

    Joined:
    Feb 23, 2003
    Messages:
    1,189
    Likes Received:
    12
    Location:
    U.S.
    And this is exactly what we saw happen with the Splinter Cell anomolies. We have good reviewers who know the benchmarks. I simply cant go back to places like Tom's anymore and read a review, without going WTF?
     
  9. Mark

    Mark aka Ratchet
    Regular

    Joined:
    Apr 12, 2002
    Messages:
    604
    Likes Received:
    33
    Location:
    Newfoundland, Canada
    A website with a couple of new demos every few weeks for each of the popular benchmark games (Splinter Cell, UT2K3, Quake3, RtCW, SS:SE, etc, etc) would work I think. Reviewers and readers can download the demos from that one central site if they want to compare performance, but only demos that are newer than the drivers used in the review should be allowed.
     
  10. digitalwanderer

    digitalwanderer Dangerously Mirthful
    Legend

    Joined:
    Feb 19, 2002
    Messages:
    17,274
    Likes Received:
    1,788
    Location:
    Winfield, IN USA
    No, no...I don't think that's it. I think it's something involving mirrors or Bush or something like that... :wink:

    That's my big problem with you folks over there right now, you CAN'T be oblivious to it...yet you don't SAY anything about it! :x

    I thought part of your site's supposed purpose was also to INFORM your members about what was going on...not just you folks yourselves knowing it! :evil:
     
  11. micron

    micron Diamond Viper 550
    Veteran

    Joined:
    Feb 23, 2003
    Messages:
    1,189
    Likes Received:
    12
    Location:
    U.S.
    We've all pretty much agree'd to lay off [H](Brent specifically) in another thread. He's a valuable part of these forums, and I would hate to see him stop coming here :wink:
     
  12. Sharkfood

    Regular

    Joined:
    Feb 7, 2002
    Messages:
    702
    Likes Received:
    11
    Location:
    Bay Area, California
    And why not? They did all that work in the past, but for nothing so noble as to be "objective"- but quite the opposite. :)

    The Quake1 + PowerVR example was a good one. If you wanted to totally annihiliate a PowerVR board- just launch a lot of rockets. Voodoo boards at the time chugged through them pretty okay. PowerVR boards choked down to 1-2 fps. If you wanted to sell 3dfx hardware, you just featured benchmarks that had 5-6 of your best friends on a rocket firing fest. The graphs made the PVR hardware look like it was beaten 15-fold.
     
  13. micron

    micron Diamond Viper 550
    Veteran

    Joined:
    Feb 23, 2003
    Messages:
    1,189
    Likes Received:
    12
    Location:
    U.S.
    It wasnt too long ago I was playing games on a Kyro1 and I dont recall experiencing 1-2 fps at any time during Quake...
    Though I might have been having too much fun to notice.....
     
  14. StealthHawk

    Regular

    Joined:
    May 27, 2003
    Messages:
    459
    Likes Received:
    0
    Location:
    I exist
    You're comparing two different testing situations there. One uses the 42.72 driver, 2x FSAA, and 8x AF, the other uses the 43.03 driver, 2x FSAA, and no AF.

    It is a known fact that nvidia increased AF scores dramatically in the 43.51+ drivers. What exactly they did to increase scores remains a mystery. But the fact remains that when using drivers prior to 43.51 enabling AF incurred a larger performance hit. Couple this with the increase in clock speed, and there is nowhere near a 3x performance increase. Although nvidia might have done some "optimizing" for the [H] UT2003 benchmark, your post does not include enough evidence to draw such a conclusion.
     
  15. jb

    jb
    Veteran

    Joined:
    Feb 6, 2002
    Messages:
    1,636
    Likes Received:
    7
    What about a different reveiw for each new set of drivers?

    This way with a given set of drivers you can run your new top secert demo, post results on your killer review then give linkage to the masses. This allows for some level of comparison to the public.

    But this also helps to prevent demo level optimization as when the IHV says here are our new drivers you, smile and say thanks, then use a different demo to test. This way the IHV never really knows all of the demo details unitl its made public.

    However this puts more work on the review sites.

    I would also recomd maybe some of the bigger sites working together. To share these demos to help keep some consistancy.

    Or was this what Sharkfood was getting at? If so my bad.
     
  16. Doomtrooper

    Veteran

    Joined:
    Feb 6, 2002
    Messages:
    3,328
    Likes Received:
    0
    Location:
    Ontario, Canada
    Yep I linked the wrong graph, still a significant increase without AF..so my assumption is just fine.

    [​IMG]

    [​IMG]

    Still over double from the 42.72 drivers:

    2X FSAA/8X AF 42.72: 20.2
    2X FSAA/8X AF 43.03: 51 FPS !! or almost 2.5 times the performance on the same benchmark, and no 50 mhz core clock doesn't account for it.
    I compared no AF scores also the speed improvement is the same, so your optimized AF idea is wrong.
     
  17. fallguy

    Veteran

    Joined:
    Jun 17, 2003
    Messages:
    1,367
    Likes Received:
    11
    One of those graphs is the older type, while one of them is of the newer type. Leading me to think there was some time between the reviews. Are both review PC's the same?
     
  18. Doomtrooper

    Veteran

    Joined:
    Feb 6, 2002
    Messages:
    3,328
    Likes Received:
    0
    Location:
    Ontario, Canada
  19. Randell

    Randell Senior Daddy
    Veteran

    Joined:
    Feb 14, 2002
    Messages:
    1,869
    Likes Received:
    3
    Location:
    London
    I dont think Sharkfood is talking Kyro..
     
  20. micron

    micron Diamond Viper 550
    Veteran

    Joined:
    Feb 23, 2003
    Messages:
    1,189
    Likes Received:
    12
    Location:
    U.S.
    Ok, which PowerVR board was he talking about then?
    Sorry if I'm showing blatant stupidity :?
     
Loading...

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...