FEAR & Quake 4 benchmarks?

Discussion in '3D Hardware, Software & Output Devices' started by Carl B, Oct 19, 2005.

  1. Carl B

    Carl B Friends call me xbd
    Moderator Legend

    Joined:
    Feb 20, 2005
    Messages:
    6,266
    Likes Received:
    63
    LOL, my bad. ;)

    I'm not up on who on the board works for who, except for some of the individuals I'm more often in discussions with.

    Sorry Humas! My context detector wasn't functioning properly. :cool:

    (great job on the engine tweaks by the way)
     
  2. Hellbinder

    Banned

    Joined:
    Feb 8, 2002
    Messages:
    1,444
    Likes Received:
    12
    They did.. All the products currently available on the market still do.

    It remains unseen how well they will do in all OpenGL games. Finally Doing something 5 years after the problem was identified for one product line thats not even available yet is an accomplishment but its hardly ground breaking or amazing now is it.

    oh and what about 2x AA and 6X AA etc...
     
  3. Moloch

    Moloch God of Wicked Games
    Veteran

    Joined:
    Jun 20, 2002
    Messages:
    2,981
    Likes Received:
    72
  4. Dave Baumann

    Dave Baumann Gamerscore Wh...
    Moderator Legend

    Joined:
    Jan 29, 2002
    Messages:
    14,079
    Likes Received:
    648
    Location:
    O Canada!
    The current OpenGL results are nothing to do with any different state of driver code and the AA patch is only a normalisation of where 4x should have been in the first place.
     
  5. Joe DeFuria

    Legend

    Joined:
    Feb 6, 2002
    Messages:
    5,994
    Likes Received:
    70
    It's just less useful than something like standard deviation, IMO.

    Except when you combine the average with the standard deviation in both cases...the average with the massive highs will be higher.

    No, because in the situations you proposed, "all else" is not equal.

    I repeat: combine "standard deviation" with "average" and you'll have relatively good idea of "how often it fluctuates into some dangerous range".

    Just give a "minimum frame rate" and you have no idea if it happens once or a hunderd times.
     
  6. trinibwoy

    trinibwoy Meh
    Legend

    Joined:
    Mar 17, 2004
    Messages:
    10,430
    Likes Received:
    433
    Location:
    New York
    Thank you.
     
  7. marco

    Regular

    Joined:
    Jan 30, 2002
    Messages:
    303
    Likes Received:
    0
    Location:
    The netherlands
    FEAR SLI benchies

    What I found rather strange is that I could not find SLI benchmarks and SLI settings tests. When using Coolbits 8 you can see the different SLI modes nVdia provides. And there are some extra FSAA modes as well. So I started to do some test of my own.

    I found some intresting stuff with F.E.A.R. For starters when I come out of the game with soft shadows on, my system hangs. On top of that the game really responds well to split frame rendering and not to alternate frame rendering. When the SLI config is set to auto setup, the game performs not SLI like.

    So here are my results, when there is a Quake4 standard recorded demo, either by us or FS, Anand I'll try that also.

    The benchmark results are displayed with min fps/avg fps/max fps

    * I used 81.85 drivers, winxp SP2
     

    Attached Files:

  8. kyleb

    Veteran

    Joined:
    Nov 21, 2002
    Messages:
    4,165
    Likes Received:
    52
    But if other than the masive highs the one system got lower FPS in general, the two could wind up showing both the same avarage and same standard deviation, eh?

    So you mean all else being equal inlcuding the amount of vram, eh? Sure that would be the case then, but then the situation in question has been where the vram is not equal since before you ever showed up in this thread to contest my point, so your arugment here is off-topic.

    How is that? Sure a small standard devation will tell you it doesn't flucuation too low at least very often, but a large standard devation could come from masive highs or nasty lows and there is no way to tell the two appart.
    Sure, but you know whether drops into "dangerous range" at all, and that is worth something.
     
  9. Joe DeFuria

    Legend

    Joined:
    Feb 6, 2002
    Messages:
    5,994
    Likes Received:
    70
    Yes.

    Eh? No.

    I mean all else being equal: two cards perform "the same" in the vast majority of situations, but for whatever reason, one setup stutters much more than the others. It's going to have a lover "average" frame rate.

    And there's no way to tell how often a "minimum" fps is hit.

    I simply disagree. Not knowing if said "dangerous range" is hit once every hour or once ever 10 seconds isn't worth much at all.
     
  10. Rodéric

    Rodéric a.k.a. Ingenu
    Moderator Veteran

    Joined:
    Feb 6, 2002
    Messages:
    3,986
    Likes Received:
    847
    Location:
    Planet Earth.
    Gizzz guy he said standard deviation & average framerate.
    Massive Highs = higher average
    nasty lows = lower average

    standard deviation = steadiness, clearly something you WANT.

    BTW, this reminds me of this : http://www.beyond3d.com/forum/showthread.php?t=24454
     
  11. Joe DeFuria

    Legend

    Joined:
    Feb 6, 2002
    Messages:
    5,994
    Likes Received:
    70
    Correct, which is why you want both an average and something like a standard deviation. (Or some other measure of framereate fluctuation over time.) The key being over time.

    Taking a "minimum" frame rate that is one instantaneous point over an entire benchmark run says almost nothing about playability.
     
  12. kyleb

    Veteran

    Joined:
    Nov 21, 2002
    Messages:
    4,165
    Likes Received:
    52
    Well then, that shows a situation where standard deviation wouldn't be helpful in ilustating the difference between a card that keeps respectable framerate and one that doesn't.


    But all else isn't going to always be equal and that settup with stutters might also have higher highs, makeing hte average the same or better.

    But at least you can see the minimum that is hit, something the avarage with standard devation won't show you.

    Its woth something when other system minimum never hits dangerous range at all.
     
  13. Joe DeFuria

    Legend

    Joined:
    Feb 6, 2002
    Messages:
    5,994
    Likes Received:
    70
    And I've shown you a situation where only giving a minimum frame rate isn't useful. Point?

    Gee, if you're going to argue like that, I'll say that in your own hypothetical situation all else isn't going to be equal, because the one with "higher highs" is going to have a better average frame rate.

    In an isolated benchmark run a single minimum FPS hit at some point in the demo doesn't tell me ANTHING useful. You need to have some measure of how often it's hit.
     
  14. mjtdevries

    Newcomer

    Joined:
    Mar 25, 2004
    Messages:
    178
    Likes Received:
    1
    Location:
    Netherlands
    What's wrong with posting a graph of the fps during the benchmark as HardOCP does?

    It shows you everything: Average, deviation, whether a minimum value is a single occurance, or if it happens often.

    I agree with kyleb that average and deviation isn't enough. (although it would be a lot better than most reviews currently give) Average, deviation combined with minimum would be better.

    If average is high and deviation is low, you know that a low minimum must have been a spike.
     
  15. kyleb

    Veteran

    Joined:
    Nov 21, 2002
    Messages:
    4,165
    Likes Received:
    52
    I know what he said, and I already pointed out the failure in your argument here before you even made it.
    No, I could care less about steadiness alone; sticking around 40fps and jumping to 200fps would give you a huge standard devation but I'd consider it running fine all the same.
    Yeah it does, and the faults of using standard deviation with average framerate were already pointed out there as well.
     
  16. kyleb

    Veteran

    Joined:
    Nov 21, 2002
    Messages:
    4,165
    Likes Received:
    52
    Exactly, that is what I suggested would be the best way to see the difference that more vram makes way erlier in this thread.

    Yeah, though the minimum in and then the average along with the standard devation does become reasonably useful, still better off with a continuous graph though.
     
  17. Rodéric

    Rodéric a.k.a. Ingenu
    Moderator Veteran

    Joined:
    Feb 6, 2002
    Messages:
    3,986
    Likes Received:
    847
    Location:
    Planet Earth.
    It's too late in here to argue, I'll check the thread tomorrow.

    (I'm not convinced min is usefull.)
     
  18. kyleb

    Veteran

    Joined:
    Nov 21, 2002
    Messages:
    4,165
    Likes Received:
    52
    I have been making the point all along and you have been doing a damn good job of ignroing that point; but again. standard devation doesn't tell you if the framerate ever got low at all, only how varied it was.

    I have been though this one with you a few times now as well, do you need me to draw you a picture of how that isn't always the case or what?
    That won't nessecarly add much, you could hit your minumum of say 14fps only for one faction of second but still wind up with 15fps quite a few second and yet have a few highs keeping the avaerage looking nice.
     
  19. kyleb

    Veteran

    Joined:
    Nov 21, 2002
    Messages:
    4,165
    Likes Received:
    52
    I'm not saying it is a be all end all by any means, simply that if you can see one card hitting something too low, and another card keeping the framerate up the whole time you at least know the second card did at least a little better of a job in the benchmark in reguard to keeping the framerate up. At least to me, that does have some use.
     
  20. HaLDoL

    Newcomer

    Joined:
    Jun 10, 2004
    Messages:
    140
    Likes Received:
    2
    What about 8x? What about 16x when CF actually will be available? Or are ATI cards only gonna be good for 4xAA?
    Maybe ATi should implement a disclaimer in the drivers: "OpenGL game detected: only 4xAA is available because we did not have time for optimising the hell out of other AA options and 4xAA is the only AA setting benchmarks will be run at"
     
Loading...

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...