9700 Pro < Ti4600 at MINIMUM fps - is this REALLY true?

Discussion in 'General 3D Technology' started by Joe Cool, Nov 9, 2002.

  1. Nupraptor

    Newcomer

    Joined:
    Nov 11, 2002
    Messages:
    79
    Likes Received:
    0
    Well... yeah. :) But I was referring, in specific, to this part:
    I downloaded their benchmarking tool and gave it a shot, and it seems to be much better (in my opinion).

    The reason why I brought it up was because, after running their benchmark utility, I noticed that my minimum framerate on one map actually clocked in at 1.xFPS, yet I never actually saw it dip down that low when playing the same map. So I wouldn't worry about minimum FPS too much.

    Cheers,
    Nup
     
  2. Althornin

    Althornin Senior Lurker
    Veteran

    Joined:
    Feb 6, 2002
    Messages:
    1,326
    Likes Received:
    5
    As i said, only problems with botmatches. All the flyby data is there - no problems getting it either.
     
  3. Nupraptor

    Newcomer

    Joined:
    Nov 11, 2002
    Messages:
    79
    Likes Received:
    0
    Well, as he said... checking the log every time could be quite tedious. At any rate, [H]'s benchmarking utility is small, so there's no real reason not to get it, since it levels the playing field a little more.
     
  4. Joe Cool

    Newcomer

    Joined:
    Nov 8, 2002
    Messages:
    11
    Likes Received:
    0
    " The reason why I brought it up was because, after running their benchmark utility, I noticed that my minimum framerate on one map actually clocked in at 1.xFPS, yet I never actually saw it dip down that low when playing the same map. So I wouldn't worry about minimum FPS too much. "

    There's good reason to worry about it when places like Tom's Hardware Guide mentions minimum frame rates, though. Especially if they give inaccurate results while doing so.

    Hardcore gamers - the kind more likely to get a card like the 4600 or 9700 Pro - really care about and check for this sort of stuff.

    And then they discuss it with their friends. Who tell their friends - and so on.

    The supposedly comparitively poor performance of the 9700 Pro in the cases I mentioned to start this thread has - I guarantee it - been enough to dissuade many people from getting the 9700 Pro, either sticking with or going to an Nvidia card instead.

    The simple phrase "Minimum frame rates are lower" will mean to most of them - like it or not - that "This card can't cut it when the going gets *tough.*" And that's the impression Tom's Hardware Guide gave about the 9700 Pro in several of their UT 2003 benchmarks. That's bad.

    I don't really care that this is costing ATI money, but as an owner of a 9700 Pro I do want it to get enough of an installed base for developers to take it seriously - enough so to optimize for it.

    Or at least test on it, which is something that (oddly enough) I seem to recall the developers of UT 2003 refused to do. Even though they had an 9700 Pro sitting in a box before UT 2003 was released.

    Now if it doesn't deserve any of this, that's fair enough. If the 9700 Pro really is slower as mentioned then too bad for me.

    But if this is all a result of a flaw in testing, we should expose this - with details and proof - and make sure the word gets out.

    Besides which, who knows what other card reviews - even those say of the NV30 when it's released - might be made to look worse than need be - if we don't?
     
  5. kmolazz

    Newcomer

    Joined:
    Nov 13, 2002
    Messages:
    39
    Likes Received:
    1
    Location:
    Portugal
    hi all,

    i´ve been reading this thread for a while now, and you've really done a great job getting to the bottom of things... that's why i like reading this forum so much :D (you don´t just "bash" at each other, but actually have serious discussions from wich one can learn a lot. thanks)



    but, there is still something i didn´t get clearly. If you compare the "in game" benchmark results with the one's obtained say with fraps, are they different for both cards? the 9700pro and the gf4 4600?

    you already showed that the 9700pro gets different results, but if it also happens with the gf4 4600 in the same way there is no point in saying that the benchmark util may "favour" one card over the other.

    the fact that the results from the game benchmark don´t correlate with "real gaming" is quite disturbing, particularly because that isn´t explained in the reviews ( that many people read and base their opinion on ) of any hardware sites. You read the graphs and think: that's the kind of fps wi'll be getting during gameplay; herrr wrong....


    so, in summary, as anyone compared the results from both cards with fraps and the "in game" benchmark util?
     
  6. Reverend

    Banned

    Joined:
    Jan 31, 2002
    Messages:
    3,266
    Likes Received:
    24
    Oh, I don't think you've been here long enough then... :)
     
  7. Randell

    Randell Senior Daddy
    Veteran

    Joined:
    Feb 14, 2002
    Messages:
    1,869
    Likes Received:
    3
    Location:
    London
    kmolazz,

    I would tend to trust the ingame benchmark CSV's rather than FRAPS. Having scanned through the CSV's on my benchmark run, I can see how people wouldnt see the ingame benchmark counter drop to those numbers. I only had 4 frames around the 30-33 minimum fps the benchmark reported, and they weren't consecutive, so the drop lasted what, 30ms? then back up to 120fps for a while then a one frame drop to 30fps then back up again for a total of 4 frames at that level across 2311 in total.

    We'll see what happens with DX9 RC0 runtime nad Dx9 drivers.
     
  8. kmolazz

    Newcomer

    Joined:
    Nov 13, 2002
    Messages:
    39
    Likes Received:
    1
    Location:
    Portugal
    you believe performance/efficiency should get better, being the 9700pro a DX9 card?

    or perhaps some issues will be resolved..
     
  9. Randell

    Randell Senior Daddy
    Veteran

    Joined:
    Feb 14, 2002
    Messages:
    1,869
    Likes Received:
    3
    Location:
    London
    both is my hope, in much the same way the 8500 needed DX8.1 for all of its functionality.
     
  10. demalion

    Veteran

    Joined:
    Feb 7, 2002
    Messages:
    2,024
    Likes Received:
    1
    Location:
    CT
    The benchmark does not favor one card over another, it reports the data objectively, regardless of whether the Ti 4600 has any single frame delays that cause minimum fps to plummet (and it seems it doesn't).

    Well, the thing is, these things about the data in question do reflect gameplay:

    The average fps (well, as much as average fps ever does).

    A graph of the fps for each frame.

    A histogram based on the occurence of fps values.

    These things may not:

    Minimum and Maximum fps from data when it is presented as representative of performance to the user.

    And the reason it may not be is because it is uninterpreted and out of context when presented by itself(as some reviews have done recently), whereas, for example, if you saw it on a graph, the context would be provided...if it occurred once you would likely not notice it, or atleast give proper consideration for what was going on elsewhere if you did, and if it occured often, you'd be able to see exactly how often.

    I think there was a post that did. But, again, I'd say the uT2k3 data is more accurate, but it is just that the way it is being used is misrepresentative.
     
  11. Nupraptor

    Newcomer

    Joined:
    Nov 11, 2002
    Messages:
    79
    Likes Received:
    0
    Joe Cool: I agree completely, except for one thing:
    Personally, I think very lowly of Tom's Hardware, and I never even look at their site anymore. Still, there are enough people who do to make it an issue.

    Cheers,
    Nup
     
  12. Magnum PI

    Veteran

    Joined:
    Feb 6, 2002
    Messages:
    1,054
    Likes Received:
    12
    i really dislike toms hardware a lot more since the sysmark 2002 affair..

    http://www.vanshardware.com/reviews/2002/08/020822_AthlonXP2600/020822_AthlonXP2600.htm

    http://www.theinquirer.net/?article=5274

    http://www.theinquirer.net/?article=5580

    and there is the only thing tom found to say against van hardware and about the bapco affair:
    http://www17.tomshardware.com/blurb/02q3/020825/blurb-03.html
    http://www17.tomshardware.com/blurb/02q3/020825/blurb-01.html#april_2002_amds_plans_to_attack_bapco

    i let you appreciate...

    my opinion is sysmark 2002 is a scam, designed to give an artificial advantage to intel processors...

    at the very least this benchmark is suspect and shouldn't be used anymore. there is a lot of others benchmarks. in the worst case it would cause no harm at all to not use this one anymore ?

    why does tom insist to use sysmark 2002, even in its latest CPU reviews ?
     
  13. Basic

    Regular

    Joined:
    Feb 8, 2002
    Messages:
    846
    Likes Received:
    13
    Location:
    Linköping, Sweden
    Giving my point on fps measuring.

    I think the cvs files from UT2003 is a realy nice thing. Even though tab-separated seems to work better with Excel, it's still easy to convert it to a plotable form.

    Is it possible to find a description of the other columns?

    Doing a point plot of "instantaneous fps" is quite easy, and you get a good feeling of what's going on. It's easy to do a "local histogram" in your mind by looking at the point density. And it's easy to see the distribution of the realy slow parts.

    One thing to concider though is what should be on the X-axis. Using frame number (=simulated game time in UT2003) will give you a good plot if you want to do a direct comparision between two cards, since both plots will do the exact same work on a given X-coordinate. But that kind of plot will likely give the wrong feeling about the distribution of slow and fast frames.
    Concider what happens if a demo runs at 10 fps for 1min and then 90fps for 1min. The graph would look like this:
    Code:
    |    ******************
    |
    |
    |
    |
    |
    |
    |
    |
    |**
    ----------------------------------------
    
    So you could get the feeling that it's running slow for just a small part of the demo.

    Ploting fps vs real time (accumulated frame times) would look better wrt this, but would otoh not be so good when doing direct comparision between cards.


    All this is of course under the assumption that the data is correct. But we know that it's difficult to do a correct measurement of frame time in software, since there's a lot of buffering between the "frame end" we can measure and the exact time it's finished and displayed. And this time can vary.

    So what can we do? Well we can of course make sure that we measure the real thing; the time when the new frame is sent to the monitor. An IHV might have some way to get into the GPU and do that measurement exactly, but an ISV could not do that. So we need a thingy on the monitor cable that can see a new frame coming. :eek:

    Luckily that doesn't have to be as hard as it seems. :D At least with a little help from the software we're benching.

    Here's my idea:
    A VGA cable contains R, G, B, hSync, and vSync.
    We need a hardware that

    • Count vSync pulses to get a screen count.

      Count hSync pulses since last vSync to get a more detailed switch time.

      Checks the first positive edge on the color channels on each line, and remember what channel it was.

      Whenever the "first positive edge" changes to a new channel, send hSync count, vSync count, and first positive edge channel over a serial port.
    The benchmark software should draw a vertical line at the left edge of the screen. This line should have a new color for each frame (pure red, pure green, pure blue, pure red, ...). Drawing this line is of course a little extra work, but should be insignificant to the work done in any modern game.

    The output of the cable dongle is captured to a file. If you don't want to disturb the benched comp, then capture it with a separate computer. Now you have very exact frame times. You just need to know the screen refresh rate of the monitor to convert the data to time.
    If the dongle sends a timestamp in addition to the data I mentioned above, then it's possible to deduce the screen refresh rate out of the captured data. You could of course just send the timestamp, but I like the connection to exact screen number and position.

    "First positive edge" data is captured in the log file to have some way to check for lost frames and other error conditions.

    How difficult would this be to build?
    Not that hard. First we need some rather fast circuit to detect first edge. That could be done with a few simple 74AC ICs.

    Something like this:
    R,G,B = Color components from the cable (digitalized to two levels).
    HSync = hSync from cable.
    nQr, nQg, nQb = captured edge
    HSPol = Control signal to adjust for hSync polarity.

    Using four 74AC ICs 00+00+10+86

    nT &lt;= !(!(nQr &amp; nQg &amp; nQb) &amp; 1 &amp; 1);
    nQr &lt;= !(HSp &amp; !(nQr &amp; !(R &amp; nT &amp; 1));
    nQg &lt;= !(HSp &amp; !(nQg &amp; !(G &amp; nT));
    nQb &lt;= !(HSp &amp; !(nQb &amp; !(B &amp; nT));
    HSp &lt;= HSync ^ HSPol;

    (Yeah, yeah, I know, some kind of pseudo C-VHDL code. :p)

    The outputs are sent to a microcontroller, along with hSync and vSync. The microcontroller can count sync pulses, check nQ*, and send data over a serial port. An AVR ATtiny processor should be enough.

    Now the only thing left is some level convertions. Maybe it needs some fast comparators, but with some good luck it could be enough to shift the ground level of the dongle logic so that the endpoints of the analog video signals can be concidered '0' and '1'. With such a shift there might not bee any need for a RS232 level converter.

    Connect it to an old monitor cable you don't care about, and it could end at a cost below $10. (At least if you have a good and cheap electronics shop nearby.)

    The output on the monitor will of course not be as good as with a good untampered cable, but this cable is only ment to be used when doing these benchmarks.

    If the cable makes it too fuzzy though, the colored line could be too smeared out. So it might be nessesary to contoll how wide it is. Even with a very bad cable, it should still be possible to find the color if the line is a few pixels wide.

    And as an extra feature, you could connect a pin from a port that can be directly controlled by the CPU. That way you could send a signal from the main CPU (Intel/AMD) when you think the frame is finished. The dongle can add this event to the datastream, and it should be possible to deduce how long delays there are in the drivers/HW.


    And yes, I know this is a hack. :D

    Anyone care to try?
    Russ, I've got the impression that you're a build-it-yourself kind of guy.
    Vogel, would you draw the line if someone wants to try?
     
  14. Nupraptor

    Newcomer

    Joined:
    Nov 11, 2002
    Messages:
    79
    Likes Received:
    0
    ... I am completely out of my league here, so I'll just take my leave of this thread now. Carry on. :)
     
  15. Randell

    Randell Senior Daddy
    Veteran

    Joined:
    Feb 14, 2002
    Messages:
    1,869
    Likes Received:
    3
    Location:
    London
    on a more basic level, the latest DX8 Catalyst 2.4's improve my low frames by quite a margin. +10fps on the lowest frame (from 31 to 42) and there is only the one frame that now drops below 60-65.

    I will take SA's advice and post the 'average below the average' from the 2.3's to the official 2.4's.
     
  16. vogel

    Newcomer

    Joined:
    Aug 23, 2002
    Messages:
    88
    Likes Received:
    0
    Location:
    Raleigh, NC
    Yes, though I won't do it until someone has it up and running with a test app.

    -- Daniel, Epic Games Inc.
     
  17. Basic

    Regular

    Joined:
    Feb 8, 2002
    Messages:
    846
    Likes Received:
    13
    Location:
    Linköping, Sweden
    OK good enough, didn't expect any more.
    And considering the storming entusiasm noticed here :)-?), I guess that someone would be me. :)

    I noticed that no store around here carried the uC i considered suitable (ATtiny28). So I'll rethink it and try to squeese it into a ATtiny15 (I love those 8pin microcontrollers with everything included).

    But first, would it be worth it for a developer/reviewer/hometweaker to get some extra hardware just to measure a more exact instantaneous framerate?

    Either way vogel, don't plan in any time for the line-drawing yet. :) You won't likely need to do it for yet some time (if at all if I know myself right).

    Btw, current idea that could be possible to squeese into a tiny15 is:
    It sends a two-byte code for each "event" happening.
    An "event" can be a vSync, a new frame or a AUX signal change from main CPU (typicaly used to say that a Present has ended).
    The two-byte code will contain a time stamp, and an event identifier (vSync, AUXlo, AUXhi, or new frame including starting color).
    It should be possible to get ~10us resolution on the time stamp. (But probably not that precision in all cases.)
     
  18. MrGaribaldi

    Regular

    Joined:
    Nov 23, 2002
    Messages:
    611
    Likes Received:
    0
    Location:
    In transit
    Hello :)

    I'm just wondering how the benchmark dongle is coming along...
    Whereas the technique is way beyond me I'm very interested in what the results show...

    :)
     
  19. megadrive0088

    Regular

    Joined:
    Jul 23, 2002
    Messages:
    700
    Likes Received:
    0
    Radeon 9700 Pro - 8 pipes - 2.6Gpixels - 325M verts/sec
    GeForce4 Ti 4600 - 4 pipes - 1.2Gpixels - 136M verts/sec

    it's got to be a driver bug

    'NUFF said.
     
  20. Basic

    Regular

    Joined:
    Feb 8, 2002
    Messages:
    846
    Likes Received:
    13
    Location:
    Linköping, Sweden
    As usual, stuff got in the way. :( :oops: And it got pushed down one more step on the priority list when I realized that the way I was planning to do it was no good, and I needed a microcontroller that I don't know how to get hold of.

    Anyone know were to find an ATtiny26?
     
Loading...

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...