4x AA performance comparisons invalid?

Discussion in 'Architecture and Products' started by Fuz, Feb 3, 2003.

  1. Fuz

    Fuz
    Regular

    Joined:
    Apr 17, 2002
    Messages:
    373
    Likes Received:
    1
    Location:
    Sydney, Australia
    I recall discussions on these forums regarding review methods, and the correct way of comparing two cards in terms of image quality and performance.

    Most agreed that the image quality should be as similar as possible when comparing performance between two cards. For instance, if 32xAF on card A produces the same image quality as B's 2xAF, then A's 32x should be compared to B's 2xAF.

    Well, the general consensus seems to be that ATI's AA seems to produce better quality than the FX's.

    Take a look at the following images for instance.

    NFS R300 2xAA
    NFS NV30 4xAA

    UT2003 R300 2xAA
    UT2003 NV30 4xAA

    Now the images look pretty similar don't they? So should we be comparing ATI's 4x to Nv's 4? Or should it be ATI's 2x to Nv's 4x?

    Ok, now ofcourse many of you will say that Nv's 4x is better than ATI's 2x in those shots I linked to. Well, would you say Nv's 4x is as superior to ATI's 2x as ATI's 4x is superior to Nv's 4x?

    Imho, comparing performance of NV30's 4xAA to R300 4xAA is not really a fair comparison. So whats your opinion?
     
    thomascoello likes this.
  2. Doomtrooper

    Veteran

    Joined:
    Feb 6, 2002
    Messages:
    3,328
    Likes Received:
    0
    Location:
    Ontario, Canada
    Yes Anand tried to do that with Anisotropic filtering, by getting a baseline IQ comparison..but not done on FSAA.
    I agree, we need to establish a baseline IQ..why should one card 'work harder' at a higher setting when the IQ is much higher with the sliders matched. Or the same 'why should a card not have to do as much work' with sliders matched if the IQ is lower'

    Baseline IQ again meaning approximations, not zoomed in 100% to compare settings..common sense.
     
  3. Brent

    Regular

    Joined:
    Apr 11, 2002
    Messages:
    584
    Likes Received:
    4
    Location:
    Irving, TX
    sometimes, but not always, it is subjective until you zoom in to see what it is doing...
     
  4. Arun

    Arun Unknown.
    Moderator Legend Veteran

    Joined:
    Aug 28, 2002
    Messages:
    5,023
    Likes Received:
    302
    Location:
    UK
    I'd suggest comparing to a lower quality ATI Anisotropic mode ( such as 4x Performance ) to compensate for the slight AA IQ difference.
    4x AA from nVidia sure looks worse than ATI's 4x AA, but it still looks slightly better than ATI's 2x IMO.


    Uttar
     
  5. Fuz

    Fuz
    Regular

    Joined:
    Apr 17, 2002
    Messages:
    373
    Likes Received:
    1
    Location:
    Sydney, Australia
    Well, that is why I asked;
     
  6. Doomtrooper

    Veteran

    Joined:
    Feb 6, 2002
    Messages:
    3,328
    Likes Received:
    0
    Location:
    Ontario, Canada
    Its starts getting subjective for the reviewer..thats why including the screenshots of all modes allows the viewers to make their own decision.
    They will get disected anyways on the forums, people pointing out what is important to them.
    Yet common sense needs to be used when comparing and using a baseline IQ, the reviewer matches it as close as he can between the two cards (shows his screenshots and settings to prove it) and start benchmarking.

    Baseline is now done
    Now benchmark all the modes
    This type of review covers two things:

    1)Comparison with IQ is centered on
    2)All other modes are covered also.

    4X on one slider doesn't meet 4X on another...as shown in screenshots.
     
  7. demalion

    Veteran

    Joined:
    Feb 7, 2002
    Messages:
    2,024
    Likes Received:
    1
    Location:
    CT
    Too much subjective evaluation if you go to far. I think Anand did it the right way (barring the GF FX 2x AA shots being incorrect still AFAIK) for the first half, where the image quality is presented and discussed and compared. The only thing lacking there, I think, is the "Application mode" for the GF FX.

    Then, as Anand did, the reviewer's opinion of what image quality they consider comparable is outlined as a result of these shots, along with reasoning (I think he must read these forums, :lol:), and then comparison going forward is based on that, with all the data presented, not just what the reviewer considers applicable. Again, I think the "Application mode" is missing.

    What Anand's review lacks in the second part, IMO, is some sort of aid in placing the "apples" to "apples" comparisons close together per the user's own choice (like some sort of custom graphing application). Strikes me as something that could be done if someone wanted to spend the time doing so. The simplest way I can think of doing it is to have the user pick the images they think equivalent, and color those bars in the graphs with highlighted and brightened colors throughout the rest of the review (with a palettized image format, this would be easy, I think, if a scripting language had the capability to change palette color values).

    The best solution would be to order the fps graphs by image quality as determined by the user.
     
  8. Nite_Hawk

    Veteran

    Joined:
    Feb 11, 2002
    Messages:
    1,202
    Likes Received:
    35
    Location:
    Minneapolis, MN
    Really, I think it's up to the reviewer how and what they want to benchmark, but the only reviewers I have any respect for are the ones that clearly state why they are performing the benchmark in a specific manner, and detail exactly how they've achieved the results they've gotten. If the reviewer thinks comparing the GF FX's 2X aa to the Radeon 9700's 6x yields interesting data and explains why, I don't mind at all.

    You know, not to rant, but reviews still have a lot of room for improvement. (though places like beyond3d and now hardocp seem to be doing pretty well.) We need standard deviations for framerate scores, image analysis comparisons (not just subjective, but quantitative), and of course subjective opinions as well. This is the kind of information I'd actually be willing to *pay* for. Some of the crappier reviews out there aren't even worth my time to read let alone money to pay the writer.

    Nite_Hawk
     
  9. Bambers

    Regular

    Joined:
    Mar 25, 2002
    Messages:
    781
    Likes Received:
    0
    Location:
    Bristol, UK
    I'd say nv30s 2x is pretty similar to the r300s 2x. Since they're the same pattern (except nv30 2x is \ way and atis 2x is / way) the only difference is gamma correction.

    r300 4x to nv2x/30 4x is not a fair comparision imo.
     
  10. THe_KELRaTH

    Regular

    Joined:
    Dec 9, 2002
    Messages:
    471
    Likes Received:
    0
    Location:
    Surrey Heath UK
    Reading ATI's blurb on their AA method it says that it uses programmable sample patterns. On that basis would that not mean that in every angle at say x2aa it will always be a true x2 whereas on a fixed grid there will always be angles that will get less than x2aa and with x4aa fixed grid there will be angles where it's only actually performing x2aa :?:
     
  11. Dave H

    Regular

    Joined:
    Jan 21, 2003
    Messages:
    564
    Likes Received:
    0
    R300 does not currently use programmable sample patterns. The capability is designed into the hardware, but is not enabled in current drivers.
     
  12. KimB

    Legend

    Joined:
    May 28, 2002
    Messages:
    12,928
    Likes Received:
    230
    Location:
    Seattle, WA
    The only problem with attempting to set a baseline FSAA image quality level for performance comparisons between the Radeon 9700 and GeForce FX is that the only time the two are very close to equal is when both are running with 2x FSAA.
     
  13. Windfire

    Regular

    Joined:
    Feb 16, 2002
    Messages:
    353
    Likes Received:
    1
    Location:
    Seattle, WA
    Are we seeing the same pictures here? The screen shots supplied above clearly show that ATI's 2x is easily equal to Nvidia's 4x, let alone Nvidia's 2x.
     
  14. Windfire

    Regular

    Joined:
    Feb 16, 2002
    Messages:
    353
    Likes Received:
    1
    Location:
    Seattle, WA
    Someone should setup several blind tests with Nvidia's 2x and 4x and ATI's 2x and 4x. Let users pick the order from "best to worst or tie" or something.

    Looking at those pictures I had to double-check the pictures to confirm that I actually thought ATI's 2x looks better than the 4x! I would almost question the legitimacy of those pictures!
     
  15. OpenGL guy

    Veteran

    Joined:
    Feb 6, 2002
    Messages:
    2,357
    Likes Received:
    28
    I doubt it would work. For one, people could use magnifying glasses to see which had more samples. Thus, results wouldn't necessarily be based on visual quality, but on how many samples were used.
     
  16. Nagorak

    Regular

    Joined:
    Jun 20, 2002
    Messages:
    854
    Likes Received:
    0
    Any difference between the 2X and 4X shots is so slight as to be nonexistant. If you had only 10 seconds to look at both shots, you wouldn't be able to tell them apart.

    My question about comparing quality is: to what end? Will you compare 2X AA on R9700 to 4X on the FX in terms of performance, or what?
     
  17. MrNiceGuy

    Newcomer

    Joined:
    Jun 21, 2002
    Messages:
    13
    Likes Received:
    0
    ok...

    Since nv 2x & ati 2x are pretty much the same, then by that argument, one should compare nv's 2x to nv's 4x??

    At low #s of samples, sample placement is very important to edge quality, which is why 2x ~= ordered 4x in general.

    For roughly equal quality, you would have to compare nv's 4xS to ati's r300 4x.
     
  18. noko

    Regular

    Joined:
    Feb 10, 2002
    Messages:
    502
    Likes Received:
    0
    Location:
    Eustis Florida
    My opinion is to run the benchmarks, show the IQ differences and make an assessment. What I've have seen so far is ATI's 4x AA for the most part is superior to any other mode on the GF FX. ATI's 2x is superior to Nvidia's 2x and looks close to Nvidia's 4x. What shocks me more is that my GF3 Ti200 looks better at 2x and 4x then the GF FX.
     
  19. THe_KELRaTH

    Regular

    Joined:
    Dec 9, 2002
    Messages:
    471
    Likes Received:
    0
    Location:
    Surrey Heath UK
    Any idea as to why at 80 to 100 and 135 to 155 degrees Nvidias AA is almost non existant even at 8xS level?
    [​IMG]
     
  20. Doomtrooper

    Veteran

    Joined:
    Feb 6, 2002
    Messages:
    3,328
    Likes Received:
    0
    Location:
    Ontario, Canada
    I thought the way Brent did it was idealy the best way, allow the readers to make their own opinion by showing screenshots...when it comes to performance well its not ideally correct to have one card looking like crap and the other great and comparing performance numbers..

    With screenshots hopefully the readers would be wise enough to realize that its not a apples to apples comparison.
     
Loading...

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...