Why isnt the R580 a lot faster in Shader intensive games?
- Thread starter boltneck
- Start date
My point was that for any site that basically says "Games are what it's all about" while also stressing on video chip architectures (by using synthetic benchmarks.... I assume synthetic benchmarks are used by such sites to study a chips architectures) that the latter (results of synthetic benchmarks that, hopefully, explains architectures) should explain the performance of the former (games).Demirug said:
True but that's the conundrum and problem that "hw review sites" (as opposed to "3D architectures reporting sites", if such exist) face.
This is relevant when an article like this is supported, and defended, by that sites EIC while we see the review that EIC writes.
I repeat -- the results of synthetic benchmarks in a review have practically little to do with results of games used in the same review. Quite impossible you may say, given the old chicken-and-egg and timespan between new hw and sw development but until and unless we are "taken back in time" by hw reviewers, using synthetic benchmarks and games in a single review just means a wastage of space and time for everyone involved.
This is relevant and is basically what I have come to want to see at this site (during and after I left). I don't think it will be all that bad traffic-wise (=financially-wise) if we have separate and individual "reviews" that either (but not together in the same "review") focus on comparisons of architectures by competing companies or focus on current (and "current" is all we can ever get) games performances.
For example, Dave can and will only write articles that compare architectures of products from competing architectures by using whatever synthetic benchmarks he wants to use. The other B3D staff can and will only write articles that compare products from competing companies (or even different products from the same companies) using only games. I think this is possible and can work quite well and given some organization be even complementary between the two types of "reviews". B3D certainly has staff for both categories.
I started a thread that is basically about this. Neeyik/Nick's response in that thread basically made me think a bit more (which is to say, what he posted isn't proven in a single article/review at B3D where synthetic benchmarks and games benchmarks are used, presumably complementary... last two words being most important..... they aren't explained why they are used in the same review... if the two aren't relevant, they should be separated... unless we're talkiing about meeting specific needs and needing to cover B3D costs).
It is also quite telling that synthetic benchmarks come before game benchmarks in all B3D reviews and that, depending on your interest, we usually skip a number of pages in a B3D review when we read them (come on now, admit it... you are either one or the other... you won't be able to connect the synthetic benches with the game benches because the review author didn't, couldn't, do so... and so...). Again, it is a waste of time for the writer and reader. Time probably better served with a change in organization (but maybe not focus).
Hopefully this will be taken as a constructive suggestion and nothing more. Maybe we need to know what Dave has to say about the purpose or reason behind the use of synthetic benchmarks and games benchmarks in the same review. He didn't address my previous post.
PS. I really wanted to write more but since I think I have an awful spy/malware problem and that the vein I'm on is basically OT in this forum, I will stop. Jawed's post was quite illuminating to me, given that I respect his 3D hw knowledge and that he is, basically, a B3D site enthusiast.
Last edited:
They (and not just limited to "shader" benchmarks... I mean all synthetic benchmarks can also be useless in the "right hands" but that depends on (1) your interest; (2) if you think your interest basically means you want better looking games in the end; and {3) if you think the results of synthetic benchmarks at a site are meaningful given (1) and (2).Ailuros said:
"In the right hands" can be awfully difficult to answer. The need to balance a site's ultimate/preferred focus; and money from traffic to keep a site alive without spending too much personal money is also difficult to answer. And we all know which type of audience (that read hw reviews) come from categorically speaking.
BTW, in case you don't understand what I was trying to say basically above, the type of reviews we read here are bewildering to not only the layman that wants the best hw for playing games but also to those that are interested in 3D technology.
I can't help but wonder what John Reynolds thinks of this...
I can't help but wonder what John Reynolds thinks of this...
This might be improved in an upcoming Catalyst. Or maybe it's already here and won't help X1800 to X1900 levels. HW.fr tested with Cat 6.2 beta and X1800 showed some improvement relative to X1900 when compared to Xbit's testing (Cat 5.12 for X1800s, 6.1 for X1900). See links below for HW.fr's and Xbit's X1900 AoE3 benches.radeonic2 said:Why is the 1800XT so slow in AOE 3 btw?
The 1900XT is twice the speed up even at 2048x1536 with 4x fsaa and 8xAF
What does this switch do, exactly? Does it switch from PS2 to PS3 shaders, or from simple PS3 shaders to more complex ones?Nv500 said:
These benchmark results? I only see the GTX-512 twice as fast in a single PS test: 3DM03's. (In fact, in that one shader the GTX-512 is twice as fast as both R520 and R580, which certainly doesn't correspond to R580's AoE 3 performance.) Otherwise, R520 is reasonably close to a GTX-256 in 3DM PS tests at 10x7 (extrapolating from GTX-512's generally 50% lead over R520XT), and so neither that single outstanding result nor these other synthetic tests seem to explain how the regular GTX is from 50% (Cat 6.2 beta) to 100% (Cat 6.1) faster than the X1800XT (in both minimum and average fps, in that last test!).
I can believe it's a PS-limited title, just not that it mirrors 3DM synthetics. Maybe the promised patch will improve things for R520 ("There must be something wrong with the game engine," Xbit speculates), or maybe it's just pixel shader ALU-limited in that game. This purported R520 pixel shader deficiency doesn't seem to be reflected in other games, though, even those with ostensibly high math-to-texture ratios, like FEAR or Splinter Cell.
(BTW, pretty funny that all three games are TWIMTBP. Not saying that helps or hurts ATI, as B&W2 is GITG and ATI gets whupped in that, too--tho a patch is apparently forthcoming for that, as well.)
BTW, 3DM synthetics test mainly PS2, and you mention AoE 3 as a PS3 game. PS3 synthetics seem to be R5x0's domain, per 3DM06's Perlin test, ShaderMark 2.1's SM3 branching tests, and iXBT's PS3 tests.
Last edited by a moderator:
Reverend, you say that games and synthetics don’t link well in current reviews and you are right. But how should we link them together if we don’t understand what the games are doing? Games developers normally don’t open their little “magic books†for you. And even more worse they sometime didn’t even tell you the truth. I remember a list of games that allegedly use SM3 and don’t do it at all. This will force anybody who want to link synthetics and games together to reverse engineer the games that are used in the review. This is a hard and time consuming job. But does it matter? I am still seeing many people “reading†only the bars and numbers in a review. How many people really want to know the dirty secrets that are hidden inside a game? A reviewer who have to pay the bills form this work need to write what the readers want or they will go away.
Reverend said:
I did get your point and I also look at what I've managed to do so far within those few attempts I dipped my toe into the water. I don't and won't consider myself as a reviewer to make that clear (no professional strings attached, no time pressures involved etc), but from my side I can tell you that I keep my fingers away from stuff I don't understand.
However if a game should show awkward behaviour that I am not able to interpret, I'll either ask around if anybody can help me decyphering what's going on (and cross-verify if possible) or I'll just skip it entirely in the end.
The easiest way "out" would be some "blah blah unoptimized drivers yadda yadda" bullshit line and call it a day. If I get a tip that game X relies heavily on functionality Y which I couldn't detect at first sight and there is a synthetic application (simple enough for me to use and understand) which concentrates on "Y", then I most certainly will go on and speculate over my findings from both game and synthetic application.
If I'd have the chance to use 10 different scenarios/applications for just one game I will. And most certainly not in the consensus of filling up more space, but in the attempt to cover as many bases as possible.
I'd be surprised if that would be the case for the second category. It's my understanding that B3D in general concentrates or seems to be concentrating more on the latter audience and rightfully so.
I tend to fall towards the point of view Revered is making about the synthetics but maybe that is because with the X1900 this was the first time I felt a need to read the synthetic results to have a look at the x3 shading architecture in operation. Normally the synthetics give two values that I just cannot relate to.
For instance
synthetic test Card X = 3.2768. Card Y 2.876
however
game test Card X = 60fps Card Y 10fps
I can relate to. If a synthetic shows card X has 1000 and card Y has 2000 but games only ever use 500 then does this synthetic show that card Y is better than X ? Yes, if you use your machine to run synthetics ! Not relevant if you do not.
Demirug is saying that this gives you a better insight, and he is right. Dave Baumann will also say that is the reason for B3D ( he has before ) and I agree totally as well. Personally though going to this level of degree does not really interest me. I do like the SM2 and SM3 individual tests on 3dmark06 though as this gives me a "feeling" towards the cards stengths and weaknesses, however you will probably say the granularity is not enough there.
For instance
synthetic test Card X = 3.2768. Card Y 2.876
however
game test Card X = 60fps Card Y 10fps
I can relate to. If a synthetic shows card X has 1000 and card Y has 2000 but games only ever use 500 then does this synthetic show that card Y is better than X ? Yes, if you use your machine to run synthetics ! Not relevant if you do not.
Demirug is saying that this gives you a better insight, and he is right. Dave Baumann will also say that is the reason for B3D ( he has before ) and I agree totally as well. Personally though going to this level of degree does not really interest me. I do like the SM2 and SM3 individual tests on 3dmark06 though as this gives me a "feeling" towards the cards stengths and weaknesses, however you will probably say the granularity is not enough there.
Hanners said:
I just read a review by the german Computerbase, thanks to the guys that linked me to it at 3DCenter and according to them Radeons use a INT10-HDR format + 4xMSAA, while GeForces use a FP16-HDR + 1.5x SSAA mode according to it.
http://www.computerbase.de/artikel/...900_cf-edition/15/#abschnitt_age_of_empires_3
Jawed
Legend
Two points:
1. I think the fill rate synthetics do have some value :smile: - e.g. they demonstrate that X1600XT's texturing architecture is less constrained by bandwidth than X1800XT/X1900XT. It's just the current set of synthetic shaders that I object to, as they're essentially unlike any game workload which consists of more than just texture fetching and ALU instructions. Even single-concept shader demos, like the shadow map app that Dave's been playing with are suspicious, because there's no complex geometry, there's no MRTs, no AA - all you're seeing is "best-case" scenarios. A 30% speed-up due to dynamic branching is lovely, but if dynamic branching for the same shader only produces a 5% speed-up in a real game then
2. I think that GPGPU-based synthetics may prove helpful. Rys's efforts, and Tridam's at www.hardware.fr (though I'm waiting for the English version - my French just aint up to it) are definitely steps in the right direction.
Rys seems to have based his work (or been inspired by) on the Stanford GPU-bench suite. Tridam has been at this somewhat longer and really seems to have the bit between his teeth.
The GPU bench guys, by the way, seem to express some reservations about their own tests - finding that they don't correspond particularly well with the actual performance of GPGPU apps.
---
I'm afraid this is prolly because the synthetic shaders are actually too simplistic. So the benchtester ends up going round in circles trying to write shader code that's meaningful (i.e. sequences of MADDs quickly lose any meaning) whilst not writing a game.
The IHVs have research labs for a reason - a single reviewer just doesn't have the overview, sadly. Having said that, enthusiasts can only learn by banging their heads against this stuff - you can't speculate (even if naively) about ALU:TEX ratios or texture-latency-hiding versus dynamic branching if the architecture's a complete mystery.
In the end, synthetic shaders should be about elucidating an architecture - not about saying "gee whizz, this new card is 80% faster when I do this!"
I think Tridam's ability to demonstrate that the quads in G70 are independent, and not all lockstep like they are in NV40, through the use of a synthetic shader to investigate the architecture, is proof that there can be value in synthetic shaders - even if very simple. Brains, not brawn.
Jawed
1. I think the fill rate synthetics do have some value :smile: - e.g. they demonstrate that X1600XT's texturing architecture is less constrained by bandwidth than X1800XT/X1900XT. It's just the current set of synthetic shaders that I object to, as they're essentially unlike any game workload which consists of more than just texture fetching and ALU instructions. Even single-concept shader demos, like the shadow map app that Dave's been playing with are suspicious, because there's no complex geometry, there's no MRTs, no AA - all you're seeing is "best-case" scenarios. A 30% speed-up due to dynamic branching is lovely, but if dynamic branching for the same shader only produces a 5% speed-up in a real game then
2. I think that GPGPU-based synthetics may prove helpful. Rys's efforts, and Tridam's at www.hardware.fr (though I'm waiting for the English version - my French just aint up to it) are definitely steps in the right direction.
Rys seems to have based his work (or been inspired by) on the Stanford GPU-bench suite. Tridam has been at this somewhat longer and really seems to have the bit between his teeth.
The GPU bench guys, by the way, seem to express some reservations about their own tests - finding that they don't correspond particularly well with the actual performance of GPGPU apps.
---
I'm afraid this is prolly because the synthetic shaders are actually too simplistic. So the benchtester ends up going round in circles trying to write shader code that's meaningful (i.e. sequences of MADDs quickly lose any meaning) whilst not writing a game.
The IHVs have research labs for a reason - a single reviewer just doesn't have the overview, sadly. Having said that, enthusiasts can only learn by banging their heads against this stuff - you can't speculate (even if naively) about ALU:TEX ratios or texture-latency-hiding versus dynamic branching if the architecture's a complete mystery.
In the end, synthetic shaders should be about elucidating an architecture - not about saying "gee whizz, this new card is 80% faster when I do this!"
I think Tridam's ability to demonstrate that the quads in G70 are independent, and not all lockstep like they are in NV40, through the use of a synthetic shader to investigate the architecture, is proof that there can be value in synthetic shaders - even if very simple. Brains, not brawn.
Jawed
Last edited by a moderator:
I simply use my own variation of the rate issue shader for my own tests, and simply to present a "the hardware is this capable in terms of meeting its theoreticals under these conditions" picture.Jawed said:
However, that then leads on to GPGPU work, and all that entails, should an editor decide to walk down that road. I likely will, at some point, but don't take my rate issue work (which actually largely sucks atm) as having GPGPU undertones. At least not yet.
As for the GPGPU bench results not corresponding to what goes on in apps written with BrookGPU, or other frameworks like FBO or RenderTexture, as you'll see on gpgpu.org, it's for exactly the same reasons other pure theoritical tests don't give you the full picture.
cho: it depends on the test, but yes, the shorter simpler shaders are more easily predicated by the assembler and their results less useful or entirely worthless.
Ailuros said:
I think they are wrong - they seem to think that the x1000 cards is not capable of MSAA+FP16 and for that reason only runs at int10 when the fact is they are fully capable of MSAA+FP16.
The HDR in the screenshots looks imho identical for both cards, so if the x1900 really is running at int10 it does not seem to any influence on IQ.