Synchroniszation issues with SLI and CrossFire

(if you're not Chris Ray, PLEASE read and don't disregard this!)

Synchronization is a BIG issue. The more I dig into it, the worse it becomes. I don't know why people keep telling "Well it's an issue but it doesn't hurt much! And I'm willing to sacrifice that much for such high frame rates!!!"

It's not an "issue" like, for example, "some artifacts on the screen while playing", that you can choose to keep in favor of playing on smoother frame rates.

It IS directly your frame rate.

We made a lot of different tests with different games in SLI:

Assassin's Creed, BioShock, Call of Juarez, Lost Planet and Crysis.

AC and Bioshock aren't hampered with bad synch issues at all. (Although there's such an issue in Kingdom in AC, it's there even with a single 8800GT, and it makes the game unplayable below 40FPS in Kingdom, which sucks. BTW, a single 3870 doesn't have such an issue at all. Yay NVidia!)

The rest, however... Make nearly all SLI gains worthless.

Now, before I publish some of the results, I want to elaborate a bit on the "practical" frame rate concept. It's not the FPS shown in FRAPS. Let me give an example from Call of Juarez:

Code:
# - Frame time - Difference

7	171.979	48
8	187.469	15
9	229.284	42
10	244.826	16
11	287.247	42
12	303.01	16
13	346.28	43
14	361.656	15

Here exactly 8 frames have been rendered in 0.19 seconds, corresponding to 42FPS. But as far as human perception goes, the general sense of fluidity, naturally, depends on the most delayed frame in the "vicinity". You don't see it as 42FPS. Every one of two frames has 45ms delay. So, yes... You see this entire scene at a practical frame rate of 1000 / 45 = 22 FPS. With additional stuttering involved due to every second frame added at a much lower delay.

Yes. Only 22 FPS. While we were benchmarking there was just no way we would believe what we were seeing was being displayed as 40-50 FPS. It wasn't fluid at all.

Lost Planet sucked as well, this time it was more like

Code:
48	753	10
49	769	16
50	784	15
51	790	6
52	814	24
53	832	19
54	838	6
55	859	21
56	877	18
57	881	4
58	906	25
59	921	15
60	928	7

This time the frame seemed to top once for every three frames. So the corresponding momentary frame rates for the above scenario were like 174-50-45-195-40-50-170-45-40 and so on. Of course no one needs to be told that the perceived frame rate is somewhere near 40 in that scenario. However, for 13 frames rendered in 0.15 seconds, FRAPS shows us a FPS of near 75 (due to every one of three frames unnecessarily being rendered in a super fast delay).

Crysis performed like a regular game that sucks in SLI (which makes it a regular game overall)- frame rate dips (and tops) every second frame, just like CoJ. Here's an example from Crysis:

Code:
6	94	26
7	107	13
8	131	25
9	144	13
10	168	24
11	184	15
12	207	23
13	220	14
14	244	24
15	258	14

Here FRAPS tells us we're seeing ~60FPS, but we don't believe its lies because we know we perceive an entire stuttering scene rendered at near 40 FPS.

So how can we "measure" this "practical" frame rate concept? By equating the "momentary FPS" (1000/frame time difference) of each frame to the minimum momentary FPS in its vicinity (vicinity being three frames). This might look like a brutal way of calculating FPS, but actually it's accurate for measuring the sense of fluidity in the scene. If two frames are rendered at 50FPS and the frame next to them at 250FPS, that 250FPS frame adds absolutely nothing to the fluidity of the game. If momentary FPS's are 50-100-50-100-50-100, you see the entire scene at near 50FPS. Sorry, but it's the truth.

With each frame being given a so-called "real" FPS, I couldn't help but average all the "real" FPS's we got. Here's what I got:

Crysis:

Avg. Benchmark FPS: 38
Avg. "Real" FPS: 27

Call of Juarez:

Avg. Benchmark FPS: 37
Avg. "Real" FPS: 25

Lost Planet:

Avg. Benchmark FPS: 63
Avg. "Real" FPS: 47

Notice a pattern? Yeah. A ratio of near 1.4; which corresponds to a little less what we usually get from the second card. (the tests were performed at 1440 and 1680 res., and the average FPS a second card gives you is somewhere between 1.4x and 1.5x) So I might conclude that for those games, a second card brings you only a little more than nothing. Which is exactly what I experienced at Crysis: I bought a second G92, which only enabled me a nice "sunshafts" IQ setting.

For the other games that the synch. thing was not an issue:

Assassin's Creed:

Avg. Benchmark FPS: 53
Avg. "Real" FPS: 53

BioShock:

Avg. Benchmark FPS: 106
Avg. "Real" FPS: 102

See, when you've nicely distributed frame times, this method of calculating isn't brutal at all.

Edit: I can happily give the frame time spreadsheets to anyone who wants. They'll reach the same result.
 
I think a more elegant solution than locking the framerate is the rumored approach of R700, i.e. a shared memory pool. Without the need to distribute data between the GPUs as with current NUMA solutions, the every-other-frame delay should disappear.

Or maybe I'm just too optimistic...
 
If possible, care to run that with vsync on and triple buffering? We should then see multiples of vsync interval for interframe intervals.
 
If possible, care to run that with vsync on and triple buffering? We should then see multiples of vsync interval for interframe intervals.
Unfortunately, Tripple buffering is not working in SLI AFR .
I believe, if properly implemented, it should fix stuttering. But there is no hope to see it (TB+Vsync),as NVIDIA didnt make it in 4 years...
 
Unfortunately, Tripple buffering is not working in SLI AFR .
I believe, if properly implemented, it should fix stuttering. But there is no hope to see it (TB+Vsync),as NVIDIA didnt make it in 4 years...

Ugh? Wtf? :oops: After 3 seconds of thinking i cant find any reason not to support it. Guess im an idiot.

What about games that control vsync themselves? IIRC trackmania has an option for frame sync latency, or sth like that, going up to 3+ frames.

And crossfire: Does it support forced triple buffering and vsync?
 
Chris why do you say that this doesn't bother you? Are you not able to tell the difference between 60fps AFR (with sync issues) and 60fps 'normal'? Is it mostly subjective? I don't have a SLI/Xfire system to test myself hence the question.
 
Chris works for Nvidia ;)

This isn't a knock against him, he's full of useful information, particularly about SLI and AA.
 
So the bottom line is SLI/Xfire are garbage for now until a better solution than AFR is discovered?
 
Last edited by a moderator:
Who really games with v-sync off though, if you're going for eye-candy at least? I only turn off v-sync for benching, never for gaming anymore.
Vsync is not really necessary when your monitor can refresh at 100hz or better. Not to mention many games don't support triple buffering and even if you force it, it makes mouse control floaty, not a big deal in some games but no good in a shooter like CS:S.
 
bottom line is that on games with AFR sync issues the reported FPS is not actually what you are experiencing, the effective FPS in these cases is actually a lot lower.

EDIT: I think anyways, as said in my previous post I don't have my own sli/xfire setup to actually test this myself. I.e I'm not sure if the difference between x FPS in AFR and x FPS normal is actually as great some people are making it out to be.
 
Chris why do you say that this doesn't bother you? Are you not able to tell the difference between 60fps AFR (with sync issues) and 60fps 'normal'? Is it mostly subjective? I don't have a SLI/Xfire system to test myself hence the question.

Depends the game. Depends the bottleneck. And depends the SLI profile. Some games have more problems than others. I can see very large spikes in interframe delay. 20-30-40-100-110-120-130. They do occasionally happen.

What I dont see are small delay spikes. 12/26/35/46/59/. The frames dont have to be identical to provide a smooth experience. A single GPU can't even do that normally. AFR can provide legitimate framerate gains. It's not perfect and is prone to flaws. You can adjust the way inter frame delay is predicted by making changes to the profiling system. I've all but pointed out where people should be looking to make these adjustments. Half the time some of the more aggressive SLI rendering algorithims are the only way to produce performance.

The interesting thing here that was brought up was 3dmark06. It uses a 0x0240005 SLI rendering profile. Unreal Tournament 3 uses the exact same one and for the most part has relatively consistent inter-frame delay. (No its not perfect. But if I run it on a single GPU its not perfect either. But a little less inconsistent)). While 3dmark06 which is strongly CPU limited in several game tests ((even with 8xAA due the way it handles vertex data)) shows really inconsistent results.

What ticks me off most here is.

1) I have acknowledged large spikes happen. I have addressed the issue not only at slizone but rage3d as well. I certainly have done nothing to hide it.

2) I'm being blasted because I think the minor variation's in inter-frame delay are largely un noticable. I never hear complaints about this except from vocal minorities. And I dont even hear about it at slizone.


3) I got people like annihilator attacking me in every other post because I don't agree with them as a blanket statement that SLI is worthless. I believe there is alot of value in it if you use it correctly.

4) I have never said that a multi GPU = Single GPU delivering the same FPS. Actually I completely agree that comparing them like that is bad idea. But to me this was always "oh really ya think?" type issue. It's a complete surprise to me that anyone would think that a multi GPUs delivered frame rate is equal or equivalent to a single GPU.


And Shaidar --You dont know what you're talking about it.
 
So the bottom line is SLI/Xfire are garbage for now until a better solution than AFR is discovered?

There's two extreme views in my mind:

1)It doesn't bother me it all.

2) It's garbage.

Usually things are in the middle but it's a very subjective middle. My views are very similar to Chrisray's actually but it may bother me a bit more. I think multi-GPU's rock because it's a way of garnering much needed performance attributes where no other way is viable for a giv'n window of time at time.

It's easy to say, a powerful single GPU is all that, but what if a potential gamer desires even more performance and desires to add higher levels of image quality by adding another? Think the choice is great!

What if a gamer spends 150 dollars to upgrade their system and offers much more performance over-all instead of spending another 300 dollars or more for a single GPU? Think the choice is great!

All ya hear is negative this and negative that.........but there's a lot of positives to consider, too.

Multi-GPU's are so welcomed it's incredible to me; doesn't anyone remember how is was to be forced to only use one?

Sure, improvements will also be welcomed moving forward.
 
Depends the game. Depends the bottleneck. And depends the SLI profile. Some games have more problems than others. I can see very large spikes in interframe delay. 20-30-40-100-110-120-130. They do occasionally happen.

What I dont see are small delay spikes. 12/26/35/46/59/. The frames dont have to be identical to provide a smooth experience. A single GPU can't even do that normally. AFR can provide legitimate framerate gains. It's not perfect and is prone to flaws. You can adjust the way inter frame delay is predicted by making changes to the profiling system. I've all but pointed out where people should be looking to make these adjustments. Half the time some of the more aggressive SLI rendering algorithims are the only way to produce performance.

The interesting thing here that was brought up was 3dmark06. It uses a 0x0240005 SLI rendering profile. Unreal Tournament 3 uses the exact same one and for the most part has relatively consistent inter-frame delay. (No its not perfect. But if I run it on a single GPU its not perfect either. But a little less inconsistent)). While 3dmark06 which is strongly CPU limited in several game tests ((even with 8xAA due the way it handles vertex data)) shows really inconsistent results.

What ticks me off most here is.

1) I have acknowledged large spikes happen. I have addressed the issue not only at slizone but rage3d as well. I certainly have done nothing to hide it.

2) I'm being blasted because I think the minor variation's in inter-frame delay are largely un noticable. I never hear complaints about this except from vocal minorities. And I dont even hear about it at slizone.


3) I got people like annihilator attacking me in every other post because I don't agree with them as a blanket statement that SLI is worthless. I believe there is alot of value in it if you use it correctly.

4) I have never said that a multi GPU = Single GPU delivering the same FPS. Actually I completely agree that comparing them like that is bad idea. But to me this was always "oh really ya think?" type issue. It's a complete surprise to me that anyone would think that a multi GPUs delivered frame rate is equal or equivalent to a single GPU.


And Shaidar. --You dont know what you're talking about it.

OK well consider a game rendering in the following manner:

Frame 1: 5ms
Frame 2: 15ms
Frame 3: 45ms
Frame 4: 55ms
Frame 5: 85ms
Frame 6: 95ms

So 10ms delay between (1,2), (3,4), & (5,6) and then a 30ms delay between (2,3) & (4,5). In this case does it look more like 100fps (1000/10) or 33.3fps (1000/300)?
 
It actually looks more like a "Stutter" than anything else. If you see something like.

12/25/36/75/88/99/110/165/180/191/202/214/274/284/294.

You will see the relative FPS value. But you will also percieve stuttering at every large MS spike interval. An important point. The higher your FPS is the less likely your going to see this. The lower your FPS ((specifically below 25 FPS)) these spikes become larger as the base time to render frames also increases, And the stutter becomes more transparent. With SLI staying above 30 FPS is typically ideal.

Chris
 
boy I'm so glad right now that Nvidia is still making monster single-gpu cards, stutter is an abomination as far as I'm concerned.
 
The question now becomes: how much longer can they keep pushing monolithic dies?
I see the writing on the wall like everyone else, but I'm glad it hasn't come to that yet. I've been around long enough to have owned SLI Voodoo 2's but this current AFR crap needs to be fixed soon.
 
Personally benefits to multi-GPU setups (allowing higher res, levels of AA and AF) by far outway the minor niggles like micro-stutters.

I think SLI makes the most sence when you already have the fastest single GPU money can buy - but you still need more perfomance/IQ.
 
Back
Top