NVIDIA SLI

[Dave Baumann]

http://www.beyond3d.com/siteimages/b3dsmall.gif (http://www.beyond3d.com/reviews/nvidia/sli/)Many that have seen the progress of PC 3D graphics over the years will remember Voodoo 2 and its dual board “SLI” capabilities. NVIDIA announced their own take on SLI in 2004 allowing users to couple two GeForce 6 PCI Express boards together in order to double the rendering potential, and now SLI platform hardware and graphics boards are in full availability.

In this article we are going to take a good look at this new form of SLI, first recapping over some of the previous multi-graphics solutions, looking at NVIDIA’s SLI and the modes it operates in as well as some of the issues that have to be tackled with parallel discrete graphics processing. On top of this we’ll also look at the performances from examples of all the currently available SLI capable boards: 6600 GT, 6800, 6800 GT and 6800 Ultra. Click here to read more (http://www.beyond3d.com/reviews/nvidia/sli/).

[Pete]

Good reading, Dave, from what I gleaned from a skim. Three questions:

1) I thought the reference 6800 PCIe speed was 325/300 (as opposed to 325/350)?

2) So many OCed card. Was this some sort of nForce+nVidia auto-OCing?

3) I can't believe how much slower the 6800 was than the 6600GT, considering their AGP performances. Is this an issue with the PCIe-native NV41?

[Mariner]

A heroic article that, Dave - must have been a herculean task. I'm surprised that the men in white suits haven't had to cart you away after all the card swapping and benchmarking!

Surely the most detailed SLI article on the Net. :)

[mczak]

3) I can't believe how much slower the 6800 was than the 6600GT, considering their AGP performances. Is this an issue with the PCIe-native NV41?
Seconded. Are you sure there is nothing wrong with those boards? There have been some packaging errors lately (e.g. 12-pipe 6800GTs and such). The scores are just abysimal, the 6600GT completely trashes it in every single benchmark, and it does not even have _half_ the performance of the 6800GT. That's not what I remember from other reviews, and considering the specs it doesn't make sense at all - the performance is about what you'd expect from a 8-pipe 6800 at that clock speed.

[digitalwanderer]

Great article, but it's really slow going for me...there is simply a massive amount of info to digest. I'm still trying to absorb how PCIe treats it's lanes/channels thingies.

(And I'm damned grateful Dave explained it in that article, I was wondering about it since they announced the PPU thingy and I didn't know how mobos would suuport all the PCIe lane/channel thingies needs)

I gotta ask, how much time did you put in on that article?!?! It's freaking amazing. :shock:

[digitalwanderer]

3) I can't believe how much slower the 6800 was than the 6600GT, considering their AGP performances. Is this an issue with the PCIe-native NV41?
Seconded. Are you sure there is nothing wrong with those boards? There have been some packaging errors lately (e.g. 12-pipe 6800GTs and such). The scores are just abysimal, the 6600GT completely trashes it in every single benchmark, and it does not even have _half_ the performance of the 6800GT. That's not what I remember from other reviews, and considering the specs it doesn't make sense at all - the performance is about what you'd expect from a 8-pipe 6800 at that clock speed.
I just got to page 7, was this italicized bit there when you two read it?
* Note that for the GeForce 6 6600 GT and 6800 we are using partner boards which don't quite meet the reference specifications. In the case of the 6600 GT it is actually clocked up slightly with a core speed of 515MHz, as opposed to the reference specification of 500MHz, while the 6800 has core and memory clocks of 334/309MHz as opposed to the default of 325/350MHz. Although the reference specification for GeForce 6800 Ultra the reference boards used in this article automatically clocked themselves to 437/565MHz on the SLI platform.

[mczak]

Seconded. Are you sure there is nothing wrong with those boards? There have been some packaging errors lately (e.g. 12-pipe 6800GTs and such). The scores are just abysimal, the 6600GT completely trashes it in every single benchmark, and it does not even have _half_ the performance of the 6800GT. That's not what I remember from other reviews, and considering the specs it doesn't make sense at all - the performance is about what you'd expect from a 8-pipe 6800 at that clock speed.
I just got to page 7, was this italicized bit there when you two read it?
* Note that for the GeForce 6 6600 GT and 6800 we are using partner boards which don't quite meet the reference specifications. In the case of the 6600 GT it is actually clocked up slightly with a core speed of 515MHz, as opposed to the reference specification of 500MHz, while the 6800 has core and memory clocks of 334/309MHz as opposed to the default of 325/350MHz. Although the reference specification for GeForce 6800 Ultra the reference boards used in this article automatically clocked themselves to 437/565MHz on the SLI platform.
Yes, seen that. This does not explain the extremely low score of the 6800. Despite the slower than reference ram, it still has more ram bandwidth than the 6600GT (and more than half of the ram bandwidth of the 6800GT), and the core clock is even higher (not that it would matter much with that tiny difference). So there's still no reason why it's not even half as fast as the 6800GT.

[Dave Baumann]

1) I thought the reference 6800 PCIe speed was 325/300 (as opposed to 325/350)?

I don't recall actually seeing reference specs for this board actually - I'd thought it was the same as the AGP version.

2) So many OCed card. Was this some sort of nForce+nVidia auto-OCing?

It might have been the ASUS board, however I'd turned off the option that I thought did this.

3) I can't believe how much slower the 6800 was than the 6600GT, considering their AGP performances. Is this an issue with the PCIe-native NV41?

Game PC's initial review (http://www.gamepc.com/labs/view_content.asp?id=msi6800pcie&page=1) indicates that it is often slower than the 6600 GT. It seems like this has fallen though the optimisation gaps with the driver development, perhaps; we've also seen other cases where 3 quad boards don't work optimally with a 256-bit bus (X800 PRO).

Seconded. Are you sure there is nothing wrong with those boards? There have been some packaging errors lately (e.g. 12-pipe 6800GTs and such).

These are two retail, off the shelf, out of the packaging boards - unforunate if there are two "wrong" boards; this is just what is being sold in this instance. I was somewhat surprised with the results and checked numerous times and trying both boards independantly.

I'll have a check to see of the GT's were OC'ing as well.

[digitalwanderer]

Game PC's initial review (http://www.gamepc.com/labs/view_content.asp?id=msi6800pcie&page=1) indicates that it is often slower than the 6600 GT. It seems like this has fallen though the optimisation gaps with the driver development, perhaps; we've also seen other cases where 3 quad boards don't work optimally with a 256-bit bus (X800 PRO).
Could you elaborate on these "optimisation gaps" a bit please Dave?

I can almost get it, but not quite. :oops:

[Geo]

I think he just means that some boards present special, unique challenges from an optimization pov even with the same family (like GF6 family), and that three-quad boards are one instance. "Optimization gap" happens when those unique challenges don't get addressed as fully as they might due to driver team resource issues, and can thus result in peculiar-looking performance results when compared to a board that doesn't have the special issue or has received more optimization-lovin' even tho the specs-on-paper aren't as impressive (like 6800 vs 6600gt as discussed).

[WaltC]

I have to say this is without a doubt the definitive SLI review published to date on the Internet--that I've seen, anyway. Great job--answered almost every single question I had! I particularly appreciated the following sections:

This test is clearly more dependant on the PCI Express upstream bandwidth and highlights that there is half the performance in SLI mode, with 8 PCI Express lanes distributed between the boards, as opposed to all 16 lanes going to a single graphics card. The reason that the performance is halved in this case is because although two boards are rendering the image, and they both have 8 PCI Express lanes each, the application uses the final rendered image and the only place this is ever present as a whole is on the primary board – when the application comes to read the final image it is doing it off this primary board which only has half the available interface bandwidth to it.

This is the first nV SLI review I've read that clarified that issue so precisely. Thanks.

And this:

This article has been a long time in the making with numerous false starts due to software and platform issues. We've been through numerous different SLI connectors, platform BIOS's, platform and graphics drivers, and even different graphics boards in the case of the 6800 Ultra - initially there were often stability issues and frequent crashes. However, as the platform has matured a little and various updates have come about things have begun to become much more cohesive and hopefully products that are in the retail channels now will already be making use of these updates so that end users shouldn't experience quite as many issues - that said, the heat issues we suffered with the 6600 GT's in SLI mode is something that may need to be watched out for and plenty of airflow through the case may be useful. With the second set of GeForce 6800 Ultra's the power related stability issues we saw with the first set were removed and we could happily run both of them and an Athlon 64 FX-55 on a 480W power supply; this is a fairly bare bones system though without any sound and only a single optical and hard drive, so more fully featured systems may still require more power and NVIDIA are recommending a 580W power supply for a 6800 Ultra configuration.

Thanks for relating what your experience actually was as it occurred as opposed to taking the other route of not mentioning any problems at all--or else of glossing over them--which is sadly so often the case in hardware reviews. Thanks again--very informative and a great read.

I guess I do have a couple of niggling questions, come to think of it...

Your description of the SLI mboard you used described plugging in a 4-pin molex psu connector to the motherboard to power the second PCIe slot, and the included pics showed the cards also plugged into their own respective 4-pin molex psu connectors. I'm not really clear on why two duplicate cards powered in this fashion, with an additional power connection for the second PCIe slot, would not at the least draw 2x the power of a single card configured in a single slot for single operation (without the additional molex connected to the mboard.) Not disputing your results in the least, but what's the explanation for falling so far short of a doubling in power consumption between single and dual-card operation? (I'm wondering if the mboard PCIe molex was perhaps left connected when you removed one of the cards and tested power consumption in single-card mode, or something similar...? Off the cuff it seems like two identical cards running at stock speeds ought to draw 2x as much power...)

I know in my current one-card system that generally I stay away from 1600x1200 because my monitor (CRT) refreshes at only 85Hz at that res which throws a noticeable bottleneck into my vsync-on frame-rates, and I find vsync-off tearing fairly objectionable--so I stay at 1280x1024 for a 100Hz refresh (or sometimes drop to a high FSAA setting at 1024x768 at 120Hz) in order to run with vsync on. As the largest gains for SLI occur at 1600x1200 I'm wondering what your subjective opinions about that were since you ran with vsync off I would guess. (I looked at your hardware specs as listed in "Test Setup" and didn't see mention of the monitor you used--forgive me if I overlooked it somewhere--just curious as to your impressions in this regard.)

Thanks again for a great read, Dave...;)

[mczak]

3) I can't believe how much slower the 6800 was than the 6600GT, considering their AGP performances. Is this an issue with the PCIe-native NV41?

Game PC's initial review (http://www.gamepc.com/labs/view_content.asp?id=msi6800pcie&page=1) indicates that it is often slower than the 6600 GT. It seems like this has fallen though the optimisation gaps with the driver development, perhaps; we've also seen other cases where 3 quad boards don't work optimally with a 256-bit bus (X800 PRO).
It runs neck and neck in all benchmarks with the 6600GT though, some faster, some slower, but always very close. But in your review it is consistently 20-30% behind. I have a hard time to believe this is due to better optimizations on the 8-piped part, with a performance hit like that why bother you could just switch that 3rd quad off... And gamepc's 6800 card is only 325/300 (OTOH they don't have an overclocked 6600GT, so that should nullify this difference).
There is another (SLI) review here, somewhat newer than that gamepc review, http://www.firingsquad.com/hardware/msi_nx6800/page2.asp which appears to use the exact same card, and the 6800 usually wins easily against the 6600GT, and sometimes again looses by a hair (I suspect it fares relatively better against the 6600GT compared to the gamepc review because the gamepc didn't use any AA/AF).

Seconded. Are you sure there is nothing wrong with those boards? There have been some packaging errors lately (e.g. 12-pipe 6800GTs and such).

These are two retail, off the shelf, out of the packaging boards - unforunate if there are two "wrong" boards; this is just what is being sold in this instance. I was somewhat surprised with the results and checked numerous times and trying both boards independantly.
Well, I believe the packaging errors affected whole batches of cards, so if you bought them together there is a quite high chance they would both come from a defective batch.

Do you have any synthetic benchmark numbers for the 6800? Fillrates or so would be interesting...

[Richard]

Excellent, excellent article and I have to agree with WaltC that it's the best SLI article I've read. I also liked how you explained a lot of the problems and nuances on your way to publishing it (like the driver difference in D3 and HL2, etc.) I liked that you explored the SLI connector more than anyone else I've seen so far.

The article is long as should be expected but I think that's a requirement for a comprehensive understanding of SLI, its strengths and drawbacks. Two thumbs up. Thanks Dave!

[Dave Baumann]

It runs neck and neck in all benchmarks with the 6600GT though, some faster, some slower, but always very close. But in your review it is consistently 20-30% behind. I have a hard time to believe this is due to better optimizations on the 8-piped part, with a performance hit like that why bother you could just switch that 3rd quad off...
Its not a performance "hit", the 6800 has 3/5 the clockrate. The architectures are not exactly the same either, and a 128-bit bus can be more efficient than a 256-bit bus. You may also note that the other reviews are not using the 7x.xx series of drivers.

Anyway, look here (http://www.beyond3d.com/reviews/nvidia/sli/images/6800_fill.jpg). Rivatuner reports as a full NV41 chip (3 quads, 5 VS) enabled, it just appears the efficiency is poor for some reason.

Your description of the SLI mboard you used described plugging in a 4-pin molex psu connector to the motherboard to power the second PCIe slot, and the included pics showed the cards also plugged into their own respective 4-pin molex psu connectors. I'm not really clear on why two duplicate cards powered in this fashion, with an additional power connection for the second PCIe slot, would not at the least draw 2x the power of a single card configured in a single slot for single operation (without the additional molex connected to the mboard.)

PCI Express for graphics calls for 75W to be supplied to the PEG x16 connector; normally this is achieved by the extra 4 pins on ATX power connector (the 20 pin power connector, with the extra 4 bringing it to 24 pins) going to the primary PEG connection. In an SLI setup a secondardy board is still going to need that 70W from the second PEG connector, which is what the "EZ_Plug" (hard drive power) connector going into the motherboard is for. The extra connectors on the graphics boards are just "supplimental power", i.e. they will be manily drawing over the PCIe slot, but the direct connections to them up. The power draw graphics are for the entire system, not the graphics alone, so just plugging in a second board won't double the overall system draw. The extra power required for a second Ultra was in the order of around 80W.

I also liked how you explained a lot of the problems and nuances on your way to publishing it (like the driver difference in D3 and HL2, etc.)

This article actually started last year, just prior to the SLI nForce4 launch, however the issues I encountered prevented me from getting it up then, and its been simmering over for a long time. In total I have fully tested 3 drivers sets on the 6800 GT and Ultra configurations alone, and more. At least by missing the initial reviews its allowed me to spend a fair bit more time on it and it also means we used the newer drivers so we can see how things have progressed.

No rest for me though, I'll be up at 6AM tomorrow ready for 3 days at CeBit!

[mczak]

It runs neck and neck in all benchmarks with the 6600GT though, some faster, some slower, but always very close. But in your review it is consistently 20-30% behind. I have a hard time to believe this is due to better optimizations on the 8-piped part, with a performance hit like that why bother you could just switch that 3rd quad off...
Its not a performance "hit", the 6800 has 3/5 the clockrate. The architectures are not exactly the same either, and a 128-bit bus can be more efficient than a 256-bit bus. You may also note that the other reviews are not using the 7x.xx series of drivers.
True. However, with those other reviews and older drivers, there was pretty much zero performance difference between the 6600GT and the 6800, and now there is suddenly a 25% performance difference? Wonder drivers and all, but I don't buy it. I don't doubt a 128-bit 500Mhz memory bus could sometimes be slightly better than a 300Mhz 256-bit memory bus, but this still wouldn't explain why the 6800 doesn't even reach half the performance of the 6800GT.

Anyway, look here (http://www.beyond3d.com/reviews/nvidia/sli/images/6800_fill.jpg). Rivatuner reports as a full NV41 chip (3 quads, 5 VS) enabled, it just appears the efficiency is poor for some reason.
indeed. Especially the single texture and z-fill numbers are poor for a 6800 (they are, however, still higher than what a 6600GT achieves).
So ok, the cards are probably ok.
However, at this point I still think something odd is going on with the 6800. Well I have not really an idea what, could also be a driver bug or whatever. Sorry Dave, but I'm not going to believe your numbers for once :shock: Any good driver comparisons out there which include 6800GT and plain 6800 :?:

Oh, and btw, very good article (those 6800 just caught my eyes, that's all...)!

[trinibwoy]

Just got paid for an hour to read through it :) Very well organized and presented article. Makes a good case for SLI in the high-end. I didn't even realize UT2004 could be so cpu limited even on a FX55.

[Anonymous]

One question though:

On both of the PCI-E bandwidth tests (mainly the serious magic): I would like to know if increases the clock of the PCI-E bus would help aleviate some of the bandwidth issues. Could you try upping your PCI-E bus to 2.8ghz or so and see if there are any performance improvements?

[Reverend]

I just skipped to the conclusion -- I no longer (after the initial excitement provided by the Voodoo2-SLI) care about multi-boards.

[digitalwanderer]

I no longer (after the initial excitement provided by the Voodoo2-SLI) care about multi-boards.
Can I ask why? :| I'm not disrespecting your opinion, just curious as to the reasons behind it.

[Pete]

Dave, I was actually thinking about the GamePC review, which (AFAIK was the first and) noted the PCIe version had slower memory. I see that the 6800 is at times slower than the 6600GT, but it's worth noting that it's only so at 10x7. It catches up at 16x12--aided, I assume, by its extra bandwidth. Note also how the FS review that mczak linked shows the SLIed MSI 6800s far outpacing the SLIed 6600GTs in Far Cry, Doom 3, and HL2 at 12x10 and 16x12 w/AA, I'm assuming b/c of the MSI's extra RAM. Also note that Rollo's 6800 256MB SLI numbers in HL2 here (http://forums.anandtech.com/messageview.aspx?catid=31&threadid=1534067&enterth read=y) smoke yours (both his AA+AF numbers compared to yours, and even his 19x14 numbers compared to your 16x12 ones).

I don't mean to distract from an epic review with such a trifling and tangential detail. I was just struck by how much slower the 6800 was than I expected it to be, especially as the res increased. Re: RivaTuner, tho: is the bit masking detection as accurate as a fillrate test, or does RT offer pipe info based on a lookup table referring the GPU name (NV41 is rather new, and I don't think 6800LEs based on it have been shipped yet)? I do recall MSI being afflicted with a good number of 8-pipe 6800s, and that would do a lot to explain the performance you saw. Or maybe yours just had really lax memory timings? Or were recently forced to disrupt their steroid regiment? :P

Anyway, no need to get to the bottom of this now. Sit back and collect some well-deserved thanks, then decide if it's worth following up on this weirdness.

Yeah, dig, the italicized bit about the clocks was there (hence my Q to Dave if the nF MB was auto-OCing, as all the boards save the GT ran higher than spec). Still doesn't explain why the 6800 fell further behind as the res increased. If anything--especially given the AGP versions--I'd have thought the 6800's greater bandwidth would prove more valuable at higher resolutions and with more AA. It's just odd, and at odds with the other reviews I've read (that Dave and mczak linked).

[Geo]

Jaysus, where are the 3200x2400 100hz monitors when you need them? :lol:

[digitalwanderer]

<sigh>

I can replace damn near any internal component without my wife noticing, but she'd spot a new monitor in a heartbeat. :(

[ZoinKs!]

Great job on the article. I doubt there'll ever be an article on SLI (and the other multichip graphic techniques) more thorough then this... it filled in a lot of gaps in my knowledge. I liked having background info on the various techniques and how they work. These days, a fellow just can't have much 3d geek cred without knowing a thing or two about them.

And those 6800 GT and Ultra results make me drool. :P

[WaltC]

PCI Express for graphics calls for 75W to be supplied to the PEG x16 connector...

Very good--thanks for the explanation!

[Anonymous]

yea the 6800 results confused me too... dave maybe u should update the article witha note or something? cause thoose numbers conflict with what I've seen from other sites... Maybe you should try and get your hands on another set of 6800s or maybe a diffrent mobo?

[Anonymous]

Great article - I only wish any of the major review sites would address the SLI Widescreen DVI issue so that NVIDIA will take notice and respond.

I'm having a problem with using SLI at a resolution of 1680x1050
(Widescreen LCD). When using that resolution that rendered frame (or part of a frame) between the two cards no longer line up when they are displayed to the screen. The offset between the two is only a few pixels, it can be just a little annoyance or make the game unplayable.

So when SLI is running in Alternate Frame Rendering (AFR) mode it
causes a horrible flicker on the screen and the game is unplayable. If I switch down to a lower resolustion like 1280x1024 the frames sync up and the flicker goes away.

In Split Frame Rendering (SFR) mode the drive dynamically changes the amount of the frame each GPU renders. With the resolution back up at 1680x1050 the offset between the two parts of the frame can be seen as a horizontal line that crawls up and down the screen. Not a show stopper but annoying none the less. Lower the resolution and the problem goes away.

I have also tried both DVI ports on the primary video card. If I use the DVI/VGA adapter and connect to the monitor with analog - I get rolling black bands in the background video.

I'm not sure if the resolution or the aspect ratio that is causing the
problem. I've tested with the current release drivers and the beta
drivers from nvidia's site.

The only way I have found to work around the problem is to run my games in a window (HL2 and CS:S) or set the resolution to 1280x1024.

Note that I have also tried using my Sony Widescreen monitor (analog) @1280x768 and it has the same issues.

[Thowllly]

Great article, but it would have been even better if you had tested even higher resolutions. I know you've said your monitor doesn't support higher resolutions, but I've never seen a 19" crt that isn't capable of displaying 2048x1536. No matter what the official max res is, as long as it supports a horizontal frequency of at least 96kHz it will be able to display 2048x1536 at 60Hz. If that resolution is not available you can try to add it through the driver control panel.

[overclocked_enthusiasm]

Dave:

I am having trouble reconciling this HEXUS blurb with your article. Any comments?

http://www.hexus.net/content/reviews/review.php?dXJsX3Jldmlld19JRD0xMDU4

"If you wander over to Prolink's stand at CeBit, you'll not only see NVIDIA's nForce4 for Pentium 4 on their stand, using EPoX's 5NVA+ SLI mainboard, but you'll also see the board running SLI using two 6800 GTs with different BIOS revisions."