What is the cause of the limit of the number of GPUs plugged into the motherboard ?

Discussion in '3D Hardware, Software & Output Devices' started by Flux, Jun 17, 2013.

  1. Flux

    Regular

    Joined:
    Nov 10, 2006
    Messages:
    313
    Likes Received:
    2
    What is the cause of the limit of the number of GPUs plugged into the motherboard?

    Windows has a limit of 4 GPUs plugged in.
    Linux has a limit of 8 GPUs pluggen in.

    What causes that? I read that there is a bios limitation on the motherboard.
    Will AMD allow you to modify the bios to thier motherboards?
     
  2. Davros

    Legend

    Joined:
    Jun 7, 2004
    Messages:
    15,050
    Likes Received:
    2,386
    amd dont make motherboards or motherboard bios's
    there is nothing for them to dis allow
    If anything its part display driver limitation and part chipset limitation you would need a hell of a lot of pci-e lanes
    i dont think that limitation exists if you use the gpu's for bitcoin mining or gpgpu (at least not the 4gpu's under windows)
     
  3. Flux

    Regular

    Joined:
    Nov 10, 2006
    Messages:
    313
    Likes Received:
    2
    No bitcoin operations or GPGPU.

    So its a amd driver limitation right?
     
  4. Davros

    Legend

    Joined:
    Jun 7, 2004
    Messages:
    15,050
    Likes Received:
    2,386
    so what do you want to use 5+ cards for
     
  5. Flux

    Regular

    Joined:
    Nov 10, 2006
    Messages:
    313
    Likes Received:
    2
    I want to see if a cloud server mainboard can hold more than 8 GPUs.
     
  6. Flux

    Regular

    Joined:
    Nov 10, 2006
    Messages:
    313
    Likes Received:
    2
    So you can get more than 8 or is it limited to 8?

    There is some kind of driver problem that doesn't allow for you to have more than 8 gpus on one board.
     
  7. nutball

    Veteran Subscriber

    Joined:
    Jan 10, 2003
    Messages:
    2,172
    Likes Received:
    488
    Location:
    en.gb.uk
    Try a different angle - what's the cost to the IHV of hardware/driver validation of configurations with a semi-arbitrary number of GPUs, and how many customers would there be in the market from whom they could recoup those costs?
     
  8. Flux

    Regular

    Joined:
    Nov 10, 2006
    Messages:
    313
    Likes Received:
    2
    You can have 16 gpus on one board?

    Is there anyway to add more than 8?
     
  9. silent_guy

    Veteran Subscriber

    Joined:
    Mar 7, 2006
    Messages:
    3,754
    Likes Received:
    1,379
    These guys had 13 GPU working: http://fastra2.ua.ac.be/

    But apparently they had their share of issues with BIOS etc.
     
  10. Flux

    Regular

    Joined:
    Nov 10, 2006
    Messages:
    313
    Likes Received:
    2
    Can you actually run real 3d applications on that?
     
  11. silent_guy

    Veteran Subscriber

    Joined:
    Mar 7, 2006
    Messages:
    3,754
    Likes Received:
    1,379
    I have no idea. But one assume that you can definitely run CUDA application on it, otherwise they wouldn't be building such a thing! :wink:
     
  12. Blazkowicz

    Legend Veteran

    Joined:
    Dec 24, 2004
    Messages:
    5,607
    Likes Received:
    256
    It may not be worth doing that anyway, if you fill a 42U rack with such GPU servers.
    Let's say you fit 12 servers there (3U without spacing, and 7 U left for networking, power etc.) with 8 GPUs, and 200W per GPU, that's 19.2 kilowatts right there without counting CPU, RAM, other and inefficiency.

    That may be a cooling nightmare. Even supplying power is not trivial (that's 83 amps if you were to use 230V). On the plus side GPU power could be interesting for residential heating and such.
    I think that stuff out to be done, buildings have underground space, power and fibre running to them ; apartments owner and tenants would benefit by having part of their heating bill paid by the "computing utility" that manages the servers.

    It's even about the only way I'd want to do cloud gaming. It's way better if the server is on the LAN :razz:, you would only need a RJ45 outlet on the wall and a client device. Have heating coming through the floor. Nice space savings.


    Of course the 16 GPU options is by using Tesla K10, which is nothing more than an 8GB GTX 690. I read it's not popular at all. A very narrow range of users would buy this (no usable DP, no full ECC, and no full feature set) and it was presented as a stop gap before GK110.
     
Loading...

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...