building up a render farm

Discussion in 'PC Hardware, Software and Displays' started by rgbaguy, Jan 23, 2010.

  1. Blazkowicz

    Legend Veteran

    Joined:
    Dec 24, 2004
    Messages:
    5,607
    Likes Received:
    256
    well raid 6 is :
    six drives with an evenly spread 4/6 data, and 2/6 parity. it's 5/6 data, 1/6 parity for raid 5.

    raid 10 is :
    a raid 0 of three raid 1

    dedicated parity is only to be found on raid 3 which no one ever uses.

    so raid 5, 6 and 10 leave 5TB, 4TB and 3TB respectively ; raid 10 is the fastest, raid 6 the slowest (slowness is mostly about writes ; presumably not a big problem if write is when you feed it with external data from a gigabit link, or rendered frames)
     
  2. rgbaguy

    Newcomer

    Joined:
    Jan 23, 2010
    Messages:
    46
    Likes Received:
    0
    http://en.wikipedia.org/wiki/RAID_10#RAID_10_.28RAID_1.2B0.29

    Thanks for your reply Blazkowicz, it helped alot :)

    I think RAID 10 will be perfect for us :)
     
  3. Silent_Buddha

    Legend

    Joined:
    Mar 13, 2007
    Messages:
    16,151
    Likes Received:
    5,086
    Aye, Raid 6 and Raid 5 are fairly good compromises if your disk subsystem is a significant portion of your build's cost. Balancing space versus cost versus speed versus data protection.

    However, in your render farm, the disk subsystem is going to be a small fraction of your overall costs.

    Just make sure to keep multiple spare drives handy in case of one or more drive failures. You'll want to replace any failed drive, ASAP.

    Regards,
    SB
     
  4. rgbaguy

    Newcomer

    Joined:
    Jan 23, 2010
    Messages:
    46
    Likes Received:
    0
    Yeah, totally. Will definitely keep spares on hand. I'm close to getting a final spec together, still waiting for quotes from our supplier.

    Will post the specs as soon as I have worked them out fully.

    Thanks for help guys :)
     
  5. rgbaguy

    Newcomer

    Joined:
    Jan 23, 2010
    Messages:
    46
    Likes Received:
    0
    Hey guys, I'm having trouble finding a switch. I just can't seem to find switches with 10 gig uplink included (appears they come as a separate module?)

    3Com Switch 4200G 48-port (3CR17662-91) with 1-port 10-gigabit Module (3C17666)
    http://www.shopmania.co.za/shopping~online-switches~buy-3com-3cr17662-91~p-1103783.html
    http://www.3com.com/products/en_US/detail.jsp?tab=prodspec&sku=3CR17662-91&pathtype=purchase

    D-Link DGS-3450 (confirm 48-port gigabit L2 switch, 4 combo SPF and 2 10-gig slots)
    http://www.shopmania.co.za/shopping~online-switches~buy-d-link-dgs-3450~p-1203526.html
    http://www.dlink.ca/products/?pid=dgs-3450

    D-Link DGS-3100-48
    http://www.wantitall.co.za/PC-Hardw...alone-L2-Gigabit-Stackable-Switch__B000UKJM9U

    The above seem what we need right? I'm using those links for a price guideline. Have contacted our supplier, the only 48-port gigabit switch they have is:

    D-Link DGS-1248T web smart 48 port 10/100/1000 gigabit + 4 combo SFP fiber connection , un-managed layer2 switch - auto-MDI/MDIX , port mirroring , 8k MAC address , 1.632mb buffer per device , rackmount ready

    Which doens't seem to have the 10 gig uplinks?

    I'm gonna go back through this post and find all the switch options you guys have posted, see if I can find them here.
     
  6. rgbaguy

    Newcomer

    Joined:
    Jan 23, 2010
    Messages:
    46
    Likes Received:
    0
  7. MfA

    MfA
    Legend

    Joined:
    Feb 6, 2002
    Messages:
    6,806
    Likes Received:
    473
    The stacking ports don't seem usable as pure ethernet ports from the sound of it (they are 5 Gb/s full duplex not 10Gb).

    DXS-3250 and DGS-3450 can have the 10 Gb uplinks.

    PS. your awfully nice to your supplier by trying to hunt down a 10 Gb uplink capable switch for him ... really if you are willing to use him exclusively he should be the one doing it. Personally I'd just say "find something for me or I'll buy the 3CR17662-91 with 10 Gb module from another supplier".

    PPS. the 10 Gb modules seem to be always sold separately.
     
    #87 MfA, Feb 5, 2010
    Last edited by a moderator: Feb 5, 2010
  8. rgbaguy

    Newcomer

    Joined:
    Jan 23, 2010
    Messages:
    46
    Likes Received:
    0
    Thanks for the reply MfA :) Ok, will look at those. Oh hardly, it's difficult getting information out of them! I have said to them, look, we need so and so. They aren't doing that, its getting annoying. Will tell them if they aren't prepared to find something that fits our requirements then I'll look elsewhere.

    Yeah, that does seem the case. I've asked IF the uplink modules ARE INCLUDED. They never answer that question. Aaaaaaaargh!

    Will keep pushing them, otherwise will try someone else. It's hard for me, I have zero experience with networking :(
     
  9. Karoshi

    Newcomer

    Joined:
    Aug 31, 2005
    Messages:
    181
    Likes Received:
    0
    Location:
    Mars
    GBIC
    SFP
    GBIC description includes the rationale:
    Same applies to SFP.

    They dont know if you want to connect to a switch 2m away (get a cheapo laser) or 40km away (we welcome your monies!).
    So unless it's a cheaper switch for local distribution it's probably not gonna have plugged in modules. There are possibly cheaper bundle offers, maybe, haggle with them?
     
  10. Blazkowicz

    Legend Veteran

    Joined:
    Dec 24, 2004
    Messages:
    5,607
    Likes Received:
    256
    That reminds asking for bootroms in shops :lol:

    reactions are a blank stare, astonishment or outright denial (what? booting with no storage media? that's impossible!). Because you're the 1% of retail shoppers asking for that feature.
    yet as a bootrom is a immensely useful $1 piece of hardware.
    End result of that story, a friend bought a bunch of so-called "bootable network cards" from a store, that supported a boot rom but did not provide it. Regular, non-bootable network cards. :mad:

    10gigabit module isn't a 30-year-old commodity tech. But similarly, small companies never buy them and they are probably bought and installed by the big ones, who have their own procurement channels.
    You need to get higher in the food chain it seems.
     
  11. MfA

    MfA
    Legend

    Joined:
    Feb 6, 2002
    Messages:
    6,806
    Likes Received:
    473
    And for 10 Gb/s to XFP/XENPAK.

    BTW rgbaguy, the manufacturers will generally on their website tell you what's included as standard ... anyway, the 3com seems by and far the cheapest ... for that you need a 3CXENPAK95 module.
     
  12. Blazkowicz

    Legend Veteran

    Joined:
    Dec 24, 2004
    Messages:
    5,607
    Likes Received:
    256
    good, by googling that product name you can easily find them, just not in your usual well known online shop. good to know it's called a "transceiver" (like the modules I've never ever seen, that go on the "joystick plug" at the back of older network cards)
     
  13. rgbaguy

    Newcomer

    Joined:
    Jan 23, 2010
    Messages:
    46
    Likes Received:
    0
    Thanks for this Karoshi, that definitely cleared up some of the tech jargon :)

    Ouch, that must have pissed him off :D You make a good point, well this is pretty much the biggest supplier here, so no other options.

    Yeah, been struggling to find out what comes standard. Thanks for that, will ask the supplier if they can get that :) So 3Com 4200G switch and two 10GBASE-CX4 XENPAK modules should do the trick then :)

    True, Google is awesome. Suppliers not :(

    Thanks guys, hopefully they will be able to get...
     
  14. rgbaguy

    Newcomer

    Joined:
    Jan 23, 2010
    Messages:
    46
    Likes Received:
    0
    Hey guys, all sorted on the hardware side :) Have got costs for everything we need including switches and uplink modules (finally!) One question on the network side of things, any standard CX4 cable will work? Will need a 50cm/1m CX4 for switch-to-server link and a 5 m for the workstation-to-switch link. Those cables are quite pricey! o_O

    Would like to ask what you guys recommend on the OS side of things. For our blades I'm looking at Open SUSE as it's free and from my experience very stable. Fedora not so much from my experience. What do you reckon? Ideally would like to install a distro that uses as few system resources as possible.

    On the workstation side, looking at Windows 7 64-bit, mainly for ease of use (not everyone is comfortable with linux). On the server side of things, not so sure myself?

    I reckon Windows Server 2008 would be good. Have asked some people I know and they recommend it. What do you guys think?

    What about Windows HPC Server 2008? As I recall from previous posts a good server can optimize network transfers with caching, what do I look at for that?

    Have looked to Clonezilla for drive cloning, anyone have experience with this? Looks production stable, what do you reckon? http://www.clonezilla.org/

    Would like to say thanks to everyone who has posted on this thread, your help has been fantastic and would be pretty lost without it :)

    Look forward to hearing back!
     
  15. Laa-Yosh

    Laa-Yosh I can has custom title?
    Legend Subscriber

    Joined:
    Feb 12, 2002
    Messages:
    9,568
    Likes Received:
    1,452
    Location:
    Budapest, Hungary
    Have you found a background storage solution yet? A simple HDD based system on its own usually can't support even a small studio and renderfarm, so you'll probably have to invest in something a bit more advanced. Then the driver and software options of that device are also going to become a factor in choosing your OS.
    All I know about ours is that it's something from StorNext... and we have some sort of tape backup system too.
     
  16. rgbaguy

    Newcomer

    Joined:
    Jan 23, 2010
    Messages:
    46
    Likes Received:
    0
    Hey Laa-Yosh, thanks for the reply! Well have looked at a 6 x 1TB drive storage server in RAID 10 for storage. That gives us 3 TB, which should be fine. Once the drives are full we just backup...

    This will also be the server that runs our queue master and licensing...

    As for tape, I guess I should have a look into that.
     
  17. -tkf-

    Legend

    Joined:
    Sep 4, 2002
    Messages:
    5,632
    Likes Received:
    36
    In regards to Partition Backup and Restore, i have been playing around with:
    http://www.paragon-software.com/home/db-express/index.html
    Boots from CD or USB and saves/restores from a USB / Firewire Disk

    On the server side i am very curious as well, with our plans we end up having a Dual Xeon Server, with 16 1.5 TB Disks serving our workstations over a 10GBit nic. My plan is to cram alot of ram into the machine and run Server 2008. But i have been wondering if there is a High Performace or Slimmed down option that focus on File serving and does a better job than a standard Server 2008
     
  18. MfA

    MfA
    Legend

    Joined:
    Feb 6, 2002
    Messages:
    6,806
    Likes Received:
    473
    Using windows as NAS for Linux clients is just asking for headaches ... why not use a nice linux install for that as well? (Also use NFS rather than SMB.)
     
  19. Blazkowicz

    Legend Veteran

    Joined:
    Dec 24, 2004
    Messages:
    5,607
    Likes Received:
    256
    I believe SMB / CIFS is the easier way to have a high performance file share. NFS might be not so good. unless maybe you tweak it.

    I don't have experience in tuning things but you probably want jumbo frames on the ethernet layer, tuning the file system / choice of file system, raid strip size etc.
    (I suppose file sizes are big for 3D rendering, and that you optimize for that)
     
  20. rgbaguy

    Newcomer

    Joined:
    Jan 23, 2010
    Messages:
    46
    Likes Received:
    0
    Hey guys, my apologies for the very late reply, and wow, have I been gone a while, whole new look to the forum! Had so much to deal with lately. I'm back now!

    Thanks for the replies!

    Will have a look at paragon, thanks! Any luck on your research on the server OS? I'm afraid I'm the last person to ask on that front :(

    Perhaps just use Open SUSE for the file server? Guess that should be fine? Why NFS over SMB?

    Cool, why SMB over NFS?

    Looking forward to more suggestions? Thanks guys :)
     
Loading...

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...