Distributed compilation?

[RussSchultz]

I'm looking at distributed compilation for a embedded compiler under windows.

I've found stuff for gcc, unix, VC++ on windows, but nothing directly on target.

Any suggestions?

 

[Frank]

Can you explain that a bit more?

 

[RussSchultz]

I'd like to be able to submit my build job and have it be parallelized across many computers.

We don't use GNU or VS.NET, or UNIX (at the moment).

 

[Frank]

I'd like to be able to submit my build job and have it be parallelized across many computers.

We don't use GNU or VS.NET, or UNIX (at the moment).

A C++ compile job, that runs on PC's in the LAN, not on the embedded devices themselves? How big is that job?

 

[RussSchultz]

Right. The build time right now is about 4 hours, all told.

I'd like to see if anybody knows a way to kick off builds across the network to shrink this down to 30 minutes or less.

 

[Frank]

Linking is probably not the problem, and would be hard to distribute anyway. If you can get the libraries accepted by gcc, that might work for the compiling. And then you could link the resulting files on your own computer, if the object format is accepted by your embedded compiler.

Alternatively, you could just split the makefile into multiple parts, if you're going to build everything anyway. If there are still interlinked dependencies, it might be best to lift them and put them in a single file you compile first and distribute with the rest. That would be good practice in any case, if possible.

Edit: you might need to cascade multiple make files for each target to handle the normal dependencies.

 

[Simon F]

I'd like to be able to submit my build job and have it be parallelized across many computers.

We don't use GNU or VS.NET, or UNIX (at the moment).

Pity because there's a simple solution involving GNU Make and "Grid Engine".

How many files are you compiling when you do a build or do you run other programs as part of the build process?

 

[Blazkowicz]

I think what you're looking for is distcc http://distcc.samba.org/

well I guess you already found it, and it's for unix/linux, not for windows; but you could run it in Cygwin I guess.

 

[RussSchultz]

electric cloud is one I found.

there's supposedly a 'dmake', but there's not much information about it.

 

[[maven]]

The new MacOS supports that quite nicely, but if you're not afraid of a bit of manual work, you might be able to work with http://www.cbloom.com/misc/index.html, look for NetSlave near the bottom of the page...

 

[Frank]

For a makefile and Windows, the "at" command ("at /?" in a dosbox) is the easiest way to distribute tasks. You might need to ask IT management to enable it, and the time on the PC's should be synchronized.

It's quite easy to write a program that handles the distribution actually. I've made a simple one that starts random tasks on random PC's, and it was ready in a few hours. It's written in Delphi.