Installing ports in parallel

[Chris Jones]

Hi,

On 11 Sep 2012, at 09:58 PM, Peng Yu <pengyu.ut at gmail.com> wrote:

> On Tue, Sep 11, 2012 at 12:50 PM, Ryan Schmidt <ryandesign at macports.org> wrote:
>> 
>> On Sep 11, 2012, at 11:57, Peng Yu <pengyu.ut at gmail.com> wrote:
>> 
>>> I don't find a way to let port install multiple independent ports in
>>> parallel to maximally use the cores on a machine. Is there a way to do
>>> so? Thanks!
>> 
>> This used to be allowed (unintentionally, I think) and would sometimes work, as long as the ports you were installing were truly independent of one another (including their complete dependency chains). But as soon as you started trying to do simultaneous port installs where parts of the dependency chains overlapped, and some of those overlapping dependencies were outdated or not installed yet, things did not work. Confusing error messages were printed that users often did not understand.
>> 
>> We "fixed" the problem a few versions of MacPorts by introducing a lock mechanism that ensures that you can only install one port at a time. I miss the parallel install capability we used to have, but appreciate that this reduces the number of confusing problems our users could encounter, and thus decreases the number of problem reports we receive, which frees up our time to work on more interesting problems.
> 
> There are many tools out there to handle dependencies for compiling
> software and for many other purpose as well. For example, GNU make,
> cmake, ant, maven, etc. I think that there should be (at least
> theoretically) some way of harness some of these tools to handle
> parallel builds of multiple ports even if they share dependency
> chains. Whether it is easy to implement is another issue and I don't
> know the answer.
> 
> My inclination is that the feature that I request can be useful. As I
> can port using binaries for only a very small fraction of ports.
> Enabling parallel port can truly use all the cores to speed the
> process. Recently, I just migrate from Snow Leopard to Mountain Loin,
> It took almost 8 hours for me to completely install what had been
> previously installed. There should be something done to improve this
> process.


I still Do not see what you really expect to gain. Installing from source is, in my experience, dominated by the build phase, and this *already* will build ports in parallel (make -j X or similar) where it can. Just run top and you will see this. Your cores will already be fully occupied most of the time, even it's only one instance of port running. At least that is exactly what I see on my machines.

Chris

> 
> -- 
> Regards,
> Peng
> _______________________________________________
> macports-users mailing list
> macports-users at lists.macosforge.org
> http://lists.macosforge.org/mailman/listinfo/macports-users