[MacPorts] #38766: Building atlas with clang 3.3 needs excessive memory

[MacPorts]

#38766: Building atlas with clang 3.3 needs excessive memory
-------------------------+---------------------
  Reporter:  bgschaid@…  |      Owner:  vince@…
      Type:  defect      |     Status:  new
  Priority:  Normal      |  Milestone:
 Component:  ports       |    Version:  2.1.3
Resolution:              |   Keywords:
      Port:  atlas       |
-------------------------+---------------------
Changes (by ryandesign@…):

 * owner:  macports-tickets@… => vince@…


Old description:

> Tried to do a routine upgrade of the installed software. Amongst others:
> {{{
> port outdated
> The following installed ports are outdated:
> atlas                          3.10.1_2 < 3.10.1_3
> }}}
> The installed atlas is
> {{{
> port installed atlas
> The following ports are currently installed:
>   atlas @3.10.1_2+gcc45 (active)
> }}}
> The update started by fetching clang3.3 which made me suspicious because
> in the past I already had the experience that clang needs much more
> memory than gcc, but I figured "Hey. Surely the guy who packaged it knows
> what he's doing". Compilation of atlas took several hours. When I left
> the computer and returned an hour later it turned out that the disk which
> previously had 20Gig free had filled because of bloated swap-files.
> Computer was thus unusable. Only thing I could see from a htop I had
> running was some program of user macports with 14Gig Virtual and 4.5Gig
> residual memory
>
> As gcc45 is no longer an option for atlas I did a "port upgrade atlas
> +gcc46" and the compilation finished after a quarter of an hour. Maybe
> that has to do with
> {{
> {Warning: GCC compilers on MacOS do not support AVX: downgrading.
> }}}
>
> Anyway: would be nice if the default settings for the port would be such
> that it compiles without problems on machines with moderate memory
> installation (mine has 8Gig. The maximum that fits into that model).
> Those who need the extra speed that CLang might provide can always choose
> that variant I think

New description:

 Tried to do a routine upgrade of the installed software. Amongst others:
 {{{
 port outdated
 The following installed ports are outdated:
 atlas                          3.10.1_2 < 3.10.1_3
 }}}
 The installed atlas is
 {{{
 port installed atlas
 The following ports are currently installed:
   atlas @3.10.1_2+gcc45 (active)
 }}}
 The update started by fetching clang3.3 which made me suspicious because
 in the past I already had the experience that clang needs much more memory
 than gcc, but I figured "Hey. Surely the guy who packaged it knows what
 he's doing". Compilation of atlas took several hours. When I left the
 computer and returned an hour later it turned out that the disk which
 previously had 20Gig free had filled because of bloated swap-files.
 Computer was thus unusable. Only thing I could see from a htop I had
 running was some program of user macports with 14Gig Virtual and 4.5Gig
 residual memory

 As gcc45 is no longer an option for atlas I did a "port upgrade atlas
 +gcc46" and the compilation finished after a quarter of an hour. Maybe
 that has to do with
 {{{
 Warning: GCC compilers on MacOS do not support AVX: downgrading.
 }}}

 Anyway: would be nice if the default settings for the port would be such
 that it compiles without problems on machines with moderate memory
 installation (mine has 8Gig. The maximum that fits into that model). Those
 who need the extra speed that CLang might provide can always choose that
 variant I think

--

Comment:

 It is intentional that atlas now defaults to clang; see r104549.

 Yes, it's known that clang can use much more memory than gcc in some
 circumstances.

 MacPorts usually starts multiple compiler processes, and by default it
 limits this to one process per CPU core or 1 process per GB of memory,
 whichever is less. But these limits were decided upon before we started
 using clang. Perhaps we should decrease this to one process per 2 GB of
 memory when clang is in use.

 Individual ports can override this e.g. using `use_parallel_build no` to
 turn off parallel building entirely, and to my surprise, the atlas
 portfile already does this. So either the atlas build system is taking
 matters into its own hands about how many jobs to start, in which case it
 should be disabused of that notion, or a single clang process is taking
 that much memory, in which case that's very unfortunate.

-- 
Ticket URL: <https://trac.macports.org/ticket/38766#comment:1>
MacPorts <http://www.macports.org/>
Ports system for OS X