[MacPorts] #62554: Let ports specify expected memory use per job

MacPorts noreply at macports.org
Thu Mar 25 09:27:48 UTC 2021


#62554: Let ports specify expected memory use per job
-------------------------+--------------------
 Reporter:  ryandesign   |      Owner:  (none)
     Type:  enhancement  |     Status:  new
 Priority:  Normal       |  Milestone:
Component:  base         |    Version:
 Keywords:               |       Port:
-------------------------+--------------------
 MacPorts sets the default value of `build.jobs` via this code from
 portbuild.tcl:

 {{{
 proc portbuild::build_getjobs {args} {
     global buildmakejobs
     set jobs $buildmakejobs
     # if set to '0', use the number of cores for the number of jobs
     if {$jobs == 0} {
         try -pass_signal {
             set jobs [sysctl hw.activecpu]
         } catch {{*} eCode eMessage} {
             set jobs 2
             ui_warn "failed to determine the number of available CPUs
 (probably not supported on this platform)"
             ui_warn "defaulting to $jobs jobs, consider setting
 buildmakejobs to a nonzero value in macports.conf"
         }

         try -pass_signal {
             set memsize [sysctl hw.memsize]
             if {$jobs > $memsize / (1024 * 1024 * 1024) + 1} {
                 set jobs [expr {$memsize / (1024 * 1024 * 1024) + 1}]
             }
         } catch {*} {}
     }
     if {![string is integer -strict $jobs] || $jobs <= 1} {
         set jobs 1
     }
     return $jobs
 }
 }}}

 In other words it uses the number of active hyperthreaded CPU cores or the
 amount of RAM in GiB plus one, whichever is less. The assumption is that
 each job will not need more than about 1 GiB of RAM.

 This assumption is wrong for some ports [ticket:58932 like py-tensorflow].
 Instead of making those ports recreate the above calculation to arrive at
 an acceptable number of jobs, let's make the assumption configurable.
 Introduce a new port option (perhaps `build.expected_memory_use_per_job`)
 set to a default value of 1024. Use it in the above calculation and let
 ports override it so that py-tensorflow could set it to e.g. 2560 instead.

-- 
Ticket URL: <https://trac.macports.org/ticket/62554>
MacPorts <https://www.macports.org/>
Ports system for macOS


More information about the macports-tickets mailing list