[MacPorts] #58932: py-tensorflow: reduce build time

MacPorts noreply at macports.org
Sat May 23 08:41:15 UTC 2020


#58932: py-tensorflow: reduce build time
----------------------------+--------------------------
  Reporter:  ryandesign     |      Owner:  emcrisostomo
      Type:  enhancement    |     Status:  assigned
  Priority:  Normal         |  Milestone:
 Component:  ports          |    Version:
Resolution:                 |   Keywords:
      Port:  py-tensorflow  |
----------------------------+--------------------------

Comment (by cjones051073):

 bazel has a quite involve inbuilt resource management system, that
 monitors the ram and cpu usage. Its just it appears its defaults are a bit
 too hungry for the resources the buildbots have (most probably the
 ram/core is a bit lower than anticipated). It also has a bunch of flags to
 limit things, like

 {{{
  --local_ram_resources=HOST_RAM*0.75 --local_cpu_resources=HOST_CPUS*.75
 }}}

 which should help.

 One other thing though - MacPorts doesn't just set `build.jobs` to the
 number of cores a machine has. It actually sets it to the number of
 hardware threads available *including* hyper threading. Which for many
 machines means 2*number of cores. e.g. on my MacBook pro here, which has 4
 actual CPU cores, its set to 8. MacPorts just uses

 {{{
  > sysctl hw.activecpu
 hw.activecpu: 8
 }}}

 what does this return on the buildbot VMs ? Does it return the actual
 number of physical cores you have assigned each VM, or something else ?

 In a lot of cases setting `build.jobs` to the number of logical cpus,
 taking hyper threading into account, is probably a good idea (its what
 they are there for). But perhaps on the build bots it would be better to
 use

 {{{
  > sysctl hw.physicalcpu
 hw.physicalcpu: 4
 }}}

 to limit things to the physical cores.

-- 
Ticket URL: <https://trac.macports.org/ticket/58932#comment:7>
MacPorts <https://www.macports.org/>
Ports system for macOS


More information about the macports-tickets mailing list