<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Maxim Abalenkov wrote:</p>
<p>
<blockquote type="cite">
<pre style="white-space: pre-wrap; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration: none;">Dear all,
I’m looking for guidance please. I would like to make sure, that I use all eight of my CPU cores, when I run Python’s 3.9.9 NumPy on my macOS BigSur 12.1. When I run my NumPy code, I see in ‘htop’, that only one ‘python’ process is running and the core utilisation is 20–25%. I remember in the past, stock MacPorts NumPy installation would use Apple’s Accelerate library including the multithreaded BLAS and LAPACK (<a href="https://developer.apple.com/documentation/accelerate" class="moz-txt-link-freetext">https://developer.apple.com/documentation/accelerate</a>). As I understand this is no longer the case.
I run Python code using a virtual environment located under
/opt/venv/zipfstime/lib/python3.9/site-packages/numpy/core
When I change there and issue
otool -L _multiarray_umath.cpython-39-darwin.so
_multiarray_umath.cpython-39-darwin.so:
@loader_path/../.dylibs/libopenblas.0.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
In other words, NumPy relies on openBLAS. Command `port variants openblas` returns
OpenBLAS has the variants:
g95: Build using the g95 Fortran compiler
* conflicts with gcc10 gcc11 gcc8 gcc9 gccdevel
gcc10: Build using the MacPorts gcc 10 compiler
* conflicts with g95 g95 gcc11 gcc8 gcc9 gccdevel
[+]gcc11: Build using the MacPorts gcc 11 compiler
* conflicts with g95 g95 gcc10 gcc8 gcc9 gccdevel
gcc8: Build using the MacPorts gcc 8 compiler
* conflicts with g95 g95 gcc10 gcc11 gcc9 gccdevel
gcc9: Build using the MacPorts gcc 9 compiler
* conflicts with g95 g95 gcc10 gcc11 gcc8 gccdevel
gccdevel: Build using the MacPorts gcc devel compiler
* conflicts with g95 g95 gcc10 gcc11 gcc8 gcc9
[+]lapack: Add Lapack/CLapack support to the library
native: Force compilation on machine to get fully optimized library
universal: Build for multiple architectures
I tried installing the “native” variant of OpenBLAS port with `sudo port install openblas +native` and setting the environment variable `OMP_NUM_THREADS=8`, but I didn’t see any improvement when running my Python code. I would welcome your help and guidance on this subject.</pre>
</blockquote>
I'm using py39-numpy with default variants:</p>
<p>% port installed py39-numpy openblas<br>
The following ports are currently installed:<br>
OpenBLAS @0.3.19_0+gcc11+lapack (active)<br>
py39-numpy @1.21.5_1+gfortran+openblas (active)</p>
<p>I see Python using around 600% CPU on my 6-core machine when
running this basic benchmark script:
<a class="moz-txt-link-rfc2396E" href="https://gist.github.com/markus-beuckelmann/8bc25531b11158431a5b09a45abd6276"><https://gist.github.com/markus-beuckelmann/8bc25531b11158431a5b09a45abd6276></a></p>
<p>If you try that and see how many cores it uses, that will at
least tell you if there is something different about your code. If
it doesn't use all the cores for you, there are some other
environment variables that OpenBLAS looks at that you could check:
<a class="moz-txt-link-rfc2396E" href="https://github.com/xianyi/OpenBLAS#setting-the-number-of-threads-using-environment-variables"><https://github.com/xianyi/OpenBLAS#setting-the-number-of-threads-using-environment-variables></a></p>
<p>- Josh<br>
</p>
</body>
</html>