openmpi-* subports, and publishing of binaries

[Ryan Schmidt]

On Feb 1, 2021, at 09:49, Christopher Nielsen wrote:

> Given how long the various openmpi-* subports take to build, it would be beneficial to publish binaries for our users.
> 
> This is currently disabled in the Portfile, with a brief mention that it’s due to ‘hwloc’... and the tight coupling between the compiled binaries, and the hardware it’s compiled on.
> 
> Digging through all information resources available - including git history, old trac tickets, mailing list archives, etc - I couldn’t find much discussion surrounding this.
> 
> So, assuming this isn’t a simple licensing issue (?), does anyone have an answer?
> 
> Of note, my goal is to eliminate any technical barriers (if possible), by engaging the OpenMPI developers. But before doing that, any relevant background would be helpful.

According to the comment in the portfile, "openmpi depends on hwloc and that would be different for a user's machine".

The hwloc project is here: https://www.open-mpi.org/projects/hwloc/

According to "port info hwloc", "The Portable Hardware Locality (hwloc) software package provides a portable abstraction (across OS, versions, architectures, ...) of the hierarchical topology of modern architectures, including NUMA memory nodes, sockets, shared caches, cores and simultaneous multithreading."

I'm not familiar with all of that but I assume the situation is that openmpi, via hwloc, will determine the capabilities available on the build machine's processor and try to use those capabilities to the fullest at runtime. If something built on our build machines with their Intel Xeon processors were distributed to a user with let's say an Intel Core 2 Duo processor, it is possible that the processor features that openmpi/hwloc determined were available on the Xeon are not available on the Core 2 Duo, and that the program would crash or work incorrectly in some way.

If what I've described is accurate, then to accomplish what you want, you would need to modify the openmpi port so that it does not do that: does not use hwloc, does not attempt to discover specific processor features. Make it build the same on any Intel processor. Then we could distribute the binary.

A variant could be added that reintroduces the processor-specific optimizations. We have standardized on the variant name "native" for this purpose. Users who wish to have the very fastest software, at the expense of portability, can enable this variant either on a port by port basis or in their variants.conf file.