Make Compiler Dependency Explicit

[Ryan Schmidt]

On Jul 10, 2019, at 10:42, Jahrme Risner wrote:

> tl;dr Can a Portfile compiler be explicitly set as a `depends_build` dependency instead of `depends_lib`?
> 
> I have been working with the LilyPond (lilypond.org) development team on setting up a workflow to efficiently produce distributable builds of LilyPond (`lilypond` and `lilypond-devel` in MacPorts) for 64-bit macOS. All current builds are produced using GUB (https://github.com/gperciva/gub) which is a custom build/bundling system that supports darwin-ppc and darwin-x86 but *not* darwin-x86_64. With the upcoming release of macOS Catalina dropping support for 32-bit binaries, we need a way to build 64-bit binaries. While I believe that ultimately the "right" solution is to suggest users install MacPorts as they can then be notified of updates, there have been posters who have made it clear in no uncertain terms that requiring users to install MacPorts is not acceptable.
> 
> An approach that several other developers and myself feel could be be both robust and efficient is to use MacPorts `mpkg` or `mdmg` commands to produce the distributable binary packages. The first big issue is that the Portfile for `lilypond-devel` currently produces an ~1.8G package which is obviously much too large (for context, the current 32-bit package is 25M). Using the trace flag for builds as well as adjusting variants, I can now produce a metapackage that is only ~304M. Of the 304M in the package, the largest component is gcc (and its dependencies) as the metapackage for gcc8 is 234M. While there may be shared dependencies that cannot be removed, the size of the gcc metapackage makes me very hopeful that is is possible to build a sub-100M distributable build of LilyPond (potentially as small as 70M).
> 
> To answer why we need gcc and cannot use clang: clang does not support one of the c++ features heavily utilized by LilyPond. For more details, you can read through the two threads ([1] and [2]) at the end of this email.
> 
> The issue I have not managed to find a solution for is that the compiler is an *implicit* dependency added through the `compiler.blacklist`/`compiler.whitelist` mechanism and is added as a `depends_lib` dependency. So thus we arrive at the actual question:
> 
> Is it possible to make whichever compiler is used (using the whitelist/blacklist feature) be declared a build dependency instead of a library dependency?
> 
> Thank you all!
> 
> Also, if this should have been directed to a different mailing list I apologize and would appreciate a nudge in the right direction.
> 
> [1] https://lists.gnu.org/archive/html/bug-lilypond/2016-07/msg00071.html
> [2] https://lists.llvm.org/pipermail/cfe-users/2018-November/001417.html

A compiler is only needed at build time, and should be declared in depends_build. gcc also has runtime libraries which your software may be using. These are needed both at build time and at runtime, so they should be declared in depends_lib.



MacPorts 2.5.4 and earlier aren't smart enough to do that. If you specify a gcc compiler, MacPorts base knows that there's a library involved so it adds the whole gcc compiler port in depends_lib.

This has been fixed [1] and will be released in MacPorts 2.6. Since this has not yet been released, it has not yet been tested by as large an audience as we would like, so there may be bugs. If you find any, please let us know.



You can get the fix now by building MacPorts base from master [2].



In almost all cases, ports should not touch compiler.whitelist. Instead, add forbidden compilers to compiler.blacklist. If that leaves no working compilers, append working compilers to compiler.fallback.

If you insist on setting compiler.whitelist, then there is no point to modifying compiler.blacklist or compiler.fallback since the first entry of compiler.whitelist that is not blacklisted will be used.



[1] https://github.com/macports/macports-base/commit/e0bd891f757606ff99e0611048de4f48f71dcce5

[2] https://guide.macports.org/chunked/installing.macports.html#installing.macports.git