Volunteer for a workshop on "setting up your own buildbot/buildslave"?

[Ryan Schmidt]

On Nov 10, 2015, at 7:26 AM, Mojca Miklavec wrote:

> But cxxstdlib is one of the most important variable that would be a
> prerequisite for allowing modern sofware to be built on older systems.
> I believe it would be worth adding it (even if we won't ever manage to
> get new buildslaves added to the official build system).

Let's proceed under the assumption that we can get new buildslaves for this.


> I assume that
> should be easy enough to add.

It would be easy enough to do, once we reach a decision on what, specifically, to do.


> A while back we were discussing about
> how to implement this. My suggestion would be to only add that on <
> 10.9 if libc++ is used. And to make things easier: unless it's a
> noarch port, always add it, even if the port doesn't link against
> libc++ explicitly. At the moment we don't have any procedure in place
> to aid with decision about whether or not "libc++" is needed in the
> filename, but we would like to start somewhere. For completeness one
> could add "libstdc++" if libstdc++ is needed on >= 10.9. This approach
> isn't 100% correct in all cases (in particular not in cases where g++
> from macports-gcc is used), but it will allow us to proceed and to fix
> more subtle problems later.

I like the idea of only adding a non-default cxxstdlib to the filename, since that would allow all the existing archives to continue to be valid. However if we were to switch to xz compression for archives, we might want to repackage everything that way, in which case it wouldn't matter.

I agree that since we don't have a mechanism for detecting C++ software, it would be simplest to add the tag to all archives, even though that will result in a rather large increase in disk usage on the packages servers. If the packages server were using some sort of advanced automatic deduplicating filesystem the impact might not be so large, since many ports would hopefully build identically, but I don't know what filesystems do that or if we're using one of them. I worry about increasing the disk space used, both for our current host Mac OS Forge as well as for the various voluntary mirror servers around the world, and it's also a consideration in the event that we might one day need to leave Mac OS Forge and have to find another place to house these mountains of data we're contemplating creating.

Another option: It should be possible to write code to detect whether any files in a destroot use a C++ library, and if so, MacPorts could include the C++ library name in the archive filename, otherwise don't. At archive fetch time, MacPorts wouldn't know whether a port uses C++ or not, but it could try to fetch both filenames and use whichever one exists.


> Archive type is implicit (and I would still vote for trying to switch
> to xz one day).

I agree xz would be better because it is smaller.

I've previously assumed that since OS X does not include an xz executable, this would require us to bundle a copy of the xz executable with MacPorts, like we currently bundle Tcl 8.5. Since the xz executable is GPL-licensed, this might be a problem. If so, we could perhaps instead have MacPorts make direct use of the lzma library, which is in the public domain. That would be a change in how MacPorts works, however.

But I've now learned (or maybe I knew this before and forgot) that GNU tar and BSD tar both have built-in support for xz. The El Capitan version of tar supports this. We would need to see how far back this support goes. It was Mavericks when Apple removed GNU tar but I don't remember if they also had BSD tar before that. Wikipedia says BSD tar gained xz support in April 2009.

Looks like GNU tar does this by running the xz executable, but BSD tar uses libarchive, which uses liblzma. The libarchive port installs the bsdtar executable too. We might be able to have MacPorts use a bundled bsdtar/bzip2/libarchive/liblzma/zlib instead of the system's tar and compression utilities.