Volunteer for a workshop on "setting up your own buildbot/buildslave"?

[Mojca Miklavec]

On Tue, Nov 10, 2015 at 11:42 AM, Ryan Schmidt wrote:
> On Nov 10, 2015, at 4:31 AM, Mojca Miklavec wrote:
>
>> Hi,
>>
>> On Mon, Nov 9, 2015 at 10:29 PM, Ryan Schmidt wrote:
>>> On Nov 9, 2015, at 2:34 AM, Artur Szostak wrote:
>>>
>>>> Let me ask another question: Is there a seamless way to add building and mirroring services from 3rd parties for the pre-built binaries?
>>>
>>> No. We want verified binaries built in a clean-room by known servers, not binaries built in unknown conditions by arbitrary contributors.
>>
>> But from what I understand one should be able to *manually* add a
>> buildbot's IP or URL to fetch the binaries? I totally agree that
>> MacPorts should not support adding arbitrary buildbots automatically,
>> but it should be possible to add your own, right?
>>
>> Me and Aljaž were discussing the idea of:
>> - bringing a recent decent mac to the MacPorts meeting (in March 2016)
>> - bringing a PowerMac G5
>> - trying to set up a local mirror (for distfiles etc.) and a local
>> buildbot/buildslave on both
>> - preparing a short workshop/tutorial about setting up your own
>> buildbot/buildslave (with someone preparing slides with clear
>> instructions and giving enough time for a hands-on exercise)
>>
>> The output/benefit of this workshop would be the following:
>> - having a 10.5 buildslave (ideally also including libc++ support) at
>> hand to get feedback about potential problems with software (and
>> hopefully having an unofficial 10.11 buildslave to play with)
>> - get more developers familiar with the process
>> - so that potentially some university/company could easily set up
>> their own buildbots according to their local needs, or maybe just to
>> build some additional software that is not part of the official
>> distribution
>> - so that a group of "hackers" with mutual trust could use their own buildbots
>> - so that a group of developers could decide to easily test some
>> nontrivial changes in private trees with portfiles (like a switch from
>> multiple versions of Perl to a single version; or a move of Qt from
>> one location to another)
>> - so that some eager user could monitor building problems of the
>> latest commits in case the official buildbots experience problems
>>
>> My question is: is there any volunteer on this list (independent on
>> whether he or she will be able to physically attend the meeting) to
>> prepare the workshop [materials] and test the procedures?
>
> You could presumably edit archive_sites.tcl to add another packages server IP address. (MacPorts does not fetch from the buildbot servers directly; the buildbot servers upload to the packages server, and MacPorts fetches from that.)
>
> I think interest in PowerPC systems is truly very low at this time. But I would like to see the ability to automatically build packages for older systems with libc++, and offer a MacPorts installer that comes preconfigured for libc++ on older systems.
>
> But we can't have a buildbot for libc++ on older systems until MacPorts base gets a mechanism for differentiating libc++ packages from libstdc++ packages, for example with an extra string in the package filename. We've brought this up before. There are many variables that go into a package: name, version, revision, variants, platform, platform version, and architectures currently go into the filename; archive type, prefix, frameworks dir and applications dir are currently specified in archive_sites.tcl; and other variables such as cxxstdlib, macosx_deployment_target and macosx_sdk are assumed to be at their defaults, and any users who modify those variables are expected to know to disable the use of binaries, lest they get a binary built with different values for those variables. It might be nice to have a more robust way of specifying all the relevant variables. I'm not sure what that method would be; we can't just go adding endless variables to package filenames.

If we start including macosx_deployment_target, macosx_sdk, prefix,
applications_dir, frameworks_dir, ... we'll sooner or later end up in
an exponential mess (even though the same is also true for variants
and architectures if we would start building for more combinations of
variants at the same time). There is also

But cxxstdlib is one of the most important variable that would be a
prerequisite for allowing modern sofware to be built on older systems.
I believe it would be worth adding it (even if we won't ever manage to
get new buildslaves added to the official build system). I assume that
should be easy enough to add. A while back we were discussing about
how to implement this. My suggestion would be to only add that on <
10.9 if libc++ is used. And to make things easier: unless it's a
noarch port, always add it, even if the port doesn't link against
libc++ explicitly. At the moment we don't have any procedure in place
to aid with decision about whether or not "libc++" is needed in the
filename, but we would like to start somewhere. For completeness one
could add "libstdc++" if libstdc++ is needed on >= 10.9. This approach
isn't 100% correct in all cases (in particular not in cases where g++
from macports-gcc is used), but it will allow us to proceed and to fix
more subtle problems later.

Archive type is implicit (and I would still vote for trying to switch
to xz one day).

We can think of better solution in the future (I'll put this on the
agenda for the meeting as well), but we need solutions to address our
needs right now (or we can keep thinking for another decade until we
will no longer care about supporting systems < 10.8). I don't think
that adding just "libc++" on the already long enough filenames would
cause any immediate problems.

Mojca