Buildbot hardware (was: Re: Framing the MacPorts discussion)

Fri May 21 12:34:32 UTC 2021

Hi,

Thanks Ryan.

My answer is very similar to Ben’s:

  *   I’d be happy to provide you exclusive access to the resources (encrypted VMs, your own users, network and machine are UPS-protected, firewalled, etc.)
  *   I completely agree with you about the safety concerns: those should not be relaxed.
  *   I volunteered because I thought they were needed: I love MacPorts, and I want it to thrive.

Bye,
Enrico

From: Ben Greenfield <ben at cogs.com>
Date: Friday, 21 May 2021 at 13:26
To: Ryan Schmidt <ryandesign at macports.org>
Cc: Andrew Janke <floss at apjanke.net>, Enrico Maria Crisostomo <enrico.m.crisostomo at gmail.com>, MacPorts Developers <macports-dev at lists.macports.org>
Subject: Re: Buildbot hardware (was: Re: Framing the MacPorts discussion)
Hey All,

Thanks for the direction Ryan.

> On May 21, 2021, at 12:46 AM, Ryan Schmidt <ryandesign at macports.org> wrote:
>
> On May 19, 2021, at 12:38, Andrew Janke wrote:
>
>> I have a small stack of Mac Minis I got to use as a buildbot farm for Octave.app; I might be able to have them pull double duty for MacPorts depending on your change volume.
>
>
> On May 20, 2021, at 08:10, Enrico Maria Crisostomo wrote:
>
>> I've got an iMac Pro in my LAN with 16 vCores and 64GB or RAM which is quite often idle.
>> I'm not privy with how our build system work, but if we could get to a point where agents can be added, stopped, throttled, trusted members of our community could volunteer the computational power they have at their disposal without fully dedicating a machine.
>> In my specific case: I'm happy to offer VMs on that machine to volunteer computational resources.
>
>
> On May 20, 2021, at 08:20, Ben Greenfield wrote:
>
>> I can definitely donate the facilities if not the talent.
>>
>> I have a symmetrical fiber connection and a static ip. I also have battery backup.
>> I’m in the final weeks of making the building legal and I haven’t configured the final network set-up for the building. I was going to set-up a vlan on my hp procurve switch.
>> I’m still shopping for a router to run OPNsense I think.
>>
>> I have been a mac sysadmin long time.
>
>
> There seem to be a lot of people suddenly volunteering hardware for our build system. First, thank you; I didn't know we had people interested in that.
>
> Our build system has never been designed to accommodate external hardware. It has always been designed as a centralized system controlled by one administrator. When it was first set up in 2011-12 it was under the control of our Apple administrator at macOS forge. I became the macOS forge administrator temporarily in late 2015, and MacPorts left macOS forge in late 2016 as that service shut down, and I recreated the buildbot system on my own hardware and have run it since then.
>
> We now have one external Apple Silicon build machine hosted at another data center, but it's still under my exclusive control so that I can keep everything working together.
>

I would be happy to provide the same service. I don’t need a log-in and I can probably provide out of band power reset. The system could be on it’s own vlan.

> There are currently many situations where the build system gets into a state that requires manual intervention. Because I control all the machines, I'm able to make those fixes and get things back up and running quickly.
>
> We currently have all the builders we need: one for each OS version / arch combination. The system was never designed to have more than that. If for example we added a second macOS 11 / x86_64 builder, there could be confusion and problems if the two machines have different OS / Xcode / command line tools / java versions installed.
>
> There are security issues to consider. The binaries produced by our buildbot workers are signed on the master with our private key. This is our "seal of approval" that says we believe these binaries to be good and safe. Users trust that. If we start allowing other people to run build machines, then we have the problem that we do not know for certain whether those other build machines are free of malware or other problems. We would be signing binaries for distribution to users without being certain of their safety or correctness. I'm not very comfortable with that.

Yes, that safety should be maintained.

>
> Why is this discussion happening? Why do people think we need more hardware? If we need more or faster CPUs or more memory, I can make those changes to the hardware I already manage.

I volunteered because it sounded like resources might be needed:).

Let me know if the free-hosting is needed.

Ben

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macports.org/pipermail/macports-dev/attachments/20210521/110bdc7c/attachment.htm>