Framing the MacPorts discussion

[David Gilman]

Over the past month or so there has been a few threads on the
direction of the project, what can be improved and what the vision
should be. I wanted to summarize my two cents because I haven't been
otherwise participating and hopefully this can adjust the community
mindset towards what the problems are and frame their solutions.

TL; DR
- We're all package maintainers so MacPorts is structurally going to
be short staffed on CI (builder) work because CI is a different set of
skills and interests. Implement a structural solution like
positions/delegation to volunteers to explicitly work on CI
- Cloud hardware too expensive to consider
- In lieu of cloud, design CI that can be run on trusted volunteer's
home/office donated machines
- Consider Linux/QEMU for builder virtualization

As a user of MacPorts I greatly value having binaries available. It
would be incredible if the only packages I build are the ones that I'm
trying to maintain. There have been a few threads where users thought
binary builds weren't available or were rarely available so it seems
like it's in high demand. It is worth it to prioritize CI changes that
allow for more binaries to be available.

MacPorts has always been short on labor for the CI system. We're still
using the roots of a buildbot setup that was originally built on
Apple's dime, if my history is correct. I don't think this should
surprise any of us, though. We're all here, involved with MacPorts and
port maintenance, because we're interested in the packaging side of
things. We're not people with the time or inclination to work on CI
tasks. That's not a bad thing, but it's something that the
organizational structure should account for in order to keep stuff
maintained. Because this is a structural problem I think the mindset
of the MacPorts community is that a fix to the issue is going to
involve structural changes.

I think the way out of this is to recruit volunteers explicitly to
work on CI tasks. Trying to get package maintainers to cover CI work
is, and has been, getting blood from a stone. On the MacPorts
administration side there needs to be a bit of work put into figuring
out what the formal relationship and expectations are between CI
volunteers and the MacPorts leadership. There are lots of ways forward
there and lots of good ways to structure it. There is going to have to
be some delegation and the relationship needs to handle that.

For the CI work that does initial builds of PRs, I unfortunately think
the only technical solution here is GitHub Actions and third party
services like it. There's a plague of cryptocurrency miners looking to
exploit these systems and nothing a MacPorts volunteer team can build
will win in that game of cat and mouse. So for the future of PR CI I
think the conversation needs to be around using and maintaining those
third party solutions.

There's a thread on here about raising money to pay for CI (for
builders). I'd like to remind everyone that cloud bills are damn
expensive and I don't think MacPorts has the dedicated user base to
donate to keep that CI alive. For the vast majority of humans on the
planet a monthly donation of $20-$40 USD is a massive expenditure, and
I would be surprised if we could get a reliable $500 a month out of
everyone for MacPorts. That $500 a month can get you some cloud
computing but not a ton, and likely not what we need for reliable CI.
It's more likely than not we'd have trouble getting donations to keep
the lights on, it would be a constant struggle to fund raise, and the
whole issue around accepting donations and incorporation and that
stuff. I encourage everyone involved to write off the idea of doing
cloud CI for MacPorts builders.

Not being able to use cloud technology also hurts the ability to
recruit CI volunteers, so this tradeoff needs to be made explicit up
front. But hopefully some people are interested in a challenge.

So if a recurring cash donation is a struggle for individual donors I
think the way to get hardware for CI is a bit of a psychological hack:
ask people to buy/donate build machines and run them out of their own
offices or houses. My gut is that individuals are willing to spend
more (or at all) if it's a one time thing and it's tied to a concrete,
physical item. There's also probably enough used and aged hardware
amongst the MacPorts community that could be donated to make up a good
enough build farm. The operators of the machines are also likely
comfortable donating the electricity cost as it'll just be lumped in
with the rest of their bill. And for new machines an achievable
Kickstarter of $1000-$2000 could probably get enough donors and would
result in a fine build server.

This has a few implementation challenges. The CI system will have to
be built to work with a build farm of machines of varying reliability
and speed. Only trusted members of the community would be allowed to
host. The CI design would also need to have network isolation between
the host's personal/corporate network and the builder, and they'd have
to be OK with giving SSH credentials to the CI team.

All of this CI design has Linux in mind. If it is acceptable to the
MacPorts team, I strongly encourage running builders under QEMU
virtualization. Linux will let us use commodity hardware and QEMU can
reliably run MacOS X versions from a variety of releases, including
PPC systems, and with hardware that is way faster than any PPC machine
Apple ever shipped. If QEMU can reliably pause, snapshot and resume
running builder VMs, and the machines have a SSD from the past 5 years
or so, we're looking at a pretty speedy build setup.

I'm aware that there are certain things about running MacOS X on QEMU
that may make it an unacceptable choice for MacPorts. Personally, my
risk tolerance there is fine with it, I think Apple's poem is a nod
and a wink to use cases like ours so you can get a solution that works
in a reasonable budget and with reasonable hardware performance. But I
don't know the history of this discussion and maybe it is a
non-starter for MacPorts.


-- 
David Gilman
:DG<