Travis CI timeouts for MacPorts builds

[Mojca Miklavec]

On Mon, 7 Sep 2020 at 18:25, Ruben Di Battista wrote:
>
> I was wondering... Would à Linux machine with some virtualization method (libvirt?) be acceptable as CI runner?

This is something we've been looking at, among others.
Now, a Linux machine with libvirt is probably not entirely legal, but
a mac with libvirt should be, and I guess it would work in exactly the
same way.

We tried to follow these instructions:
    https://github.com/kholia/OSX-KVM
and we got it mostly working, it's just annoyingly slow because the
image gets downloaded as part of the process (I don't know if you can
easily cache it if the installation fails).

The other thing is that I'm not sure if one could get this working
with older OS versions, or how to handle ARM.

Running the image on macOS should be acceptable.

> Any of you has experience in terms of performance?

I wouldn't worry too much. It's not like we are getting thousand PRs
per day. And if that becomes the bottleneck, we could always set up
another worker for the same macOS version.

We are already running the existing builders inside virtual machines.
It may take two weeks when the new macOS comes out (and days after
1000+ perl submodules get new subports), but the machines keep up with
the load. The only bottleneck was PowerMac.

> Also, Gitlab CI allows to attach personal runners to project very easily (just a package to install from the os package manager) . How does it work for Github?

GitHub doesn't have the CI as "tightly integrated" as the gitlab runners.
Basically you can "attach" almost any given CI.
If you look at commit history, you'll see green checks and red crosses.
If you click on those, you'll see the list of builds with links to our
buildbot builds.

Those are the "official builds" of the packages. We could have
something similar for the CI.

On Mon, 7 Sep 2020 at 18:52, Ralph Seichter wrote:
>
> * Mojca Miklavec:
>
> > If you volunteer to do some research / work in this area ... that
> > would likely be the most significant step in "increasing your chances"
>
> I actually have some experience in this field. I use GitLab (Omnibus
> edition), Docker, and GitLab Runners inside Docker for CI/CD. That's
> basically the setup which gitlab.com uses (albeit sans their S3-based
> caching), and it can handle serious builds.
>
> Doing research re MacPorts would require knowledge about the MP build
> infrastructure, which I don't yet have. I'd be willing to look into it,
> though, if that infrastructure is properly documented somewhere.

The part that requires research hasn't been implemented yet, so the
good news is that you don't need any special knowledge about the
existing infrastructure :)

What we need is:
- a list of recipes to set up images for a bunch of different macOS
versions (as far back into history as possible)
- a recipe for how to fire up a VM, do something basic (port install
foo) and trash the result

The next step would be to perhaps integrate this into buildbot, but
there's no need to use exactly buildbot for this. Anything else that
does the job would also serve just fine.

Mojca