Travis CI timeouts for MacPorts builds

[Mojca Miklavec]

On Mon, 7 Sep 2020 at 20:25, Ralph Seichter wrote:
> * Mojca Miklavec:
>
> > We should figure out:
> > - which dependencies time out
> > - why they are not installed from (the private) binary package repository
>
> I'm guessing that "we" means the devs who maintain the build config.

Oh, no, that's the wrong interpretation.
"We" includes anyone interested in helping out.
There are certain tasks (like enabling some external service on the
github, connecting with some new hardware, ...) that can only be done
by a small group of people with administration access, but everything
else happens via pull requests (or sometimes direct commits) or new
code / repositories.

The community depends exclusively on volunteers. Many of the
contributors who devoted more time in the past and made some major
contributions have a lot less time to spend with the project nowadays,
so anyone willing to help out is mostly welcome.

> If
> I can assist, let me know, but I don't think it likely at this point.

Because you lack time or for some other reason?
If you think you could help, we would be most grateful.

> If you ignore Travis results, why run Travis CI in the first place?

Travis was the first CI that we supported for pull requests.
@l2dy wrote the initial support as a GSOC project which included
writing tons of workarounds for Travis limitations.
He redirected output to pastebin in order to be able to even show it
(most of our output exceeds the Travis limit for how long the log can
be), he had to inject bogus writes to prevent Travis from thinking
that the job was stuck, he implemented a lot of tricks to ensure that
the bootstrapping phase was as fast as possible, ... but we could not
change the time limit itself.

After a while Azure came around with 6 hours of timeout which was a
lot more useful, and the same solution was ported from Travis to
Azure. I forgot why we never switched from Azure to GitHub Actions.

It's not that Travis is completely ignored. There are some useful
cases, like this one for example:
    https://travis-ci.org/github/macports/macports-ports/builds/724968783
(I checked a random failed build from the recent PRs.)

You can see that the build passed on 10.14, but failed on 10.11 up to
10.13. It failed quickly with
    "Xcode 9.0 or greater is needed to build carthage; only found
version 7.3.1."
which is something that the portfile should have declared and it
should have at least failed in pre-fetch phase already (ideally we
would need a declarative syntax for this).

This case is useful. Now, we do have Azure build for 10.13, so this
one is overlapping (and we should probably remove that one), but
there's no builder for 10.11 and 10.12 on Azure.

In reality most of the Travis build failures fall into two categories:
- timeout (most of the time)
- network or timing issues (sources could not be fetched with "server
not found", or the clocks didn't match, and MacPorts refuses to run if
file appear to be "from the future")

As far as I know there were attempts to address both of these. Timeout
will always be the case when a single PR touches many ports, or where
ports are simply time-consuming to build (no chance for Qt, ...) There
might be room for improvement as far as build time of dependencies is
concerned though.

On Tue, 8 Sep 2020 at 18:28, Ryan Schmidt wrote:
> On Sep 7, 2020, at 11:24, Ruben Di Battista wrote:
>
> > Would à Linux machine with some virtualization method (libvirt?) be acceptable as CI runner?
>
> I can't see how, since we're trying to verify builds of ports, and there is no general expectation that port builds would work on non-Mac operating systems.

We are not talking about running the builds on Linux, but running the
virtual machines on Linux, and if wasn't for the legal reasons, this
would be a perfectly viable solution (provided you get all the VMs
working, to start with). But if you get it working on linux for
testing purposes, a very similar recipe might work on macOS.

Mojca