CI system for PR builds

[Mojca Miklavec]

On 7 April 2018 at 15:45, db wrote:
> On 7 Apr 2018, at 14:37, Ryan Schmidt wrote:
>
> Is buildbot running on your basement???

Yes (not mine).

> Streamlining this process could have been something for GSoC.

Until the autumn of 2016 we had no concept of pull requests at all and
we had nearly no influence on what exactly ran on the buildbot
infrastructure.
Until the late summer of 2017 we had no checking of pull requests at all.

We in fact had a GSoC student in 2017 who set up automated testing of
pull requests using Travis. We were discussing alternatives (adjusting
the buildbot) to achieve that goal before GSOC, but the student was
not interested in working on that, also considering all the
security-related issues which would be causing headaches that he was
not willing to deal with. Despite that,

It is too late for a GSoC project in 2018 (we did discuss this idea a
month or two months back when we had slightly less insight into how
this could work properly in the first place; that was before Rainer
knew what he wrote in one of these emails; and the student would have
to be self-motivated to figure out the solutions himself since most of
us would not be able to help without spending a lot of time
ourselves). If you want to have this functionality as a product of
GSoC, we cannot get it before autumn 2019 and even then if and only
if:
- we actually find a student willing to work on that,
- the student successfully finishes the project.

> Testing and reproducibility, doesn't seem to me as user to be a prime concern in MP.

We still have a long way to go before we can achieve 100% binary
reproducibility. However, regarding other points:

- We only have 4 developers with mostly full-time jobs (unrelated to
MacPorts) who work on MacPorts infrastructure in their free time.
Apart from the infrastructure those same developers are almost the
only ones developing MacPorts base and maintaining more than 1000
ports. It is unfair to expect magic powers from them.

- Making sure that 20k ports keep being up-to-date and working on
almost everything from 10.6 to 10.13 is anything but trivial. HomeBrew
"solved" the problem by archiving lots of working formulas and only
keeping the most popular ones and leaving it up to others to provide
private taps (leaving a lot of users quite unhappy, some of them
switching to another "scientific" cross-platform package manager).

- I don't think that our #1 problem are new commits. OK, some
committers take somewhat less care before committing. In my (personal)
opinion a much bigger problem is:
  * a huge number of existing broken or outdated ports (or both)
  * our inability to efficiently address 2400 known bugs (no patch),
400 submissions (with code), 300 other patches from Trac, ...
    see https://trac.macports.org/wiki/Tickets
  * that we currently have no overview which ports exactly are broken
and on which platforms

We now have a very details plan to address the latest point (getting a
clear overview of which ports are broken), with a pretty high
priority.

What we need right now is mostly *code* (hardware would also help) as
well as time & effort to help us reduce the number of all those Trac
tickets, rather than pointing fingers to what MacPorts volunteers are
not doing.

> That streamlined process is what keeps new and updated portfiles in my local repo…

I have no clue what you wanted to say with this.

Mojca