Request for enabling some web hooks for the new buildbot test setup

[Mojca Miklavec]

Dear Rainer,

(or anyone else from the infra team; I'm forwarding this to the
development mailing list as there's some more background info, I hope
that's ok)

Would it be acceptable to enable some web hooks in the base & ports
repository for testing?

On Tue, 2 Jul 2019 at 08:41, Rajdeep Bharati wrote:
> On Tue, Jul 2, 2019 at 11:21 AM Rajdeep Bharati wrote:
>>
>> Hi,
>> I had a query: are webhooks enabled in the MacPorts repos?
>
> I guess it's not.
> Would it be possible to add webhooks in the base and ports repositories:
> payload url: http://34.67.147.76/change_hook/github
> send push and pull request events
>
> Thanks, Regards
> Rajdeep

I believe they could be for some things, but of course that doesn't
help you much if you don't get them delivered to your IP, so they
definitely need to be enabled "just for you" :)


At the moment we send the information from the buildbot back to github
when the port builds are finished (the same information is not enabled
for the base; it probably could be, but nobody cares so much; it's way
more important for the ports, precisely due to the lack of other ways
of retrieving information about the past success or failure of builds
of individual ports; which is hopefully about to change).

You can see that if you look at
    https://github.com/macports/macports-ports/commits/master
and check the red crosses or green checkmarks.

The functionality is really nice, albeit not perfect as far as the
port builds go. What's currently missing (partially also due to
limitations of buildbot 0.8) is that clicking the red cross will tell
you "build failed", but it doesn't tell you which ports failed to
build, just that "portwatcher" failed, and it gives you a link to the
portwatcher in question, which in itself is pretty cryptic and
unhelpful when you try to extract some information (you see hundred
links to port builders, but you don't know which link is which; again
partially a limitation of 0.8). We initially sent the information from
each and every port builder, but there were some issues with some IDs
of builders being identical, so GitHub did not recognise them as
different builds, and only somehow squashed the results into a single
random link to one ofd the many port builders.


I believe that the following piece of code is an extensive user of the
web hooks:
    https://github.com/macports/mpbot-github
This is a bot written in go (a past GSOC projects from 2017) which
attaches various labels to PRs and takes care for CI builds of ports
on Travis, parsing the logs and publishing summary of build results
from Travis as comments on PRs.


I just want to reiterate one relatively important point: testing the
PRs is going to be somewhat tricky, so I would suggest you to first
make sure that he basic setup without PRs is workng, and the views
work efficiently. We could easily test PRs of base since that's
build-and-throw-away situation.

The PRs for ports would be tricky for various reasons, unless we also
set up disposable VMs (Pierre pointed us to Anka a while ago). If we
don't set up disposable VMs, and if we don't worry about security, we
should at least reverse the effects of building a port. Note that a
port installs itself and all of its dependencies, and then it stays
installed on the machine, as well as pushed the package to the
mirrors. If someone submits a faulty PR updating one port, then that
faulty installation would be kept on the mirror unless you modify the
way ports are built first (uninstalling everything that has been
changed in the PR). But when you also take security into account, it
may become a bit of a nightmare since users can submit just about
anything, get full admin access to the machine, modify a bunch of
stuff, and potentially install malware on every machine using MacPorts
(which updates packages before this gets discovered).

It would definitely be nice to look into this stepwise, but I would
put a much much much higher priority into getting the basics working,
and only seriously look into PRs afterwards (PRs for base might be ok
to a certain extent and much easier to set up, but also not a
particularly high priority compared to efficient views).

Mojca