GSOC 2019 project: Improvements of Buildbot views (@ MacPorts)

Wed May 29 18:29:25 UTC 2019

(CC-ing Arjun as parts of this might be of interest for the app as well,
and the general mailing list as they might provide further feedback.)

On Wed, 29 May 2019 at 08:57, Rajdeep Bharati <rajdeepbharati13 at gmail.com>
wrote:

> Hi,
> I wanted to be sure about one thing:
> In view #2
>  Screenshot 2019-05-29 at 12.21.13 PM.png
> <https://drive.google.com/file/d/118N91qeOPFXpBHuQs5Bo085uKuOF0i4_/view?usp=drive_web>
> do you want me to put information about
> 1) only the builds
> 2) builds as well as commits
> I assume that the table (as shown in the screenshot) would have info of
> the
> builds and their results on every port and OS version.
>

For anyone reading the mailing list, we are talking about
    https://trac.macports.org/ticket/55978

I would primarily like to see the commits (as well as potentially forced
builds, but those with a somewhat lower priority; forced builds could be
discussed later; as I understand one cannot even trigger builds at the same
time on multiple builders without modifying configuration).

The builds by themselves are hardly useful. My primary goal for this is as
follows. Let's say that I know that I changed the python37 port in commit
X. Two months later I want to know on which workers the build failed and on
which ones it succeeded. With the existing build.macports.org this info is
literally impossible to find unless you write a bot that fetches json data
from the builds as far back as you can get (maybe doing some bisection or
so). And once you find the relevant build 13584 for Mac OS X 10.6, you need
to repeat the exercise for 10.7, 10.8, ... 10.14 all from scratch.

With the new buildbot the console view partially does what I would like to
see. Take a look at
    https://nine.buildbot.net/#/console
If you ignore the bug with "unknow revision x-y" and scroll down to
something like a pull request or an actual commit, you can extend the view
and get something like this:

[image: image.png]

We would like to be able to browse commits (that's why we talked about
"calendar widget"), click on a commit of interest and get to a page with
information for that commit (group) only. Mostly because there will be so
much information that a table like this might be simply too crowdy, but
also because it would be nice to have a link that can be shared, I'm not
sure if you can share a link to the screenshot I sent above. Having a
single page is not a strict requirement though. With some folding it could
be ok to have it implemented in a similar way as on the above screenshot.

The screenshot above already contains a lot of information that we want to
have:

   - author name & timestamp are already there
   - (committer name & timestamp are already on your short-term TODO list)
   - commit message is already there
   - link to github is already there
   - (link to trac tickets could be extracted from commit message; it's
   already in URL form if present, it's just a matter of making it active,
   i.e. adding "a href" around it)
   - (link to the PR that resulted in that commit might need to be added;
   or maybe it's there already, but I cannot see it on this view anyway)
   - (I forgot to mention, but probably the list of files modified would be
   useful as well, and maybe it's there already, at least it says "No files"
   in the view above.)

The console also shows a nice view of the results across different builders.

What's special about the way ports are built for MacPorts is that one
commit (or rather: one build potentially encompassing multiple subsequent
commits) triggers the builds of an arbitrary number of ports. And the
information about whether that build in total (one single job on port
watcher) passed or not is useless if it is not split into multiple ports.
We want to make some kind of equivant of the above console view, but the
view needs to contain a table of all the ports that were built as part of
that "port watcher" job or "port builders".

As an intermezzo I want to explain one thing. Initially we had one single
job (which could run for days), and it built all the ports in a single job.
However one could not infer any information about what worked and what
failed without downloading logs first, which could have reached hundreds of
megabytes each. We wanted to rewrite it with a custom number of steps, so
that a build of 100 ports would contain 100 steps (probably more like a few
times hundred; with "clean", "build dependencies", "build port", "upload
port", ... for each of the ports individually). The reason for creating
port watcher and port builder as separate entities was used as a workaround
for lack of functionality in buildbot 0.8, which is no longer the case in
the latest buildbot. So maybe it's even time to revise this decision and
figure out if there's a better way to organize port watcher/builder into a
single builder.

One problem with current setup is that it may happen that one presses the
"rebuild" button and starts building a port on the portbuilder at the same
time as portindex is being regenerated, or macports is being updated in the
portwatcher. These two actions should not happen at the same time. Another
problem is that, say, a perl version increase may result in rebuilding a
few thousand ports. Buildbot 0.8 usually choked and became uselessly slow
(with builders idling for as long as an hour just to get the next job
assigned). Before that rewrite, when everything ran as a single job, we
would sometimes schedule builds of all ports. With current setup it's more
of a mission impossible to imagine that. I have no idea how buildbot 2.x
can handle builds with tens of thousands of steps.

If you look at a random build in current setup, like

https://build.macports.org/builders/ports-10.12_x86_64-builder/builds/92478
for example, it tells you which commit(s) it belongs to, but you cannot ask
the buildbot to show you all builds belonging to those commits. Also, if
you then go to the portwatcher, say,

https://build.macports.org/builders/ports-10.12_x86_64-watcher/builds/23943
you don't even know on link in the list to find a particular port.

Mojca
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macports.org/pipermail/macports-dev/attachments/20190529/1a9aef42/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 49586 bytes
Desc: not available
URL: <http://lists.macports.org/pipermail/macports-dev/attachments/20190529/1a9aef42/attachment-0001.png>