GSOC 2019 project: Improvements of Buildbot views (@ MacPorts)

[Mojca Miklavec]

On Thu, 30 May 2019 at 16:43, Rajdeep Bharati wrote:
>
> How do we decide which ports are supposed to be built? Is it only the ports (files) that are affected by a particular `change`?

We fetch a list of modified directories.
Then a bash script from mpbb runs though all of the ports in those
directories and creates a list of candidate ports.
The next step checks whether any of those ports already exist as a
binary package (for that particular version and revision). If it does,
the port is removed from the list.

The remaining ports get built.

> Assume that I made a change to py-django port. Which ports should be built in the following situations:
>
> Version of py-django is updated (no other changes)

We start the builds for the following ports: py-django, py27-django,
py37-django (or whatever other subports are there, I didn't check). If
you only change the version of django 2 (py27-django is kept back at
django 1), then py27-django might be skipped as it already existed
before.

Any of those three builds will also install the dependencies as part
of the build process. Generally the dependencies are already present,
but let's say that one of dependencies was broken before, or that we
just set up a new builder for macOS 10.15, the dependencies would be
built and uploaded to the server (from within the "install
dependencies of pyXY-django" step) and not reported as a separate
build.

> Version of py-django is updated, new dependency added

Same as above.

> Version of py-django is updated, dependencies' ports also updated

This would update more that one folder, so the commit would report
both python/py-django and python/py-foo, and the list would contain
py-django, py27-django, py37-django, py-foo, py27-foo, py36-foo,
py37-foo.
mpbb would first sort the list, so that dependencies would be built
first and shown as standalone builds.

> Does any port depending on py-django also be built in any of these situations?

No.
In the long term it would probably make sense to set up also something
like that, probably just on a single builder, and make sure that no
dependent port breaks or changes, and report in case something like
that happens.

In theory you need to revbump all dependent ports if you expect those
ports to change as a consequence of py-django update, and then all
those ports would be rebuilt, but of course developers forget etc.,
and it would be nice to have some automation to assist with that
process.

Fixing it immediately would be a bit of a too heavy task on one hand,
but it would also require more resources on the other hand. The
powerpc builder can hardly handle the traffic it gets right now.
Adding additional builds would kill it. We would also want to perform
slightly different steps when checking dependent ports.

> Which dependencies are built along with py-django?

All dependencies get installed. Most of them would only get activated
(if present on the worker) or fetched from the mirror (if present
there). If not present, they would be built and at the end uploaded to
the server in the last uploading step.

Mojca