Buildbot 1.0

[Ryan Schmidt]

On Feb 19, 2018, at 16:49, Mojca Miklavec wrote:

> Are there any thoughts about migration to buildbot 1.0 (from Ryan in
> particular)?
> 
> I wanted to seize the opportunity of some hacking during the MacPorts
> meeting next month to potentially migrate to buildbot version one, but
> there is one quite severe drawback, and that's the display of
> waterfall view (https://github.com/buildbot/buildbot/issues/3884).
> 
> On the other hand we could at the same time fix a number of other
> problems, like:
> - random build order
> - login with OAuth (using GitHub username)
> - ...
> 
> Any thoughts about whether or not to migrate now (= soon) and what
> issues would make most sense to try to address during hacking
> sessions?

With the old macOS forge buildbot setup, I tended to use the console view. With the rewritten buildbot setup that we now use, the console view has become useless because it only shows one row for each portwatcher task, with no way to see the results of each individual portbuilder that was spawned from a portwatcher. Therefore I now use the waterfall view.

Buildbot 0.9's waterfall view was terrible when it first came out. I have not followed buildbot's development since then but as far as I know 1.0 hasn't fixed that problem. We could consider writing a new waterfall frontend that meets our needs better.

To be honest I found the entire buildbot 0.9 web interface terrible, incomprehensible, unrecognizable. I consider "buildbot 0.9" (and later) to be a new piece of software, unrelated to buildbot 0.8. Maybe I would come to understand it over time, but I don't know. Maybe someone should fork and continue the development of buildbot 0.8.

As far as I know there is no reason why we could not fix the random build order problem while staying on buildbot 0.8. We would just need to write the code to add an "order" integer to each spawned portbuilder, then write a function to return the lowest-ordered portbuilder object when deciding which one to start next.

Getting the builds to spawn in the *correct* (dependency) order is a more involved task, but we've survived without it for this long already.

My understanding is that buildbot 0.9 and later would indeed help us log in with GitHub OAuth, which would be nice. I originally thought we had a way to work around that with 0.8, but it sounded like it would make anonymous viewing of the web interface impossible, and I'd rather not prevent that. It is unfortunate that GitHub login support was never added to 0.8. I wonder how hard it would be to add it.

If I were to work on rewriting the buildbot configuration, and I've thought about it, I would focus on merging the portwatcher and portbuilder into a single thing, as I mentioned on the -infra list in October 2017. I also mentioned in that email that I was writing a longer email about the advantages I believe this consolidation will give us, but I never wrote that longer email. I should probably do that to explain what I'm trying to achieve.

If we succeed at merging portwatcher and portbuilder, that might make the console view useful again, so that we might not care about the 0.9+ waterfall view's deficiencies, though there are still all of the other 0.9+ web interface deficiencies to contend with.