Buildbot proposal: add custom build properties for order and priority

[Ryan Schmidt]

We currently build ports in a somewhat random order (see https://trac.macports.org/ticket/52766). By default, the next build that buildbot starts is the one that was scheduled first. But if multiple builds happened to get scheduled at the same exact time, the order in which they will build is undefined.

We can fix this by adding a custom "order" property to each build, and modifying our nextBuild function (mentioned in my previous email) so that it looks not only at the scheduled timestamp but also the order property to decide which build to pick.

I don't know whether we currently compute the *correct* order (dependencies should build first) but if not that can be tackled later. A predictable incorrect order is better than a random incorrect order.


I'd also like to be able to assign priorities to builds. Low, Medium, High would be enough. We could add a custom "priority" property to each build for this.

Use case for High priority: suppose hundreds of builds are pending, and you see that a build of a library that lots of ports use is broken, let's say libnetpbm. You want to be able to fix libnetpbm and force it to build right away, before the pending builds go through, because you know many of those pending builds would fail without libnetpbm. Fixing libnetpbm might not even require a commit: libnetpbm might have been marked as failed because of a fetch failure--due to the recent SourceForge outages, for example--and the server issue that caused the fetch failure might have been resolved in the mean time.

Use case for Low priority: suppose we set up a new builder for a new macOS version. We want to eventually build all ports, but it's more important that ports that have just had updates committed are built first. So we would force a build of all ports with Low priority, and while the builder is otherwise idle, it works its way through building all ports, but when a commit comes in, it switches over to building that.

We may not even need to allow the user to specify a priority; we could possibly infer it. For example, if a large number of ports (more than 100, say) are scheduled by a single commit or force, we might infer that it is of low priority. It might be a mass update commit of maintainer lines or checksum types or removing $Id$ lines or adjusting whitespace, and we don't want those builds to block actual port updates. And if a build of a few ports is forced, we might infer that it is of high priority.