Inconsistent criteria of success on the buildbot

[Ryan Schmidt]

> On Feb 25, 2016, at 5:50 AM, Mojca Miklavec <mojca at macports.org> wrote:
> 
> On 25 February 2016 at 11:22, Joshua Root wrote:
>> On 2016-2-25 18:18 , Mojca Miklavec wrote:
>>> 
>>> Hi,
>>> 
>>> A while ago I committed a new subport for geant4.10.2 which is
>>> currently broken (in the meantime I lacked time/motivation to look
>>> into it; and nobody else complained so far either):
>>>     https://trac.macports.org/ticket/50426
>>> 
>>> The initial builds failed, see for example:
>>> 
>>> https://build.macports.org/builders/buildports-mtln-x86_64/builds/27442
>>> but judging from
>>>     http://packages.macports.org/geant4.10.2/
>>> it seems as if the "build all ports" picked up and uploaded the
>>> "broken" package with "no questions asked".
>>> 
>>> Shouldn't this be fixed somehow? (Not just the port, but also the
>>> behaviour.) Or did the package automagically start to work properly?
>>> 
>>> I don't dare to even start parsing the logs from the all-ports build.
>>> The progress log from the same buildslave
>>> 
>>> https://build.macports.org/builders/buildports-mtln-x86_64/builds/28037/steps/compile/logs/progress
>>> says "package found, not building again".
>> 
>> That happens if installation succeeded but activation failed, or rev-upgrade
>> failed afterwards. What behaviour are you wanting?
> 
> Good question.
> 
> I would expect:
> - from the buildbot to send me an email about the failure (so that I
> know that I have to look at it and fix it)

The buildbot does send emails on build failures, to all the maintainers of the ports that were built.

> - consistency in uploading: either an upload or no upload of the
> package in both build attempts
> 
> But I do realize that implementing the second might be tricky and
> might add a lot of overhead to the process.

That does sound like a reasonable request. I have not read through the code to see how the current buildbot configuration is implemented or how we might achieve this. But it seems as though the buildbot should just be told to upload a package if it finds it, regardless of whether activation succeeded or not. That would at least be consistent. I would also like for packages to be uploaded immediately after they're created, and not in a separate phase after all ports are built. That would help packages get into users' hands quicker.


>> For it to uninstall the
>> port in this case?
>> 
>> I agree it's not really ideal, but it does speed things up for users as
>> archives are intended to do, even though in this case what they end up with
>> is broken either way. And you'll need a version or revision bump to fix it,
>> so uninstalling and attempting the build again in future doesn't really
>> help.
> 
> The only thing that comes to my mind would be to flag a package when
> it fails to build for one reason or another

The buildbot does flag ports as failed or succeeded, and skips them on subsequent builds if they haven't changed since the prior success or failure. If failure, buildbot also skips ports that depend on the failed port.

> In case a port is
> flagged, the buildbot should try to at least activate the port and run
> some sanity checks (if not rebuilding the port completely) to make
> sure that the port is flawless.

The buildbot should not behave differently from user systems, since the buildbot is meant to be our "ideal user". If a port installs but fails to activate, a revbump is needed to correct the problem on user systems, so it is reasonable to require that to correct the problem on the buildbot as well.

> On the other hand a package might become broken after a while if one
> of its dependencies gets upgraded without revbumping the package. So
> the situation is not much better in that case.

A revbump is required to fix broken packages too, so we needn't consider that scenario for the buildbot.