GSoC 2019 [Collect build statistics]

[Craig Treleaven]

> 
> On Mar 24, 2019, at 1:09 PM, Mojca Miklavec <mojca at macports.org> wrote:
> 
> On Sun, 24 Mar 2019 at 01:06, Craig Treleaven wrote:
>> 
>> please note that we can’t expect all ports to build successfully on the buildbots.
> 
> Nobody said that, but we cannot blame the student collecting the data
> from buildbot for macports internal deficiencies :)
> 
You thought I was blaming the student?  Merely pointing out something they might not know.

>> There are a number of ports that require a dependency to be installed with a non-default variant in order to build successfully.  A short-coming of MacPorts is that this cannot be done progammatically
> 
> This is a deficiency that could be fixed, but this is outside of scope
> of this project. The project should still report the port as broken,
> consistent to buildbot, it's then up to others to fix MacPorts. The
> work done by Jackson in 2015 was supposed to address that, but it
> turned out to be way more difficult than initially anticipated.
> 
>> When this is rolled out, we don’t want to make users think that a port will fail to build on their system when it is just a case of needing a non-default variant.
>> 
>> However, I don’t know how to handle this cleanly.   Perhaps we could parse the build log looking for the message that informs the user how to install the required variant.  If found, instead of saying the build failed, we could indicate that the build was not attempted as the buildbot configuration could not support a successful install.
> 
> I totally agree with your request, but this is completely out of scope
> of the proposed app. This either needs a proper extension in the base,
> or a workaround in mpbb, preferably the former. I believe a much
> bigger general issue is reporting failure of port builds on OSes which
> are know not to be supported (like: attempting to build the latest Qt
> on 10.5). Again, this needs to be addressed elsewhere.
> 
This issue hits very close to home for me.  None of my MythTV ports, nor the hdhomerun-gui port, will build successfully on the buildbots.  They never have.  They *will* build successfully (on supported OS versions) if the proper variants are specified.

If an unknowing potential user came to page for any of these ports and found nothing but failure messages for all of the buildbots, why on earth would they want to proceed to install the port?

If we won’t expand the scope to handle this relatively common issue, we should at very least add some static text to the web page explaining that buildbot failures don’t mean necessarily mean the port will fail for a particular user.  Even so, that is a very poor workaround.

Failing that, I would modify the description of each of these ports to note the cause of the buildbot ‘failures’ and how it is irrelevant.

I’m not familiar with the 2015 work.  ‘port’ now returns zero for successful completion.  Have we considered having port set a return code that indicates the general class of an unsuccessful operation?  For example, we have 8 port phases defined (fetch through destroot).  We could return -1 through -8 to indicate failure in a particular phase.  More to the point, we could return a specific value for failures such as when active_variants determines that a required variant is not installed.  Similarly, if the port is not supported on a particular OS configuration another specific code could be returned.  I don’t contribute to base but that would seem to be a minimally invasive modification.

Craig