buildbot build time

[Mojca Miklavec]

On 25 October 2017 at 16:23, Ryan Schmidt wrote:
> On Oct 25, 2017, at 03:28, db wrote:
>> On 25 Oct 2017, at 07:35, Mojca Miklavec wrote:
>>> On 23 October 2017 at 10:40, db wrote:
>>>> On 9 Oct 2017, at 12:46, Mojca Miklavec wrote:
>>>>> That info is easy to read and collect from the buildbot. The keyword is: "to collect".
>>>> I checked https://build.macports.org/json/help, but couldn't find it.
>>> For example:
>>> - https://build.macports.org/json/builders/ports-10.13_x86_64-builder/builds/10627?as_text=1
>>> which corresponds to
>>> - https://build.macports.org/builders/ports-10.13_x86_64-builder/builds/10627
>>>
>>> The idea would be to write a script to iterate through all numbers
>>> from 1 on and store results somewhere.
>>>
>>> If you want the time estimate specifically, here might be the relevant
>>> information:
>>>
>>>     "name": "install-port",
>>> ...
>>>     "times": [
>>>       1508901749.648058,
>>>       1508901850.387383
>>>     ]
>>>
>>> I guess the difference represents the seconds spent building the port.
>>
>> Thanks. I was searching for "to collect" there and in base, to no avail.

"To collect" was not meant as a literate strings to search for, but as
the main action that needs to be done.

>> Those are the start/end epochs that I need to subtract, for a single port.
>>
>> How can I correlate portname to buildnumber?

You cannot at the moment.

What I envision is a script that would run once from number 1 on and
then either daily or hourly to update for new builds. The script would
then construct a database of builds and do some simple visualisation.
The most important part would be to ask some simple app to return data
about one particular port. I would expect it to return the list of all
builds (including URLs) for that particular port, but most
importantly, for each buildslave:
- whether the last build succeeded or failed
  - (if failed, whether it was a dependency or the port installation
that failed)
- commit timestamp / shasum & version of the port
- whatever else seems relevant, potentially including the build time
and size of the package

> I don't think scraping information from buildbot json files is the right way to collect this data. If we want to collect it, the buildbot scripts could be modified to do so.

I don't see any big difference. The important part would be to do it
in the first place. How to do it most efficiently can be a subject for
discussion, but parsing json files with a library seems the easiest
approach to me, in particular because we are currently unable to
modify the buildbot script to return us data about packages built in
the past. We can always modify the build scripts later.

> But as I said before, I don't think the information is useful to collect.

Build time doesn't seem *the most* interesting part of information,
but it is useful to know in any case.

> Besides the fact that the buildbot worker machines have different hardware specs than user hardware, not all of the buildbot workers run on the same hardware. A build will go faster on our High Sierra worker than on the Sierra worker, because the High Sierra worker has a faster processor. In addition, the buildbot workers are virtual machines, with multiple VMs running on a single host. So a build may be faster or slower depending on whether the other VMs that run on the same host are busy or not.
>
> If all you want to do is display on a web site somewhere that a particular build of a port took a particular amount of time on our buildbot workers, ok. But I wouldn't try to use this information to, for example, modify MacPorts base to have it offer the user a prediction of how long the port will take to build on their system, because I don't think we can accurately predict that.

I would definitely not use it for that purpose. But having *some
statistics* about which ports built and failed on which builder would
be super super useful. What I miss most is quickly finding the build
where a particular port failed to build. At the moment it's nearly
impossible to find it.

Mojca