Gsoc 18 Project | Collect build statistics

[Mojca Miklavec]

Dear Vishnu,

On Mon, Mar 26, 2018 at 2:58 AM, Vishnu wrote:
>
> Hi,
>
> I wanted some help in build bot summary.
> I am not sure i completely understand what's happening here.
> Could you please explain me about buildbot summary more?
>
> what i understand is :
> There can be n number of builds for each port.
> So how am i supposed to get the information of what all builds have
> already been done for that particular port.Where can i find that history
> from? from build.macports.org ?

Yes, that information can be obtained from build.macports.org.

> First and foremost I think first i have to implement the Buildbot idea
> .That would create logs.
> But for that idea..how to get all the exiting build history of ports?Build
> history such as timestamp,os,Succesful or not?
> Where to get exclusive information about a port.

You cannot get that at the moment.
That's what the whole idea is about: to implement the functionality
that will let developers query a single port.

Take a look at
    https://build.macports.org/builders/ports-10.13_x86_64-builder

You'll see a list of builds. The one currently being built is
    https://build.macports.org/builders/ports-10.13_x86_64-builder/builds/21976
for the port "sumo".

The task is then to go back to the history by decreasing that number
(21976) one by one and fetching information from each individual build
until you reach number 1 or the first build where the information is
no longer available. I believe the build logs get deleted after cca.
10.000 builds and we have 9 builders, so you would end up querying
roughly 100k entries. That could take a couple of days and might need
some coordination with Ryan, but you can start with as little as two
entries per builder.

This information can be queried via JSON interface
(https://build.macports.org/json/help). You can infer OS version from
the build slave name. Timestamp, port name, whether the build was
successful etc. are all available in that JSON API, but there might be
some information missing which you can only get by reading additional
files/logs.

For example,
    https://build.macports.org/builders/ports-10.13_x86_64-builder/builds/21973/steps/install-port/logs/files
will give you the list of files installed by the package. I don't know
if we want to store that particular information or not.
And
    https://build.macports.org/builders/ports-10.13_x86_64-builder/builds/21973/steps/gather-archives/logs/stdio
might give you information about whether the tarball was uploaded or not.

It might be necessary to adapt/improve the buildbot setup to make some
variables more easily accessible. At the moment it's pretty cumbersome
to extract the version information, for example. Assigning the correct
commit that lead to this build would also need some additional work.


I just wanted to say that in the long term it's more important to make
sure that new builds are stored soon after they finish, but it's
probably much easier to start with historical data (and it's also
super useful in the short term), so that you have immediately
something to work with.


The part "A" (buildbot idea) I initially mentioned was meant to
simplify such queries. If we extended the buildbot (configuration),
this might enable querying a single port (making the need for a
separate website like the one in proposal slightly less urgent, but
not irrelevant as we still want installation statistics etc.).

Mojca