GSoC 2019 [Collect build statistics]

[Mojca Miklavec]

Dear Arjun,

Thanks a lot for the update.

On Thu, 11 Apr 2019 at 16:15, Arjun Salyan wrote:
>
> I have added maintainer views and tables to the demo app.
>
> List of maintainers is clickable on the port-detail page.
> A maintainer-detail view that display info and list of maintained ports

Awesome!

I just noticed that short before you wrote the email as I tried to
find some info on the page :)

> Examples:
> maintainer-detail: https://frozen-falls-98471.herokuapp.com/maintainer/ryandesign__macports.org/

Thank you.

One thing that "urgently" needs to be done is to obfuscate the email
(if written at all). I'm not even sure whether we actually want the
email being displayed there. We need it to send automated emails from
the buildbot in case some failure happens, or to occasionally contact
the maintainer directly. But exposing that info on the website might
be too much.

As far as accessing the information is concerned: at the moment you
use email as unique identifier. I would probably use github handle by
default / as the main entry point. I think that '@' is an allowed
character in URL. If so, we could use
    <url>/maintainer/@ryandesign
to access the same page.

Alternatively we could allow macports handles (for those with
@macports.org email addresses), so
    <url>/maintainer/ryandesign
could work just as well.

Email access could of course be left as it is, but I would not make it default.

I would show the github handle prefixed with the '@' character.

For maintainers with a super long list of ports, or, for
non-maintained ports in particular, we might eventually need a way to
shorten that list (have multiple pages) or provide a similar search
functionality as for global ports, except that here it would be
limited to the ports by that particular maintainer.

I would put that list of ports in a table and add version & short description.

Nice, in any case.

> port-detail: https://frozen-falls-98471.herokuapp.com/ports/faust/

Thanks a lot for adding the links to Trac tickets.

I still need to check the code: what's your current strategy for
showing links to the tickets?
At some point we could differentiate different types of tickets (for
example mark bugs separately).


One minor suggestion. I really like the "search for port" field. Could
this be added to every page? There is "ports" in the top right corner,
but that one is a lot less useful in itself (not saying that it should
go, just suggesting search on each page).

> But, while extracting 'maintainers' from the portindex, maintaining uniqueness was very difficult.

This was totally expected :)

> There are a lot of inconsistencies
> - Same maintainer has provided GitHub details for one port and not for the other.
> - Same maintainer has provided different email for different ports.
>
> I understand that it should be web-app's job to detect this and for now the problem is mostly solved. But in future, one odd case and things can break. What best can be done about this?

Here's what I would do, but feel free to propose an alternative and/or
discuss further. (Actually, I have two slightly different ideas for
implementation in my mind, I'll describe one of them first.)

For the maintainers you could declare a unique keyword over the
combination of github handle + email. Every maintainer of every port
has a uniquely specified pair (github + email) when you import it to
the database (neither github handles not emails would be unique on its
own). Note that you still have index specified on both columns
(separately on each, but the uniqueness is only defined on combination
of the two).

When you read the port, you check the pair (@github, email). If the
pair already exists in the database, you enter the (port, author) pair
into the database of maintainerships. If it doesn't exist, you create
it first, and then assign the maintainership to the port. (Note that
whenever you are updating the port, you also need to check if you need
to remove some maintainers from that port.)

When you display a particular maintainer, say @somerandomgithubhandle,
you run a query and if you hit more that one entry with that github
handle in the database:
- simply display ports from all the entries with that github handle
(you can potentially sort them according to email, or lack thereof, so
that it's easier to figure out which ports need a fix)
- display a red warning saying that there are inconsistencies /
problems with that maintainer which need to be fixed

You could also have a separate page which runs different queries and
looks for all maintainers with inconsistencies (that's for later). It
would generally be helpful to have a collection of such pages for
different things, like: all broken builds on buildbot, all outdated
ports, ...

The other idea would be to keep just one entry per "maintainer", but
then add an additional column with a flag saying that there's
something wrong with that entry. When you would do daily or weekly
updates of the full database, you could clear the flags and set them
again when hitting an inconsistency. But the first idea gives you a
lot more flexibility and sounds more robust in general.

Mojca