GSoC 2019 [Collect build statistics]

Thu Mar 21 15:04:28 UTC 2019

Hi

For me, this project can be divided into three major parts:

1. ports page:

a) We have seen a quick demo of this already. However, the major part I
think is missing is the search. We can brainstorm over the details like
search-as-you-type, adding new ports etc according to the timeline. Not
sure how much browsing by the first letter helps.

b) You cannot have a class Ports when it represents a Port:
https://github.com/arjunsalyan/MacPorts-Demo-App/blob/master/ports/models.py#L6
.

2. build statistics:

a) In Time Elapsed of builds, it would be incorrect to show time taken for
only one of the build stages. Example, in the case of
https://build.macports.org/builders/ports-10.12_x86_64-builder/builds/87301,
total-time-taken (12 min 59 secs) is right to be shown and not "6 mins 45
secs".

b) There are some things which seem hard-coded to me. I
see '10.14_x86_64', '10.13_x86_64' at multiple places - in port detail
view, build to database view and jinja templates. It's time to define some
constants config file now. For build statuses as well. With a new release
of macOS, we do not want to have to change multiple files in code. In this
project, it is important that a part which works, it is accurate and
complete.

c) Also, as Mojca mentioned, errors like these:
http://frozen-falls-98471.herokuapp.com/ports/database/ should not be
exposed. What is it intended to do anyway?

3. installation statistics:

a) Last year we had a quick attempt at this project (installation
statistics part). It's still lying in my account. Probably the only part
where we spent some time was models.py [2]. You could review it sometime
soon and come back with doubts/suggestions. It almost has the model
implementation on the installed ports statistics with a sample submission
data file present at the root of the project. Class Port, Category, etc can
help with the first part of the project as well.

The tricky part was to figure out (still needs to be figured out) unique
ports and what to keep as the primary key. Similarly, in the case of
maintainers (and look for more such cases).

There are several variations of how maintainers are mentioned in the
Portfiles, example GitHub handles, or emails, or personal websites at
times. You may want to look into the different formats (or ask on the list
separately), and come up with a solution to parse and store.

b) I would suggest keeping all the parsing scripts together at some place
in your project. Keep a document where you have changed the parsing output
(before and after) and also, what do they parse. Naming a script parse.py
doesn't tell much. I am having difficulty tracking down and reviewing all
the scripts.

==

As Mojca said, I am not seeing any way to provide code review on Github
when it's already merged. Since you have the base application ready, it's
time to use PRs. I would also advise starting to follow at least some of
the PEP8 style guide conventions, it's good to follow clean code practices
from the beginning. You can either use coala lint or pylint before pushing
the code, if familiar.

You recently added a .gitignore file but there are .pyc, __pycache__/, etc
files from before. Consider cleaning them up.

These are all the suggestions for later, but only a small portion of what a
proposal is expected to include. Right now, I'd suggest working on the
parsing the maintainers as best as possible. It'd then be good to tackle
the project part by part.

[1]:
https://github.com/umeshksingla/macports-stats/blob/master/firstproject/firstapp/models.py

Best,
Umesh

On Thu, Mar 21, 2019 at 3:57 PM Arjun Salyan <arjun.salyan.che17 at itbhu.ac.in>
wrote:

> On Thu, Mar 21, 2019 at 10:20 AM Mojca Miklavec <mojca at macports.org>
> wrote:
>
>> (2) You made a simple PR last time to fix portindex2json for a more
>> reasonable output of categories. Would you be willing for a tiny bit
>> more difficult task and try to improve the output for maintainers as
>> well? We would want a list of all maintainers with two optional keys
>> for each (email &  github handle) plus a boolean value to tell whether
>> the port is under openmaintainer policy.
>>
>
> Hi, I was working on this. What do I do with "nomaintainer" ? For now I am
> getting the following output.
>
> {
>
>     "variants"         : "clang33 clang34 clang37 clang38 clang39 clang40
> clang50 clang60 clang70 mpich mpich_devel openmpi openmpi_devel python26
> python27 python33 python34 python35 python36 python37 debug no_static
> no_single regex_match_extra universal",
>
>     "subports"         : "boost-numpy",
>
>     "portdir"          : "devel\/boost",
>
>     "description"      : "Collection of portable C++ source libraries",
>
>     "homepage"         : "http:\/\/www.boost.org",
>
>     "epoch"            : "0",
>
>     "platforms"        : "darwin",
>
>     "name"             : "boost",
>
>     "depends_lib"      : "port:zlib port:expat port:bzip2 port:libiconv
> port:icu port:python27",
>
>     *"openmaintainer"   : True,*
>
>     "long_description" : "Boost provides free portable peer-reviewed C++
> libraries. The emphasis is on portable libraries which work well with the
> C++ Standard Library.",
>
>     "license"          : "Boost-1",
>
>     *"maintainers"      : [{*
>
> *        "github" : "https://www.github.com/ryandesign
> <https://www.github.com/ryandesign>"*
>
> *    },{*
>
> *        "github" : "https://www.github.com/michaelld
> <https://www.github.com/michaelld>"*
>
> *    }],*
>
>     "categories"       : [devel],
>
>     "version"          : "1.66.0",
>
>     "revision"         : "3"
>
> }
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macports.org/pipermail/macports-dev/attachments/20190321/30d75e07/attachment.html>