GSoC 2019 [Collect build statistics]

[Mojca Miklavec]

On Mon, 18 Mar 2019 at 16:52, Arjun Salyan wrote:
>
> Some improvements to the Demo App: https://frozen-falls-98471.herokuapp.com

Thank you very much.

> All Ports and All Categories are now available (Although not all ports have populated yet, I am on AWS Free Tier and the process is really slow. At the time of drafting this email: around 500 have populated).

I can imagine that the free plan would not be the fastest one in the
world, but to me 500 entries in what I could imagine might be an hour
since you started the job sounds like potential efficiency problem. A
forgotten index in a table can easily increase the runtime
polynomially or even exponentially.

> On the Port-Detail page, the categories are now clickable and lead to the list of ports under that category.

Cool!

What I would be potentially missing there is a number of ports under
that category. When I clicked around, several categories were empty.
You just need to make sure that all the relevant columns in the
database are indexed to make this efficient enough.

> I was able to parse the entire PortIndex.json using a python script and successfully converted it to Django fixtures which could then be populated to the database.

That's very good news. What exactly did you have to change to make it
work compared to last time?

> (I used the portindex.json outputted by current version of portindex2json.tcl and fixed the issues with categories using same python script)
>
> Parse.py : https://github.com/arjunsalyan/MacPorts-Demo-App/blob/master/MacPorts/parse.py

Thank you, I will look into it and provide further feedback.

Mojca