GSoC 2019 [Collect build statistics]

[Mojca Miklavec]

On Mon, 18 Mar 2019 at 17:37, Mojca Miklavec wrote:
> On Mon, 18 Mar 2019 at 16:52, Arjun Salyan wrote:
>
> > All Ports and All Categories are now available (Although not all ports have populated yet, I am on AWS Free Tier and the process is really slow. At the time of drafting this email: around 500 have populated).
>
> I can imagine that the free plan would not be the fastest one in the
> world, but to me 500 entries in what I could imagine might be an hour
> since you started the job sounds like potential efficiency problem. A
> forgotten index in a table can easily increase the runtime
> polynomially or even exponentially.

And in fact I'm unable to find any indices in your DB model.

https://docs.djangoproject.com/en/2.1/topics/db/optimization/
https://www.djangorocks.com/snippets/indexing-your-django-models.html

Also, TextField might be suitable for description etc, but for short
entries like port name, this probably offers suboptimal performance
and CharField would make more sense. I did not time it though, and
this is not the bottleneck in your code, but the indices are
definitely critical for perfomance.

Mojca