[GSoC 2019] Web App Project Update

[Mojca Miklavec]

Dear Arjun,

Thanks a lot for sending out the report.

Dear MacPorts users,

I would warmly welcome everyone to check what has been done so far,
provide feedback, but in particular volunteer to run
    sudo port install mpstats-gsoc
to help us get more statistics ready by the time Arjun starts working
on that part (about two weeks from now; it would be really cool to
have more than one month worth of statistics from a sufficient number
of users well before the GSOC ends to be able to know whether we'll be
heading in the right direction).

The detailed statistics is not there yet, but you can probably guess
whether your statistics submission has been successful by checking the
following page:
    https://frozen-falls-98471.herokuapp.com/statistics/
right before and after you install the port. You can also manually run
    /opt/local/libexec/mpstats-gsoc submit
or
    /opt/local/libexec/mpstats-gsoc show
if you just want to see what gets submitted.

Thank you very much,
    Mojca

(I CC-ed macports-users to get some more visibility and hopefully get
more volunteers to submit their statistics, but any further discussion
probably only makes sense on macports-dev. Even though ... one of the
student contributors complained that we didn't post anything about
GSOC to the macports-announce ...)

On Fri, 7 Jun 2019 at 22:21, Arjun Salyan wrote:
>
> Hi,
>
> I am working on the web app to collect build and installation statistics and show information about ports, maintainers etc.
>
> Repository: https://github.com/macports/macports-webapp
>
> We have deployed a demo on Heroku: https://frozen-falls-98471.herokuapp.com . Any feedback and feature requests on this are very welcome.
>
> Currently we are working on build history and port information. Port information is fetched from Portindex after converting it to JSON using portindex2json.tcl and build history is fetched from JSON API of buildbot, it is planned to be kept updated using HttPStatusPush reporter on Buildbot.
>
> Keeping the port informtation up-to-date is a challenge given the large size of the portindex. We are looking at two aspects:
> - Incremental updates using difference between two port indices [1]
> - Full updates to completely sync the database with latest portindex
>
> Full updates are dead slow on an Amazon RDS free tier instance, and probably it won't be feasible on any externally hosted database. On a local database, full updates take more than 5 minutes.
>
> For installation statistics, we have setup a temporary port (mpstats-gsoc) which would submit stats to the demo app. This is important becuase it would give us enough data to write proper views and do testing. Please consider installing mpstats-gsoc.
>
> We are looking to deploy the app on MacPorts' server and host it on a macports.org subdomain using Docker.
>
> Thank you for your time, any suggestions would be very helpful.
>
> [1] https://github.com/macports/macports-base/pull/121
>
> Arjun