Gsoc 18 Project | Collect build statistics

[Vishnu]

Hey

Mojca you said once that you have lot to comment on the current database.
So it would be great if you could do that.
As today is the last day of Community bonding period.
>From tommorrow coding starts.

Thanks

On 12 May 2018 at 23:14, Mojca Miklavec <mojca at macports.org> wrote:

> Dear Vishnu,
>
> Sorry for being slightly out-of-sync (sending emails I wrote quite a
> while ago and in the meantime no longer being up-to-date with other
> responses).
>
> On 12 May 2018 at 17:26, Vishnu wrote:
> > Hi
> >
> > "The PortIndex file itself is generated with the 'portindex' command,
> > which only updates the data for Portfiles that have a modification time
> > that is newer than the existing PortIndex file."
> >
> > So where can i see that code that checks the modification time?
>
> https://github.com/macports/macports-base/blob/master/src/
> port/portindex.tcl
>
> > So suppose
> > Our ports tree contains three ports A, B, C last modified time : 1:00 UTC
> > present port index:
> > [ A
> >   B
> >   C
> > ] @ 1:00 UTC
> >
> > There was a edit in Port A @ 1:30 UTC
> > so something will check all the port files modification time and if
> > relatively newer it will update the port index.
> >
> > so new  port index:
> > [ A
> >   B
> >   C
> > ] @ 1:30 UTC
> >
> >
> > "The portindex2postgres.sql script merely converts the PortIndex file
> > from the custom format based on Tcl lists to SQL. The output will always
> > contain SQL statements with the full data for every port."
> >
> > Then after the port index is updated this script runs and flushes the db
> > then fills it with new portindex data?
> > or just updates the db with the modified port data?
>
> I don't know the answer without looking at it first (you can check
> what the code does yourself), but from the perspective of your app
> development this is irrelevant.
>
> > So after every commit the entire 15mb portindex being processed and data
> > being uploaded to the db.
> > I think this is very inefficient.
> > rather it would be best if after every commit the buildbot/ webhook
> updates
> > the database itself.
>
> I totally agree that it might make a lot more sense to submit just
> differential data, but my suggestion would be to first implement the
> functionality and optimise it at the end if there will be time. No
> user will suffer from suboptimal data transfer (even if that database
> only gets updated once per day, it will still be perfectly
> acceptable), while nobody will benefit from spending a lot of time
> figuring out how to properly implement a super efficient solution and
> then running out of time to actually finish the app.
>
> Let's just work with what we have for now.
>
> Mojca
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macports.org/pipermail/macports-dev/attachments/20180513/c0aaeb16/attachment-0001.html>