Gsoc 18 Project | Collect build statistics

[Vishnu]

Hi

Also i wanted to know
For this link
https://www.macports.org/ports.php?by=name&substr=python27

There is already existing database with loads of information.
Is it not updated?
is  it static ?

If it is updated automatically...Then half of the work for database would
already be over.

Could i get access to the DB ?
I got the rough idea of the db from here:
https://github.com/macports/macports-infrastructure/blob/2129f0cd0eb80f207d2cc62542b65c197733ac51/jobs/PortIndex2PGSQL.tcl#L249

So is this updated regularly?How?

And there seems to be lot of data already existing. Which i could just copy.
Existing maintainer table,many other.

Thanks


On 11 May 2018 at 07:11, Vishnu <vishnum1998 at gmail.com> wrote:

> Hi
>
> So should I start working on the same database?
>
> Community does not have any suggestions I guess.
> So should i go ahead with the existing structure?
>
> I had a doubt regarding version.
>
> How can i check the existing versions of any port.
> Say python27?
>
> Any list of maintainers i can get?
>
> Thanks
>
> On 9 May 2018 at 04:06, Vishnu <vishnum1998 at gmail.com> wrote:
>
>> Hi
>>
>> And also i couldn't figure out any way to hide passwords/ Sensitive
>> information while creating app.
>>
>> Thanks
>>
>> On 9 May 2018 at 03:43, Vishnu <vishnum1998 at gmail.com> wrote:
>>
>>> Hi
>>>
>>> I had one doubt.
>>> Should i switch the link in heroku account for integration with macports
>>> github ?
>>>
>>> Because i think then you need to give accesss to heroku of your account.
>>>
>>> I think it would be wise for me to do the commit update in my local
>>> repository itself..
>>>
>>> Once every 2 weeks or something ill push all the changes to macports
>>> repository.
>>> Do comment .What should be done?
>>>
>>> Thanks
>>>
>>> On 8 May 2018 at 11:40, Jackson Isaac <ijackson at macports.org> wrote:
>>>
>>>> On Tue, May 8, 2018 at 10:47 AM, Mojca Miklavec <mojca at macports.org>
>>>> wrote:
>>>> >
>>>> > On 8 May 2018 at 01:12, Vishnu wrote:
>>>> > > Hi
>>>> > >
>>>> > > I will work on all the points mentioned today..
>>>> > >
>>>> > > But i have been trying many different ways.
>>>> > > To get 20k ports onto the database.. The site is crashing.. Timeout
>>>> error.
>>>> > > Or the site cant be reached.
>>>> > > Maximum i got till 9k ports successfully entered.
>>>> > >
>>>> > > Tried many ways..
>>>> > >
>>>> > > If you have any suggestions do let me know.
>>>> >
>>>>
>>>> We can even try firebase for the DB backend and host the frontend on
>>>> heroku.
>>>>
>>>> Firebase free version gives 1GB of storage and 10GB of bandwidth per
>>>> month.
>>>> I guess for testing out the app on complete portindex we could give it
>>>> a try.
>>>>
>>>> https://firebase.google.com/pricing/
>>>>
>>>> Then again we would have some kind of limitations over here in long run.
>>>>
>>>> >
>>>> > I was pretty sure that we would reach the database limit here.
>>>> >
>>>> >
>>>> > This site
>>>> >     https://devcenter.heroku.com/articles/heroku-postgres-plans
>>>> > says that the limit is 10k rows which is not anywhere near enough and
>>>> > fully consistent with your observation. The plan that lets you have
>>>> > 10M rows is 9 USD per month which I gladly pay if that would solve the
>>>> > issues (I'm not sure if those are the only ones, we might not have
>>>> > sufficient memory etc.)
>>>> >
>>>>
>>>> 9USD is only for the DB though. We might need to pay for the compute
>>>> separately.
>>>>
>>>> Looks like free VMs come with only 512MB RAM.
>>>>
>>>> >
>>>> > (I also remembered that I might have access to create a clean virtual
>>>> > machine, but I need a couple of days to physically reach a place where
>>>> > I have the additional info. That would require setting up the whole
>>>> > machine manually of course, but we need to do that for the final
>>>> > deployment anyway.)
>>>> >
>>>> > > I think i have to break the data entry in separate chunks and do
>>>> it. But not
>>>> > > sure how to do that.
>>>> >
>>>> > In any case Heroku will apparently not let you import the full
>>>> > database anyway until we switch to a payed account. If you still have
>>>> > troubles with performance issues afterwards, this also won't help.
>>>> > What you could do is temporarily copy or move a few folders (say,
>>>> > math, science and python) and run portindex just on those. This will
>>>> > give you a smaller number of ports to work with, but still sufficient
>>>> > to figure out what other issues you'll need to deal with (port listing
>>>> > will have to be paginated etc.)
>>>> >
>>>> > It's a good exercise to know what the limitations are. When you play
>>>> > with new features and database design, it's always easier to play
>>>> > with, say 10-100 ports than with full 15 MB portindex file.
>>>> >
>>>>
>>>> This plan sounds good. +1
>>>>
>>>> --
>>>> Jackson Isaac
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macports.org/pipermail/macports-dev/attachments/20180511/b87b9a28/attachment-0001.html>