macports-dev Digest, Vol 151, Issue 25
Rajdeep Bharati
rajdeepbharati13 at gmail.com
Tue Mar 19 12:05:16 UTC 2019
Hi Pierre,
I was trying to set up buildbot using a python3.6 venv, as shown here
<https://github.com/uglycoyote/buildbot-react-plugin-boilerplate>.
When I run `make frontend` I'm getting this error:
error: cannot copy tree 'buildbot_www/static': not a directory
Command "python setup.py egg_info" failed with error code 1 in
/Users/rajdeep/trial/buildbot/www/base/
On Tue, Mar 19, 2019 at 3:59 PM <macports-dev-request at lists.macports.org>
wrote:
> Send macports-dev mailing list submissions to
> macports-dev at lists.macports.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.macports.org/mailman/listinfo/macports-dev
> or, via email, send a message with subject or body 'help' to
> macports-dev-request at lists.macports.org
>
> You can reach the person managing the list at
> macports-dev-owner at lists.macports.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of macports-dev digest..."
>
>
> Today's Topics:
>
> 1. Re: GSoC 2019 [Collect build statistics] (Arjun Salyan)
> 2. Re: GSoC 2019 [Collect build statistics] (Mojca Miklavec)
> 3. Re: GSoC 2019 [Collect build statistics] (Mojca Miklavec)
> 4. Re: GSoC 2019 [Collect build statistics] (Arjun Salyan)
> 5. Re: GSoC 2019 [Buildbot ideas] (Pierre Tardy)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 18 Mar 2019 21:22:06 +0530
> From: Arjun Salyan <arjun.salyan.che17 at itbhu.ac.in>
> To: Mojca Miklavec <mojca at macports.org>
> Cc: MacPorts Development <macports-dev at lists.macports.org>
> Subject: Re: GSoC 2019 [Collect build statistics]
> Message-ID:
> <
> CAHToqhg-s-vBhMVqH-4D3q5fpNmM5NBH68ozveKAtKfWJiHxMw at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Some improvements to the Demo App:
> https://frozen-falls-98471.herokuapp.com
>
> - All Ports and All Categories are now available (Although not all ports
> have populated yet, I am on AWS Free Tier and the process is really
> slow.
> At the time of drafting this email: around 500 have populated).
> - On the Port-Detail page, the categories are now clickable and lead to
> the list of ports under that category.
>
> I was able to parse the entire PortIndex.json using a python script and
> successfully converted it to Django fixtures which could then be populated
> to the database. (I used the portindex.json outputted by current version of
> portindex2json.tcl and fixed the issues with categories using same python
> script)
>
> Parse.py :
>
> https://github.com/arjunsalyan/MacPorts-Demo-App/blob/master/MacPorts/parse.py
>
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.macports.org/pipermail/macports-dev/attachments/20190318/faa62877/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 2
> Date: Mon, 18 Mar 2019 17:37:30 +0100
> From: Mojca Miklavec <mojca at macports.org>
> To: Arjun Salyan <arjun.salyan.che17 at itbhu.ac.in>
> Cc: MacPorts Development <macports-dev at lists.macports.org>
> Subject: Re: GSoC 2019 [Collect build statistics]
> Message-ID:
> <
> CALBOmsZNtSJZC2OnJJRe8OFNsMZ2_vvmDCypAwOMa6ffnCRj0Q at mail.gmail.com>
> Content-Type: text/plain; charset="UTF-8"
>
> On Mon, 18 Mar 2019 at 16:52, Arjun Salyan wrote:
> >
> > Some improvements to the Demo App:
> https://frozen-falls-98471.herokuapp.com
>
> Thank you very much.
>
> > All Ports and All Categories are now available (Although not all ports
> have populated yet, I am on AWS Free Tier and the process is really slow.
> At the time of drafting this email: around 500 have populated).
>
> I can imagine that the free plan would not be the fastest one in the
> world, but to me 500 entries in what I could imagine might be an hour
> since you started the job sounds like potential efficiency problem. A
> forgotten index in a table can easily increase the runtime
> polynomially or even exponentially.
>
> > On the Port-Detail page, the categories are now clickable and lead to
> the list of ports under that category.
>
> Cool!
>
> What I would be potentially missing there is a number of ports under
> that category. When I clicked around, several categories were empty.
> You just need to make sure that all the relevant columns in the
> database are indexed to make this efficient enough.
>
> > I was able to parse the entire PortIndex.json using a python script and
> successfully converted it to Django fixtures which could then be populated
> to the database.
>
> That's very good news. What exactly did you have to change to make it
> work compared to last time?
>
> > (I used the portindex.json outputted by current version of
> portindex2json.tcl and fixed the issues with categories using same python
> script)
> >
> > Parse.py :
> https://github.com/arjunsalyan/MacPorts-Demo-App/blob/master/MacPorts/parse.py
>
> Thank you, I will look into it and provide further feedback.
>
> Mojca
>
>
> ------------------------------
>
> Message: 3
> Date: Mon, 18 Mar 2019 18:19:27 +0100
> From: Mojca Miklavec <mojca at macports.org>
> To: Arjun Salyan <arjun.salyan.che17 at itbhu.ac.in>
> Cc: MacPorts Development <macports-dev at lists.macports.org>
> Subject: Re: GSoC 2019 [Collect build statistics]
> Message-ID:
> <
> CALBOmsYgpDYDM9N45UE4czYavg8DZcuvXXLaAz6K0Jw0xGh2ug at mail.gmail.com>
> Content-Type: text/plain; charset="UTF-8"
>
> On Mon, 18 Mar 2019 at 17:37, Mojca Miklavec wrote:
> > On Mon, 18 Mar 2019 at 16:52, Arjun Salyan wrote:
> >
> > > All Ports and All Categories are now available (Although not all ports
> have populated yet, I am on AWS Free Tier and the process is really slow.
> At the time of drafting this email: around 500 have populated).
> >
> > I can imagine that the free plan would not be the fastest one in the
> > world, but to me 500 entries in what I could imagine might be an hour
> > since you started the job sounds like potential efficiency problem. A
> > forgotten index in a table can easily increase the runtime
> > polynomially or even exponentially.
>
> And in fact I'm unable to find any indices in your DB model.
>
> https://docs.djangoproject.com/en/2.1/topics/db/optimization/
> https://www.djangorocks.com/snippets/indexing-your-django-models.html
>
> Also, TextField might be suitable for description etc, but for short
> entries like port name, this probably offers suboptimal performance
> and CharField would make more sense. I did not time it though, and
> this is not the bottleneck in your code, but the indices are
> definitely critical for perfomance.
>
> Mojca
>
>
> ------------------------------
>
> Message: 4
> Date: Mon, 18 Mar 2019 23:21:56 +0530
> From: Arjun Salyan <arjun.salyan.che17 at itbhu.ac.in>
> To: Mojca Miklavec <mojca at macports.org>
> Cc: MacPorts Development <macports-dev at lists.macports.org>
> Subject: Re: GSoC 2019 [Collect build statistics]
> Message-ID:
> <CAHToqhjWNwYkrobZ+4Ti0Z=8C9kuQ=
> y6sUHG89+1zQkNeSrzPw at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> On Mon, 18 Mar 2019 at 10:49 PM, Mojca Miklavec <mojca at macports.org>
> wrote:
>
> > And in fact I'm unable to find any indices in your DB model.
>
>
> Thanks, I shall add this. I am dealing with this huge data set for the
> first time.
>
> Also, TextField might be suitable for description etc, but for short
> > entries like port name, this probably offers suboptimal performance
> > and CharField would make more sense. I did not time it though, and
> > this is not the bottleneck in your code, but the indices are
> > definitely critical for perfomance.
>
>
> Yes, I have finalised the ports table now and hence, I shall change the
> field types accordingly as to which one is the most suitable for the data
> type in that column.
>
> Also, I have terminated the process of populating the database- my internet
> today and the free tier both are making it really difficult. I was able to
> load the entire database within seconds locally.
>
> Thank You
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.macports.org/pipermail/macports-dev/attachments/20190318/740f60ae/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 5
> Date: Tue, 19 Mar 2019 11:29:09 +0100
> From: Pierre Tardy <tardyp at gmail.com>
> To: Mojca Miklavec <mojca at macports.org>
> Cc: MacPorts Development <macports-dev at lists.macports.org>, Rajdeep
> Bharati <rajdeepbharati13 at gmail.com>
> Subject: Re: GSoC 2019 [Buildbot ideas]
> Message-ID:
> <CAJ+soVefa_Uver+uuKC20xR_HgJ=
> FongoQ9jTvUzLgKqLux1qg at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Rajdeep,
>
> Sorry if I have not been clear.
> The project I have linked is quite a large project which will actually
> takes several summer in order to finish. What I meant is that it would be
> good for synergy if we have another student on that project, as this is on
> the same domain, and you could work with him/her to share your findings and
> best practices.
>
> You can start working on macport custom buildbot UI without waiting for
> this project, and we can start working on that in modern javascript of
> typescript if needed.
>
> uglycoyte did a poc which you can find here:
> https://github.com/uglycoyote/buildbot-react-plugin-boilerplate
>
> I am not sure of the state of the PoC.
> I suggest you as a welcome challenge to build yourself a buildbot
> environment, with the react plugin installed in, displaying "hello from
> Rajdeep's Buildbot"
>
> Would that work for you?
>
> Regards
> Pierre
> On Mon, Mar 18, 2019 at 10:53 AM Pierre Tardy <tardyp at gmail.com> wrote:
>
> > Hi,
> >
> > We have our own GSoC project to transition the UI from coffeescript:
> >
> >
> https://github.com/buildbot/buildbot/wiki/Buildbot-GSoC-Projects-2019#transition-web-uis-data-module-from-coffeescript-to-typescript
> > The ideal would be to have one student for each project.
> >
> > All the pointers are there. I would be happy to help on either project.
> >
> > Regards,
> > Pierre
> >
> > On Mon, Mar 18, 2019 at 9:41 AM Mojca Miklavec <mojca at macports.org>
> wrote:
> >
> >> Dear Rajdeep,
> >>
> >> On Sun, 17 Mar 2019 at 19:55, Rajdeep Bharat wrote:
> >> >
> >> > Hello, I am Rajdeep, a GSoC aspirant.
> >> > I was going through the Buildbot views project and had a few doubts:
> >> > - Do I need to work on the dashboards SPA of buildbots (
> >> https://github.com/buildbot/guanlecoja-ui) and rewrite it using a
> modern
> >> library like React?
> >> > - I have read this ticket: https://trac.macports.org/ticket/55978. Do
> >> I have to implement them separately in a MacPorts repository of
> buildbot,
> >> or do it in upstream directly?
> >> >
> >> > I am familiarizing myself with the buildbot codebase and would love to
> >> start contributing.
> >> > Can someone tell me if I'm on the right track?
> >>
> >> Thank you very much for reaching to us. I still owe you the response
> >> to the earlier email, but let's start with this one.
> >>
> >> The "buildbot idea" would be (co)mentored by someone from the Buildbot
> >> development team for the technical part ("How to do it?"), while the
> >> MacPorts mentor would help to steer towards the desired goal ("What to
> >> do?").
> >>
> >> So I would like to ask Pierre to explain the technical part in more
> >> details.
> >>
> >> (Pierre, how does the unfinished Coffeescript to Typescript conversion
> >> of the core affect designing new views and their "future-proof-ness"?)
> >>
> >> What I want to say from the MacPorts point of view is that what mainly
> >> kept us from switching to buildbot version beyond 0.8.x was a less
> >> user-friendly waterfall view. We kept postponing the switch since the
> >> time the buildbot was still at version 0.9.x. Now it's already at 2.x,
> >> and it would be about a high time that we do the upgrade. The existing
> >> buildbot setup was written in cca. 3 days during our in-person
> >> developer meeting in 2016, and then slightly improved over time.
> >>
> >> The changes would be added wherever it makes the most sense. If
> >> there's some view that's specific to MacPorts, we would keep it
> >> locally, while any other change that's considered useful for any
> >> buildbot user would best be pushed to their own repository (and
> >> reviewed by the buildbot development team).
> >>
> >> I wanted to add that this project is well related to the "collection
> >> of build statistics" idea. I will elaborate on this later, but we
> >> definitely need changes in the buildbot configuration to make it
> >> easier for our (hopefully) future web application to talk to the
> >> buildbot master.
> >>
> >> Mojca
> >>
> >>
> >> Mojca
> >>
> > --
> >
> >
> > --
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.macports.org/pipermail/macports-dev/attachments/20190319/247fcfb9/attachment.html
> >
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> macports-dev mailing list
> macports-dev at lists.macports.org
> https://lists.macports.org/mailman/listinfo/macports-dev
>
> ------------------------------
>
> End of macports-dev Digest, Vol 151, Issue 25
> *********************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macports.org/pipermail/macports-dev/attachments/20190319/63fcc623/attachment-0001.html>
More information about the macports-dev
mailing list