<div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Hi Pierre,</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">I was trying to set up buildbot using a python3.6 venv, as shown <a href="https://github.com/uglycoyote/buildbot-react-plugin-boilerplate">here</a>.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">When I run `make frontend` I'm getting this error: </div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:15px;line-height:normal;font-family:Monaco;color:rgb(33,255,6);background-color:rgba(0,0,0,0.75)"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>error: cannot copy tree 'buildbot_www/static': not a directory</span></p></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:15px;line-height:normal;font-family:Monaco;color:rgb(180,36,25);background-color:rgba(0,0,0,0.75)"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">Command "python setup.py egg_info" failed with error code 1 in /Users/rajdeep/trial/buildbot/www/base/</span></p></div></div><div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Mar 19, 2019 at 3:59 PM <<a href="mailto:macports-dev-request@lists.macports.org">macports-dev-request@lists.macports.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Send macports-dev mailing list submissions to<br> <a href="mailto:macports-dev@lists.macports.org" target="_blank">macports-dev@lists.macports.org</a><br>
<br>To subscribe or unsubscribe via the World Wide Web, visit<br> <a href="https://lists.macports.org/mailman/listinfo/macports-dev" rel="noreferrer" target="_blank">https://lists.macports.org/mailman/listinfo/macports-dev</a><br>or, via email, send a message with subject or body 'help' to<br> <a href="mailto:macports-dev-request@lists.macports.org" target="_blank">macports-dev-request@lists.macports.org</a><br>
<br>You can reach the person managing the list at<br> <a href="mailto:macports-dev-owner@lists.macports.org" target="_blank">macports-dev-owner@lists.macports.org</a><br>
<br>When replying, please edit your Subject line so it is more specific<br>than "Re: Contents of macports-dev digest..."<br>
<br>
<br>Today's Topics:<br>
<br> 1. Re: GSoC 2019 [Collect build statistics] (Arjun Salyan)<br> 2. Re: GSoC 2019 [Collect build statistics] (Mojca Miklavec)<br> 3. Re: GSoC 2019 [Collect build statistics] (Mojca Miklavec)<br> 4. Re: GSoC 2019 [Collect build statistics] (Arjun Salyan)<br> 5. Re: GSoC 2019 [Buildbot ideas] (Pierre Tardy)<br>
<br>
<br>----------------------------------------------------------------------<br>
<br>Message: 1<br>Date: Mon, 18 Mar 2019 21:22:06 +0530<br>From: Arjun Salyan <<a href="mailto:arjun.salyan.che17@itbhu.ac.in" target="_blank">arjun.salyan.che17@itbhu.ac.in</a>><br>To: Mojca Miklavec <<a href="mailto:mojca@macports.org" target="_blank">mojca@macports.org</a>><br>Cc: MacPorts Development <<a href="mailto:macports-dev@lists.macports.org" target="_blank">macports-dev@lists.macports.org</a>><br>Subject: Re: GSoC 2019 [Collect build statistics]<br>Message-ID:<br> <<a href="mailto:CAHToqhg-s-vBhMVqH-4D3q5fpNmM5NBH68ozveKAtKfWJiHxMw@mail.gmail.com" target="_blank">CAHToqhg-s-vBhMVqH-4D3q5fpNmM5NBH68ozveKAtKfWJiHxMw@mail.gmail.com</a>><br>Content-Type: text/plain; charset="utf-8"<br>
<br>Some improvements to the Demo App: <a href="https://frozen-falls-98471.herokuapp.com" rel="noreferrer" target="_blank">https://frozen-falls-98471.herokuapp.com</a><br>
<br> - All Ports and All Categories are now available (Although not all ports<br> have populated yet, I am on AWS Free Tier and the process is really slow.<br> At the time of drafting this email: around 500 have populated).<br> - On the Port-Detail page, the categories are now clickable and lead to<br> the list of ports under that category.<br>
<br>I was able to parse the entire PortIndex.json using a python script and<br>successfully converted it to Django fixtures which could then be populated<br>to the database. (I used the portindex.json outputted by current version of<br>portindex2json.tcl and fixed the issues with categories using same python<br>script)<br>
<br>Parse.py :<br>
<a href="https://github.com/arjunsalyan/MacPorts-Demo-App/blob/master/MacPorts/parse.py" rel="noreferrer" target="_blank">https://github.com/arjunsalyan/MacPorts-Demo-App/blob/master/MacPorts/parse.py</a><br>
<br>><br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <<a href="http://lists.macports.org/pipermail/macports-dev/attachments/20190318/faa62877/attachment-0001.html" rel="noreferrer" target="_blank">http://lists.macports.org/pipermail/macports-dev/attachments/20190318/faa62877/attachment-0001.html</a>><br>
<br>------------------------------<br>
<br>Message: 2<br>Date: Mon, 18 Mar 2019 17:37:30 +0100<br>From: Mojca Miklavec <<a href="mailto:mojca@macports.org" target="_blank">mojca@macports.org</a>><br>To: Arjun Salyan <<a href="mailto:arjun.salyan.che17@itbhu.ac.in" target="_blank">arjun.salyan.che17@itbhu.ac.in</a>><br>Cc: MacPorts Development <<a href="mailto:macports-dev@lists.macports.org" target="_blank">macports-dev@lists.macports.org</a>><br>Subject: Re: GSoC 2019 [Collect build statistics]<br>Message-ID:<br> <<a href="mailto:CALBOmsZNtSJZC2OnJJRe8OFNsMZ2_vvmDCypAwOMa6ffnCRj0Q@mail.gmail.com" target="_blank">CALBOmsZNtSJZC2OnJJRe8OFNsMZ2_vvmDCypAwOMa6ffnCRj0Q@mail.gmail.com</a>><br>Content-Type: text/plain; charset="UTF-8"<br>
<br>On Mon, 18 Mar 2019 at 16:52, Arjun Salyan wrote:<br>><br>> Some improvements to the Demo App: <a href="https://frozen-falls-98471.herokuapp.com" rel="noreferrer" target="_blank">https://frozen-falls-98471.herokuapp.com</a><br>
<br>Thank you very much.<br>
<br>> All Ports and All Categories are now available (Although not all ports have populated yet, I am on AWS Free Tier and the process is really slow. At the time of drafting this email: around 500 have populated).<br>
<br>I can imagine that the free plan would not be the fastest one in the<br>world, but to me 500 entries in what I could imagine might be an hour<br>since you started the job sounds like potential efficiency problem. A<br>forgotten index in a table can easily increase the runtime<br>polynomially or even exponentially.<br>
<br>> On the Port-Detail page, the categories are now clickable and lead to the list of ports under that category.<br>
<br>Cool!<br>
<br>What I would be potentially missing there is a number of ports under<br>that category. When I clicked around, several categories were empty.<br>You just need to make sure that all the relevant columns in the<br>database are indexed to make this efficient enough.<br>
<br>> I was able to parse the entire PortIndex.json using a python script and successfully converted it to Django fixtures which could then be populated to the database.<br>
<br>That's very good news. What exactly did you have to change to make it<br>work compared to last time?<br>
<br>> (I used the portindex.json outputted by current version of portindex2json.tcl and fixed the issues with categories using same python script)<br>><br>> Parse.py : <a href="https://github.com/arjunsalyan/MacPorts-Demo-App/blob/master/MacPorts/parse.py" rel="noreferrer" target="_blank">https://github.com/arjunsalyan/MacPorts-Demo-App/blob/master/MacPorts/parse.py</a><br>
<br>Thank you, I will look into it and provide further feedback.<br>
<br>Mojca<br>
<br>
<br>------------------------------<br>
<br>Message: 3<br>Date: Mon, 18 Mar 2019 18:19:27 +0100<br>From: Mojca Miklavec <<a href="mailto:mojca@macports.org" target="_blank">mojca@macports.org</a>><br>To: Arjun Salyan <<a href="mailto:arjun.salyan.che17@itbhu.ac.in" target="_blank">arjun.salyan.che17@itbhu.ac.in</a>><br>Cc: MacPorts Development <<a href="mailto:macports-dev@lists.macports.org" target="_blank">macports-dev@lists.macports.org</a>><br>Subject: Re: GSoC 2019 [Collect build statistics]<br>Message-ID:<br> <<a href="mailto:CALBOmsYgpDYDM9N45UE4czYavg8DZcuvXXLaAz6K0Jw0xGh2ug@mail.gmail.com" target="_blank">CALBOmsYgpDYDM9N45UE4czYavg8DZcuvXXLaAz6K0Jw0xGh2ug@mail.gmail.com</a>><br>Content-Type: text/plain; charset="UTF-8"<br>
<br>On Mon, 18 Mar 2019 at 17:37, Mojca Miklavec wrote:<br>> On Mon, 18 Mar 2019 at 16:52, Arjun Salyan wrote:<br>><br>> > All Ports and All Categories are now available (Although not all ports have populated yet, I am on AWS Free Tier and the process is really slow. At the time of drafting this email: around 500 have populated).<br>><br>> I can imagine that the free plan would not be the fastest one in the<br>> world, but to me 500 entries in what I could imagine might be an hour<br>> since you started the job sounds like potential efficiency problem. A<br>> forgotten index in a table can easily increase the runtime<br>> polynomially or even exponentially.<br>
<br>And in fact I'm unable to find any indices in your DB model.<br>
<br>
<a href="https://docs.djangoproject.com/en/2.1/topics/db/optimization/" rel="noreferrer" target="_blank">https://docs.djangoproject.com/en/2.1/topics/db/optimization/</a><br>
<a href="https://www.djangorocks.com/snippets/indexing-your-django-models.html" rel="noreferrer" target="_blank">https://www.djangorocks.com/snippets/indexing-your-django-models.html</a><br>
<br>Also, TextField might be suitable for description etc, but for short<br>entries like port name, this probably offers suboptimal performance<br>and CharField would make more sense. I did not time it though, and<br>this is not the bottleneck in your code, but the indices are<br>definitely critical for perfomance.<br>
<br>Mojca<br>
<br>
<br>------------------------------<br>
<br>Message: 4<br>Date: Mon, 18 Mar 2019 23:21:56 +0530<br>From: Arjun Salyan <<a href="mailto:arjun.salyan.che17@itbhu.ac.in" target="_blank">arjun.salyan.che17@itbhu.ac.in</a>><br>To: Mojca Miklavec <<a href="mailto:mojca@macports.org" target="_blank">mojca@macports.org</a>><br>Cc: MacPorts Development <<a href="mailto:macports-dev@lists.macports.org" target="_blank">macports-dev@lists.macports.org</a>><br>Subject: Re: GSoC 2019 [Collect build statistics]<br>Message-ID:<br> <CAHToqhjWNwYkrobZ+4Ti0Z=8C9kuQ=<a href="mailto:y6sUHG89%2B1zQkNeSrzPw@mail.gmail.com" target="_blank">y6sUHG89+1zQkNeSrzPw@mail.gmail.com</a>><br>Content-Type: text/plain; charset="utf-8"<br>
<br>On Mon, 18 Mar 2019 at 10:49 PM, Mojca Miklavec <<a href="mailto:mojca@macports.org" target="_blank">mojca@macports.org</a>> wrote:<br>
<br>> And in fact I'm unable to find any indices in your DB model.<br>
<br>
<br>Thanks, I shall add this. I am dealing with this huge data set for the<br>first time.<br>
<br>Also, TextField might be suitable for description etc, but for short<br>> entries like port name, this probably offers suboptimal performance<br>> and CharField would make more sense. I did not time it though, and<br>> this is not the bottleneck in your code, but the indices are<br>> definitely critical for perfomance.<br>
<br>
<br>Yes, I have finalised the ports table now and hence, I shall change the<br>field types accordingly as to which one is the most suitable for the data<br>type in that column.<br>
<br>Also, I have terminated the process of populating the database- my internet<br>today and the free tier both are making it really difficult. I was able to<br>load the entire database within seconds locally.<br>
<br>Thank You<br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <<a href="http://lists.macports.org/pipermail/macports-dev/attachments/20190318/740f60ae/attachment-0001.html" rel="noreferrer" target="_blank">http://lists.macports.org/pipermail/macports-dev/attachments/20190318/740f60ae/attachment-0001.html</a>><br>
<br>------------------------------<br>
<br>Message: 5<br>Date: Tue, 19 Mar 2019 11:29:09 +0100<br>From: Pierre Tardy <<a href="mailto:tardyp@gmail.com" target="_blank">tardyp@gmail.com</a>><br>To: Mojca Miklavec <<a href="mailto:mojca@macports.org" target="_blank">mojca@macports.org</a>><br>Cc: MacPorts Development <<a href="mailto:macports-dev@lists.macports.org" target="_blank">macports-dev@lists.macports.org</a>>, Rajdeep<br> Bharati <<a href="mailto:rajdeepbharati13@gmail.com" target="_blank">rajdeepbharati13@gmail.com</a>><br>Subject: Re: GSoC 2019 [Buildbot ideas]<br>Message-ID:<br> <CAJ+soVefa_Uver+uuKC20xR_HgJ=<a href="mailto:FongoQ9jTvUzLgKqLux1qg@mail.gmail.com" target="_blank">FongoQ9jTvUzLgKqLux1qg@mail.gmail.com</a>><br>Content-Type: text/plain; charset="utf-8"<br>
<br>Hi Rajdeep,<br>
<br>Sorry if I have not been clear.<br>The project I have linked is quite a large project which will actually<br>takes several summer in order to finish. What I meant is that it would be<br>good for synergy if we have another student on that project, as this is on<br>the same domain, and you could work with him/her to share your findings and<br>best practices.<br>
<br>You can start working on macport custom buildbot UI without waiting for<br>this project, and we can start working on that in modern javascript of<br>typescript if needed.<br>
<br>uglycoyte did a poc which you can find here:<br>
<a href="https://github.com/uglycoyote/buildbot-react-plugin-boilerplate" rel="noreferrer" target="_blank">https://github.com/uglycoyote/buildbot-react-plugin-boilerplate</a><br>
<br>I am not sure of the state of the PoC.<br>I suggest you as a welcome challenge to build yourself a buildbot<br>environment, with the react plugin installed in, displaying "hello from<br>Rajdeep's Buildbot"<br>
<br>Would that work for you?<br>
<br>Regards<br>Pierre<br>On Mon, Mar 18, 2019 at 10:53 AM Pierre Tardy <<a href="mailto:tardyp@gmail.com" target="_blank">tardyp@gmail.com</a>> wrote:<br>
<br>> Hi,<br>><br>> We have our own GSoC project to transition the UI from coffeescript:<br>><br>> <a href="https://github.com/buildbot/buildbot/wiki/Buildbot-GSoC-Projects-2019#transition-web-uis-data-module-from-coffeescript-to-typescript" rel="noreferrer" target="_blank">https://github.com/buildbot/buildbot/wiki/Buildbot-GSoC-Projects-2019#transition-web-uis-data-module-from-coffeescript-to-typescript</a><br>> The ideal would be to have one student for each project.<br>><br>> All the pointers are there. I would be happy to help on either project.<br>><br>> Regards,<br>> Pierre<br>><br>> On Mon, Mar 18, 2019 at 9:41 AM Mojca Miklavec <<a href="mailto:mojca@macports.org" target="_blank">mojca@macports.org</a>> wrote:<br>><br>>> Dear Rajdeep,<br>>><br>>> On Sun, 17 Mar 2019 at 19:55, Rajdeep Bharat wrote:<br>>> ><br>>> > Hello, I am Rajdeep, a GSoC aspirant.<br>>> > I was going through the Buildbot views project and had a few doubts:<br>>> > - Do I need to work on the dashboards SPA of buildbots (<br>>> <a href="https://github.com/buildbot/guanlecoja-ui" rel="noreferrer" target="_blank">https://github.com/buildbot/guanlecoja-ui</a>) and rewrite it using a modern<br>>> library like React?<br>>> > - I have read this ticket: <a href="https://trac.macports.org/ticket/55978" rel="noreferrer" target="_blank">https://trac.macports.org/ticket/55978</a>. Do<br>>> I have to implement them separately in a MacPorts repository of buildbot,<br>>> or do it in upstream directly?<br>>> ><br>>> > I am familiarizing myself with the buildbot codebase and would love to<br>>> start contributing.<br>>> > Can someone tell me if I'm on the right track?<br>>><br>>> Thank you very much for reaching to us. I still owe you the response<br>>> to the earlier email, but let's start with this one.<br>>><br>>> The "buildbot idea" would be (co)mentored by someone from the Buildbot<br>>> development team for the technical part ("How to do it?"), while the<br>>> MacPorts mentor would help to steer towards the desired goal ("What to<br>>> do?").<br>>><br>>> So I would like to ask Pierre to explain the technical part in more<br>>> details.<br>>><br>>> (Pierre, how does the unfinished Coffeescript to Typescript conversion<br>>> of the core affect designing new views and their "future-proof-ness"?)<br>>><br>>> What I want to say from the MacPorts point of view is that what mainly<br>>> kept us from switching to buildbot version beyond 0.8.x was a less<br>>> user-friendly waterfall view. We kept postponing the switch since the<br>>> time the buildbot was still at version 0.9.x. Now it's already at 2.x,<br>>> and it would be about a high time that we do the upgrade. The existing<br>>> buildbot setup was written in cca. 3 days during our in-person<br>>> developer meeting in 2016, and then slightly improved over time.<br>>><br>>> The changes would be added wherever it makes the most sense. If<br>>> there's some view that's specific to MacPorts, we would keep it<br>>> locally, while any other change that's considered useful for any<br>>> buildbot user would best be pushed to their own repository (and<br>>> reviewed by the buildbot development team).<br>>><br>>> I wanted to add that this project is well related to the "collection<br>>> of build statistics" idea. I will elaborate on this later, but we<br>>> definitely need changes in the buildbot configuration to make it<br>>> easier for our (hopefully) future web application to talk to the<br>>> buildbot master.<br>>><br>>> Mojca<br>>><br>>><br>>> Mojca<br>>><br>> --<br>><br>><br>> --<br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <<a href="http://lists.macports.org/pipermail/macports-dev/attachments/20190319/247fcfb9/attachment.html" rel="noreferrer" target="_blank">http://lists.macports.org/pipermail/macports-dev/attachments/20190319/247fcfb9/attachment.html</a>><br>
<br>------------------------------<br>
<br>Subject: Digest Footer<br>
<br>_______________________________________________<br>macports-dev mailing list<br>
<a href="mailto:macports-dev@lists.macports.org" target="_blank">macports-dev@lists.macports.org</a><br>
<a href="https://lists.macports.org/mailman/listinfo/macports-dev" rel="noreferrer" target="_blank">https://lists.macports.org/mailman/listinfo/macports-dev</a><br>
<br>------------------------------<br>
<br>End of macports-dev Digest, Vol 151, Issue 25<br>*********************************************<br>
</blockquote></div></div>