GSoC 2019 [Buildbot ideas]

[Pierre Tardy]

My fear is that this is part of the many stretch goals, and this is
beginning to be very optimistic schedule.
I think it is best to make a great finished GSoC rather than lots of very
cool but unfinished mini projects.

>From my experience, I'd say settuping securing and optimising macos
precommit CI is a full 3 month work.

The number one mistake for young talented people is to underestimate things.

https://en.wikipedia.org/wiki/Pareto_principle

Regards,
Pierre


Le jeu. 28 mars 2019 à 18:50, Rajdeep Bharati <rajdeepbharati13 at gmail.com>
a écrit :

> I will try to set up libvirt. I can keep the PR comment from admin as a
> backup option.
>
> Rajdeep
>
> On Thu, Mar 28, 2019 at 5:37 PM Pierre Tardy <tardyp at gmail.com> wrote:
>
>> You can take control of the VM by downloading a ransomware or botnet or
>> whatever.
>>
>> You usually counter that by making sure the PR VMs are restricted in term
>> of network access they can do, and also restricted in the number of time it
>> is alive (basically just the time of the build)
>>
>> Another much more simple option is to trigger the PR testing  via a PR
>> comment from an admin.
>>
>> If a macPort maintainer sends a message like "Go Buildbot", then buildbot
>> would catche that a start a build, provided that the PR got basic review,
>> and is not suspicious.
>>
>>
>> Pierre
>>
>>
>> Le jeu. 28 mars 2019 à 13:03, Rajdeep Bharati <rajdeepbharati13 at gmail.com>
>> a écrit :
>>
>>> All right. Could you please give an example of a malicious PR? Would it
>>> be one which is done (locally tested) from an old version of macOS?
>>>
>>> On Wed, Mar 27, 2019 at 9:55 PM Mojca Miklavec <mojca at macports.org>
>>> wrote:
>>>
>>>> Dear Rajdeep,
>>>>
>>>> It's not just a question of how to fetch a PR. That shouldn't be too
>>>> difficult, I hope (and probably the link you provided works as intended).
>>>>
>>>> The tricky question is how to prevent malicious PRs from doing damage
>>>> on the builders. I assume that a proper solution would require starting a
>>>> fresh VM for each build. There is some support in the buildbot already:
>>>>
>>>> http://docs.buildbot.net/2.1.0/manual/configuration/workers-libvirt.html
>>>>     https://github.com/kholia/OSX-KVM
>>>> but we would need to find a way to create VMs with macOS, so it might
>>>> not be trivial to do it. On top of that what we would really need the PRs
>>>> for are the old machines (say, 10.6, or even 10.4 if we would want to go to
>>>> extremes) where it might be even less trivial to automate this in a nice
>>>> way.
>>>>
>>>> (A compromise solution would be to only allow trusted developers to
>>>> test pull requests on devoted builders, where we would also need to make
>>>> sure to uninstall the software after the PR is done building.)
>>>>
>>>> While implementing this remains almost the number one requested thing
>>>> when people contribute to packages, I'm not sure how much time doing this
>>>> would take. It could be that this could be done in a day or a few days, but
>>>> it's also possible that there would be some stumbling block that would
>>>> require more hacking skills and would prevent us from proceeding, and not
>>>> even two months would suffice. In one way, I wouldn't mind if a student
>>>> would work on this for the full summer to get this working; on the other
>>>> hand, if there's a block and none of us is skilled enough to overcome it,
>>>> it makes more sense to proceed with other stuff that can certainly be done.
>>>>
>>>> Mojca
>>>>
>>>>
>>>> On Wed, 27 Mar 2019 at 16:05, Rajdeep Bharati <
>>>> rajdeepbharati13 at gmail.com> wrote:
>>>>
>>>>> I could use the GitHubPullrequestPoller
>>>>> <http://docs.buildbot.net/current/manual/configuration/changesources.html#chsrc-GitHubPullrequestPoller> which
>>>>> periodically polls the Github API for new/updated PRs.
>>>>>
>>>>> Here is an example:
>>>>> https://github.com/halide/build_bot/blob/master/master/master.cfg
>>>>>
>>>>> c['change_source'].append(GitHubPullrequestPoller(
>>>>> owner = 'halide',
>>>>> repo = 'Halide',
>>>>> token = token,
>>>>> pullrequest_filter = pr_filter,
>>>>> pollInterval = 60*5, # Check Halide PRs every five minutes
>>>>> pollAtLaunch = True))
>>>>> Rajdeep
>>>>>
>>>>> On Wed, Mar 27, 2019 at 3:59 AM Mojca Miklavec <mojca at macports.org>
>>>>> wrote:
>>>>>
>>>>>> Dear Rajdeep,
>>>>>>
>>>>>> On Tue, 26 Mar 2019 at 19:51, Rajdeep Bharati wrote:
>>>>>> >
>>>>>> > I have submitted a draft proposal:
>>>>>> https://docs.google.com/document/d/12wRjA8sOWNOuApHZ_fm0n1aIPLVPt9Xm2yGiMwiK3AI/edit.
>>>>>> Could you please provide some feedback?
>>>>>>
>>>>>> Cool, thank you very much, it looks nice, please give us a bit of
>>>>>> time.
>>>>>>
>>>>>> One question: what precisely is your plan for setting up disposable
>>>>>> builds for PRs?
>>>>>>
>>>>>> Mojca
>>>>>>
>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macports.org/pipermail/macports-dev/attachments/20190329/8be60c9f/attachment-0001.html>