CI system for PR builds (was: Re: Changing default cxx_stdlib to libc++)

[Ryan Schmidt]

On Mar 14, 2018, at 07:25, db wrote:
> On 14 Mar 2018, at 04:08, Ryan Schmidt wrote:
>> I was not aware of the existence of GitLab CI, but I haven't done a survey of CI systems, mainly because we already selected one many years ago: Buildbot.
> 
> GitLab CI is now integrated in GitLab, but AFAIK doesn't integrate with GitHub right now unless you mirror from it. There was an issue open to support it.

Again, I'm not really interested, since we already selected Buildbot as our CI system.


>> The problem, to my mind, was not that of selecting which CI system to use, but the fact that we want to have a virtual machine template that the CI system should clone, boot up, run a job in, and then delete, or even multiple templates, one for each OS version we want to test.
> 
>> But out Buildbot VMs run on VMware ESXi, so if I were to add builds for PRs to the system, I would of course want to continue using VMware.
> 
>> We also have the problem of what exactly to put on the VM templates, and how to keep them updated.
> 
> Have you considered vagrant? It can use ESXi as hypervisor (provider) and you can provision your machines with a shell script in a var or use ansible to maintain your versions updated.

No, I haven't. I'm aware that we have a port for vagrant. Beyond that, I don't know anything about it.


>> But one of the reasons why our Buildbot setup is as fast as it is is that each Buildbot worker keeps previously-built ports installed (but inactive).
> 
>> We could pre-populate the VM templates with lots of installed but inactive ports, but in addition to making the templates and their clone(s) take up more disk space, which is at a premium, the installed ports would quickly become outdated as port updates are committed. We would need a mechanism for updating the ports installed on the templates fairly frequently.
> 
> That shouldn't happen if every update triggers the CI, right?

Because PRs come from untrusted sources, we have to assume their contents are tainted. So after any PR is finished building, the VM is tainted and we have to throw it away and make a new one from our template for the next PR build. The template would remain unchanged between builds. If the template were prepopulated with a selection of installed ports, that would become out of date over time.

> Otherwise, you could make the machines sync to the packages public server for the distributable, and to a private server for the non-distributable binaries.

I can't find an interpretation of that sentence that helps to solve the prepopulation problem. I don't think we need to worry about it too much. To start with, we can just not prepopulate any ports. Having the MacPorts installation on the build worker obtain any needed archives from the packages server, like any other MacPorts installation, should be fine. We would just need to tell it to also try to download archives from the private server that we would create. I don't remember if MacPorts has a facility for telling it to do that.