Volunteer for a workshop on "setting up your own buildbot/buildslave"?

[Ryan Schmidt]

On Nov 11, 2015, at 7:05 AM, Rainer Müller wrote:
> 
> On 2015-11-11 12:17, Ryan Schmidt wrote:
>> 
>> On Nov 10, 2015, at 12:21 PM, Joshua Root wrote:
>> 
>>> On 2015-11-11 00:26 , Mojca Miklavec wrote:
>>>> If we start including macosx_deployment_target, macosx_sdk,
>>>> prefix, applications_dir, frameworks_dir, ... we'll sooner or
>>>> later end up in an exponential mess
>>> 
>>> These are the type of settings that are associated with an entire
>>> source in archive_sites.conf, because all the archives from a given
>>> source will have them set the same. If any of these settings for a
>>> source don't match the ones used locally, the source is simply not
>>> used. Putting cxx_stdlib in here as well would be a good fit.
>> 
>> That sounds reasonable, except that it would create duplicate
>> archives (one "libstdcxx", one "libcxx") for noarch ports that
>> definitely don't use any C++ library, wouldn't it?
> 
> Technically you are correct, an archive for a noarch port would be
> identical regardless of the value of the cxx_stdlib option.
> 
> However, an archive site can only host archives with one configuration
> set (prefix, applications_dir, ...). If we add cxx_stdlib there, either
> all ports will use libstdc++ or all will use libc++.

So then there will be two archive sites: the existing one for libstdc++ on older systems and libc++ on newer systems, and another for libc++ on older systems.


>> Wouldn't it also have been reasonable to have the os name and version
>> as part of those settings? Why was that put into the filename
>> instead?
> 
> Historically evolved?

I think you're right. The archive filenames were created with what were thought to be all the relevant variables, and it was found later on that other variables were relevant as well, so they were put into the conf file.

If we had the chance to redo it now, would we still choose to divide these variables up the same way? If not, 

> You usually do not want to fetch archives for
> anything else than the current OS,

Of course we would not want that.

> so it is implicit.

I don't know what you mean by this.


> We could also host the packages in subdirectories:
>  .../${os.platform}/${os.major}/${build_arch}/*.tbz2
> However, you would have to replicate this structure on fetch. It would
> not make much difference whether this is in the path or in the filename.

Yes, for my hypothetical scenario, I was assuming that, in the absence of the OS name and version being in the filename, it would be in a directory name.

packages.m.o/darwin_15/boost/...
packages.m.o/darwin_10/boost/...
packages.m.o/darwin_10-libc++/boost/...

Having all of a port's packages -- for all platforms, variants and build options -- in a single directory is nice though because it's a single directory listing to look at to figure out if a particular binary exists. I use this all the time when deciding whether I need to force a buildbot build or investigate a buildbot build failure, and it would be less good for there to be even two directories to need to check, much less more than that.


>> If we used this strategy, what hypothetical base URL would we use for
>> libc++ packages on older systems? Would you define a second hostname
>> in addition to packages.macports.org (inconvenient for mirrors), or
>> would you create a subdirectory on that server?
> 
> Is your intention to build all packages twice, for both libc++ and
> libstdc++? Is that even worth the effort?

Yes, my intention is that there would be additional buildbot slave servers set up for libc++ on each older OS version. Why wouldn't that be worth the effort? Are you suggesting as an alternative that we would immediately switch the older OS buildbot slaves over to libc++ and no longer provide binaries for libstdc++? That would mean we would need an immediate plan for how to switch users from libstdc++ to libc++. I was hoping to take this one problem at a time, not all problems at once.