Volunteer for a workshop on "setting up your own buildbot/buildslave"?

[Mojca Miklavec]

On Wed, Nov 11, 2015 at 2:18 PM, Ryan Schmidt wrote:
> On Nov 11, 2015, at 7:05 AM, Rainer Müller wrote:
>> On 2015-11-11 12:17, Ryan Schmidt wrote:
>>> On Nov 10, 2015, at 12:21 PM, Joshua Root wrote:
>>>
>>>> On 2015-11-11 00:26 , Mojca Miklavec wrote:
>>>>> If we start including macosx_deployment_target, macosx_sdk,
>>>>> prefix, applications_dir, frameworks_dir, ... we'll sooner or
>>>>> later end up in an exponential mess
>>>>
>>>> These are the type of settings that are associated with an entire
>>>> source in archive_sites.conf, because all the archives from a given
>>>> source will have them set the same. If any of these settings for a
>>>> source don't match the ones used locally, the source is simply not
>>>> used. Putting cxx_stdlib in here as well would be a good fit.
>>>
>>> That sounds reasonable, except that it would create duplicate
>>> archives (one "libstdcxx", one "libcxx") for noarch ports that
>>> definitely don't use any C++ library, wouldn't it?
>>
>> Technically you are correct, an archive for a noarch port would be
>> identical regardless of the value of the cxx_stdlib option.
>>
>> However, an archive site can only host archives with one configuration
>> set (prefix, applications_dir, ...). If we add cxx_stdlib there, either
>> all ports will use libstdc++ or all will use libc++.
>
> So then there will be two archive sites: the existing one for libstdc++ on older systems and libc++ on newer systems, and another for libc++ on older systems.

I find this split more than a bit weird. Either we have a separate
directory for each configuration or we keep all the archives together
unless there are strong technical reasons against that.

>> We could also host the packages in subdirectories:
>>  .../${os.platform}/${os.major}/${build_arch}/*.tbz2
>> However, you would have to replicate this structure on fetch. It would
>> not make much difference whether this is in the path or in the filename.
>
> Yes, for my hypothetical scenario, I was assuming that, in the absence of the OS name and version being in the filename, it would be in a directory name.
>
> packages.m.o/darwin_15/boost/...
> packages.m.o/darwin_10/boost/...
> packages.m.o/darwin_10-libc++/boost/...
>
> Having all of a port's packages -- for all platforms, variants and build options -- in a single directory is nice though because it's a single directory listing to look at to figure out if a particular binary exists.

Truth to be told it would be even nicer if we had a nice website where
you would get all the information collected on the same site: which
binaries exist, when were the builds attempted (with a link to the
build) and whether the build failed or succeeded. If we had such a
site, this argument would no longer be important.

> I use this all the time when deciding whether I need to force a buildbot build or investigate a buildbot build failure, and it would be less good for there to be even two directories to need to check, much less more than that.
>
>>> If we used this strategy, what hypothetical base URL would we use for
>>> libc++ packages on older systems? Would you define a second hostname
>>> in addition to packages.macports.org (inconvenient for mirrors), or
>>> would you create a subdirectory on that server?
>>
>> Is your intention to build all packages twice, for both libc++ and
>> libstdc++? Is that even worth the effort?
>
> Yes, my intention is that there would be additional buildbot slave servers set up for libc++ on each older OS version. Why wouldn't that be worth the effort? Are you suggesting as an alternative that we would immediately switch the older OS buildbot slaves over to libc++ and no longer provide binaries for libstdc++? That would mean we would need an immediate plan for how to switch users from libstdc++ to libc++. I was hoping to take this one problem at a time, not all problems at once.

I agree. We should first introduce buildbot slaves for libc++, then
take enough time to deal with all the new problems discovered during
the transition, and only then suggest users to switch. After a while
(maybe a year or so) of stability it would perhaps be acceptable to
switch off the buildbot slaves for libstdc++. Users would then have a
choice of either continuing to use it, but having to compile
everything on their own, or switching to libc++ and still getting
binary updates.

Doing the switch all at once would be a bit too challenging in my
opinion. Users will have to reinstall everything and you cannot ask
them to do that until at least the most obvious problems have been
solved.

Mojca