Mirorring distfiles or enabling libcurl/openssl from MP

[Mojca Miklavec]

On 23 December 2016 at 11:54, Rainer Müller wrote:
> On 2016-12-23 08:42, Mojca Miklavec wrote:
>> (was: Build Failure: cctools, libmacho, libmacho-headers)
>>
>> On 21 December 2016 at 12:42, Rainer Müller wrote:
>>> On 2016-12-21 04:20, Jeremy Huddleston Sequoia wrote:
>>>> Can we please configure base on these older builders to use MacPorts' libcurl+libressl/newer OpenSSL (or just a special one we can install to /opt/curl) instead of the host's version?  More and more source mirrors are dropping support for older SSL and TLS versions that aren't supported by Leopard out of the box.
>>>>
>>>> DEBUG: Fetching distfile failed: SSL certificate problem, verify that the CA cert is OK. Details:
>>>> error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
>>>
>>> It would be more favorable to restore mirroring to
>>> distfiles.macports.org, which is currently not happening. That would
>>> also solve the fetch problems for older macOS versions.
>>
>> I agree that mirroring the distfiles again would be ideal, but I'm not
>> sure how to set that up and it's been months with a lacking
>> functionality. In any case it could take some hours (days?) of work to
>> set it up properly and someone should start looking into that. This is
>> probably a job that would have to be run before anything else,
>> probably even prevent new builds from starting until done, so it might
>> be a bit more tricky to set it up properly.
>
> 1. Create a new builder with a SingleBranchScheduler on ports
>    that mirrors distfiles

And creates portindex etc.

> 2. Remove SingleBranchScheduler for portwatchers
> 3. Trigger all portwatchers in the last step of the new builder
>
> But so far we have not even deployed the buildbot builders for
> sychronizing www/guide/portindex which is also waiting for months in a
> finished state.

Exactly. We'll need quite some time to get this working.

>> Configuring the builders to use libcurl+libre/openssl from MacPorts is
>> something that can probably be done in 10-20 minutes per build slave.
>
> We deactivate all ports between builds, so special handling in mpbb
> would be necessary to keep these ports as active and avoid operations
> that would break the port(1) command.

No, see below.

> The buildbots are useful as a test system especially for missing
> dependencies, but when curl and openssl are always active we would loose
> that functionality.
>
> I don't think it is worth to invest time into this instead of fixing
> mirroring.

No, no, no. We are not talking about keeping some ports active all the
time. We need to have two instances of MacPorts installed anyway. The
libcurl+libre/openssl would come from that other MP installation which
also provides some tools like subversion, git, buildbot-slave, ... I
think it's just a matter of a few additional parameters when calling
the ./configure script of MacPorts base, so that MP will link against
the latest libcurl from the "tools MP installation" rather than
against the system one. I don't know how selfupgrade would behave when
a new release comes out in that case, but since we don't release that
often, this shouldn't be so much of an issue and we might have already
fixed the problem by the time of the next release.

>> (10.5 could just as well be shut down unless one of those two things
>> gets deployed. Without having a functional python, py-setuptools, ...
>> and with many builds with unmet dependency taking one hour for not
>> doing anything ... we're just consuming build cycles for absolutely
>> zero benefit and annoying committers with build failure emails. We had
>> 4 successful builds out of 200.)
>
> If 10.5 is that broken and nobody wants to fix it, it is probably time
> we declare 10.5 as too old and leave it behind.

I still find it useful and we are not asking maintainers to fix
problems unless they want to and are able to, but it is "stupid" that
simple things like inability to fetch distfiles prevents it from
building anything at all.

Mojca