upgrade to openssl 3.0.0

Mon Oct 4 09:23:03 UTC 2021

> On 2 Oct 2021, at 9:03 pm, Renee Otten <reneeotten at macports.org> wrote:
> 
> I was initially suggesting the same approach, which is a slightly more conservative approach, in the Trac ticket (https://trac.macports.org/ticket/63461 <https://trac.macports.org/ticket/63461>) but by now I am in favor of the approach that Ken suggested. This was doable for boost as there were a much more limited number of ports affected (probably about one third or so), but even there I presume that you spend a significant amount of time to implement this (which is of course much appreciated). I am not saying that it cannot be done and the advantage would be that things don’t need to be done en masse, but the drawback is that chances are high that the transition will happen very slowly and might never finish as probably many ports are “nomaintainer”. 
> 
> So the approach outlined by Ken (i.e., testing now moving ports that don’t build to 1.1.1) and then make the change and fix stuff if it breaks seems the faster way to get to the finish line. 

The approach Ken suggests is a minor variant of what I suggested. The only difference is what openssl version you make the ‘default’ in the new openssl PortGroup. I was suggesting starting with with making that 1.1.1, to minimise the disruption, but sure once that change is made and everything is OK we can try migrating that to 3.0.0 and see what happens.

You can see what version the shim port provides for boost here

https://github.com/macports/macports-ports/blob/master/devel/boost/Portfile <https://github.com/macports/macports-ports/blob/master/devel/boost/Portfile>

see how the shim port version is bound to the same version as the default version in the PG. This way ports still using the shim, and those using the default PG version, are actually using the same binaries.

Chris

> 
> Renee
> 
> 
>> On Oct 2, 2021, at 3:06 PM, Ken Cunningham <ken.cunningham.webuse at gmail.com <mailto:ken.cunningham.webuse at gmail.com>> wrote:
>> 
>> That is exactly the approach that I don't find workable, Chris.
>> 
>> Specifically:
>> 
>> 1. every single port (think: rust, ghc, blah blah) will have to be needlessly managed to build against an alternate openssl location.
>> 2. there will be no default openssl ever in prefix.
>> 
>> So I am not on board for trying to implement that myself, but should that be the selected course I will watch from the sidelines as others proceed down that path.
>> 
>> K
>> 
>>> On Oct 2, 2021, at 12:02 PM, Chris Jones <jonesc at hep.phy.cam.ac.uk <mailto:jonesc at hep.phy.cam.ac.uk>> wrote:
>>> 
>>> Hi,
>>> 
>>> Personally I would strongly recommend following a migration path that does not require an enmass change but allows ports to migrate  at their own pace. I would personally very much recommend an approach similar to that we used for boost recently
>>> 
>>> 1. Introduced a set of new isolated opensslXYZ ports that install specific versions into isolated install areas under libexec.
>>> 2. Create a new portgroup that handles the confirmation of ports to pick the openssl version they wish to use, with some default, and performs various standard configurations for standard build systems such that for most ports they work out the box with the new portgroup. For those that don’t provide various proc methods that return the configured install paths such that ports can used them to configure themselves as required.
>>> 3. Make the current openssl port a basic shim port that just installs sym links into the primary install area to the default openssl version (for now 1.1.1), so ports building against the current openssl port carry on working just as they are now, but are redirected to use the same versioned 1.1.1 port as those using the PG.
>>> 
>>> Once this is in place, ports can be migrated to use the new PG individually as and when we want.
>>> 
>>> Chris
>>> 
>>>> On 2 Oct 2021, at 6:17 pm, Ken Cunningham <ken.cunningham.webuse at gmail.com <mailto:ken.cunningham.webuse at gmail.com>> wrote:
>>>> 
>>>> 
>>>> openssl 3.0.0 is out, with a new and very favourable Apache-2 license that will let many previously non-distributable ports become distributable.  However, there are 758 ports that indicate they link against openssl. That is a daunting number of ports indeed.
>>>> 
>>>> One option might be to move all of our openssl ports (1.0, 1.1, and 3.x) into subdirectories out of ${prefix}, have no default openssl installed in $[prefix} directly, create a new PortGroup, openssl_select or some such, add that PG to all 758 ports, default it to openssl 1.1.1, and then allow people to gradually move the 758 ports to openssl 3.0.0 as they get to it.
>>>> 
>>>> Although that is possible with suitable effort, I don’t think that is workable for many reasons, the most obvious being that we would then have to modify every single port in some way to find an openssl installed into a non-default prefix. Creating the PG and adding it to 758 ports might be work enough, but then finding the right way to force all 758 ports to build properly against an openssl that is not in the default prefix is the real horror, and seems like a nightmare of wasted labours.
>>>> 
>>>> So it would appear that the same option that was chosen last time, ie upgrade the default openssl in ${prefix} to the newer openssl, and then fix the subset of ports that will not build against it to use an older openssl appears both the better option and lot less work (assuming most ports do build against openssl 3.0.0, which seems to be the case so far). Some will disagree, but I put it to you that it is going to be far less work in the end to force a few % of the ports to a specific alternate openssl than force all of them, all the time, forever.
>>>> 
>>>> Most things I have attempted to rebuild over the past few days have rebuilt without any issues, but a few things don’t build with openssl 3.0.0 yet and will need to stay with openssl 1.1.1 for a while until patched or updated (or forever). That will require both forcing those ports to find an alternate openssl installation, and also (the tricky part) forcing them to ignore the openssl in the default prefix.
>>>> 
>>>> To support this, there is a new openssl11 port that provides openssl 1.1.1 tucked away in a subdir, much like we have openssl10, and a few new options were added to the old_openssl PortGroup to allow most ports to be forced to the alternate openssl with minimal fuss. Add the PortGroup, spec the branch, and choose the method, for the most part.
>>>> 
>>>> If this plan holds, I would anticipate that we move ports that we find need to stay on openssl 1.1.1 to openssl11 using the old_openssl PortGroup soon or now, before we upgrade to openssl 3.0.0 to minimize fuss. Then once we have done that, upgrading the default openssl to 3.0.0 and revbumping the deps would occur.
>>>> 
>>>> So I am asking that anyone with an interest in a port that uses openssl please try building it against openssl 3.0.0 in the PR below, and if there is a problem that can’t be solved by an update or a patch, consider trying to use the old_openssl PortGroup to fix the build and move it over. As there are so many ports, it would help if people pitched in with the ones that are important to them.
>>>> 
>>>> The openssl 3.0.0 upgrade PR is here:
>>>> 
>>>> https://github.com/macports/macports-ports/pull/12410 <https://github.com/macports/macports-ports/pull/12410>
>>>> 
>>>> and my own WIP repo of a few ports that I have found won’t build with openssl 3.0.0 at present and fixed to use the old_openssl PG is here:
>>>> 
>>>> https://github.com/kencu/mybigsuroverlay/ <https://github.com/kencu/mybigsuroverlay/>
>>>> 
>>>> the idea is that the ports in that repo would be moved into the main repo as we work through the main ones that use openssl at least.
>>>> 
>>>> 
>>>> K
>>>> 
>>>> PS. One thing I have not yet sorted out how to do is how to support libressl in the setting of needing to force a port to openssl 1.1.1 using the old_openssl PortGroup. Open to any thoughts on this. This case wsa basically just ignored last time with openssl10, and we may have to do the same again unless someone has a bright idea.
>>>> 
>>>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macports.org/pipermail/macports-dev/attachments/20211004/a0a84a8c/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1930 bytes
Desc: not available
URL: <http://lists.macports.org/pipermail/macports-dev/attachments/20211004/a0a84a8c/attachment-0001.bin>