upgrade to openssl 3.0.0

Christopher Jones jonesc at hep.phy.cam.ac.uk
Mon Oct 4 20:51:07 UTC 2021 


> On 4 Oct 2021, at 5:54 pm, Ken Cunningham <ken.cunningham.webuse at gmail.com> wrote:
> 
> as you wish, Chris.
> 
> 
> I was hoping to move this along for the overwhelming benefit of the license, but TBH the push-back so far is 99.99% negative about moving to openssl 3.0.0 this year, so too controversial for me to get involved with. I'll sit back for six to twelve months and see what you guys work out over the coming year.

All the more reason to follow my suggested migration path then I would say, as it allows an openssl30 port to be made available, and those ports that wish to can use it via the new PG, but it doesn’t have to become the default until some later date.

TBH I also was quite dubious of making 3.0.0 the default any time ’soon’

Chris

> 
> K
> 
> On Mon, Oct 4, 2021 at 8:18 AM Chris Jones <jonesc at hep.phy.cam.ac.uk <mailto:jonesc at hep.phy.cam.ac.uk>> wrote:
> 
> Reading this thread again, I think there is some confusion over what I 
> am suggesting, compared to Renee/Ken, as really the differences are 
> minor and cosmetic packaging differences.
> 
> The main difference is I am suggesting aiming for consistency across 
> where all the openssl ports install to. Instead of having some install 
> to libexec, and then one 'blessed' default port install directly into 
> ${prefix} have opensslXYZ ports for *all* versions we need, that all 
> install to their own libexec area. Then, have a trivial shim port for 
> the general 'openssl' port that just installs a few sym links from 
> whatever the 'default' openssl version is deemed to be (currently 1.1.1) 
> into the primary ${prefix} install.
> 
> We could do this, right now, i.e. move the current openssl port to an 
> isolated libexec openssl11 port, and update openssl to just be a shim to 
> that install. This would be completely transparent to all ports using 
> openssl.
> 
> The advantage of this to me would be
> 
> 1. Consistency across all the openssl ports, as they would all, 
> regardless of their version, be treated (installed) the same way.
> 
> 2. Once there, we could then introduce a new port group that extends 
> what the current old_openssl PG does but in a more general way, as it 
> would work for not just 'old' openssl versions, but also the current 
> default and also new ones. e.g. we could then add a openssl30 port and 
> allow a few ports to experiment with it.
> 
> 3. Once we are OK with a few ports using openssl30 we could 'throw the 
> switch' and update the shim port to point to the 3.0 version.
> 
> The above, if you think about it, is really no different to what 
> Ken/Renee are suggesting in terms of what ports have to do. If ports are 
> OK with just relying on the default they can carry on using the 
> 'openssl' (shim) port, just as they are now. Nothing is forcing us to 
> change these ports. *If* ports wanted to migrate to use the new PG that 
> is fine, but its an optional choice (unless they are not compatible with 
> 3.0 of course, but then they will need updating anyway once 3.0 is the 
> default).
> 
> So why do the above instead of what Ken/Renee suggest ? for me the 
> primary reason is we end up with a system where in future it will be 
> much easier to deal with future new major versions, without having to 
> have these discussions again... Say 3.1 or 4.0 comes out, a new isolated 
> opensslXY port can be added to provide that, that will live alongside 
> the current ones. A few ports could be tested against it, and as and 
> when we are ready the default can be flipped again to point to it.
> 
> Chris
> 
> 
> On 04/10/2021 10:24 am, Christopher Jones wrote:
> > 
> > 
> >> On 2 Oct 2021, at 8:06 pm, Ken Cunningham 
> >> <ken.cunningham.webuse at gmail.com <mailto:ken.cunningham.webuse at gmail.com> 
> >> <mailto:ken.cunningham.webuse at gmail.com <mailto:ken.cunningham.webuse at gmail.com>>> wrote:
> >>
> >> That is exactly the approach that I don't find workable, Chris.
> >>
> >> Specifically:
> >>
> >> 1. every single port (think: rust, ghc, blah blah) will have to be 
> >> needlessly managed to build against an alternate openssl location.
> > 
> > why ? They can start of using the shim port and only use the isolated 
> > builds as and when they are ready
> > 
> >> 2. there will be no default openssl ever in prefix.
> > 
> > why not ? The boost set has one and I see no reason why openssl cannot 
> > as well.
> > 
> > Chris
> > 
> >>
> >> So I am not on board for trying to implement that myself, but should 
> >> that be the selected course I will watch from the sidelines as others 
> >> proceed down that path.
> >>
> >> K
> >>
> >>> On Oct 2, 2021, at 12:02 PM, Chris Jones <jonesc at hep.phy.cam.ac.uk <mailto:jonesc at hep.phy.cam.ac.uk> 
> >>> <mailto:jonesc at hep.phy.cam.ac.uk <mailto:jonesc at hep.phy.cam.ac.uk>>> wrote:
> >>>
> >>> Hi,
> >>>
> >>> Personally I would strongly recommend following a migration path that 
> >>> does not require an enmass change but allows ports to migrate  at 
> >>> their own pace. I would personally very much recommend an approach 
> >>> similar to that we used for boost recently
> >>>
> >>> 1. Introduced a set of new isolated opensslXYZ ports that install 
> >>> specific versions into isolated install areas under libexec.
> >>> 2. Create a new portgroup that handles the confirmation of ports to 
> >>> pick the openssl version they wish to use, with some default, and 
> >>> performs various standard configurations for standard build systems 
> >>> such that for most ports they work out the box with the new 
> >>> portgroup. For those that don’t provide various proc methods that 
> >>> return the configured install paths such that ports can used them to 
> >>> configure themselves as required.
> >>> 3. Make the current openssl port a basic shim port that just installs 
> >>> sym links into the primary install area to the default openssl 
> >>> version (for now 1.1.1), so ports building against the current 
> >>> openssl port carry on working just as they are now, but are 
> >>> redirected to use the same versioned 1.1.1 port as those using the PG.
> >>>
> >>> Once this is in place, ports can be migrated to use the new PG 
> >>> individually as and when we want.
> >>>
> >>> Chris
> >>>
> >>>> On 2 Oct 2021, at 6:17 pm, Ken Cunningham 
> >>>> <ken.cunningham.webuse at gmail.com <mailto:ken.cunningham.webuse at gmail.com> 
> >>>> <mailto:ken.cunningham.webuse at gmail.com <mailto:ken.cunningham.webuse at gmail.com>>> wrote:
> >>>>
> >>>> 
> >>>> openssl 3.0.0 is out, with a new and very favourable Apache-2 
> >>>> license that will let many previously non-distributable ports become 
> >>>> distributable.  However, there are 758 ports that indicate they link 
> >>>> against openssl. That is a daunting number of ports indeed.
> >>>>
> >>>> One option might be to move all of our openssl ports (1.0, 1.1, and 
> >>>> 3.x) into subdirectories out of ${prefix}, have no default openssl 
> >>>> installed in $[prefix} directly, create a new PortGroup, 
> >>>> openssl_select or some such, add that PG to all 758 ports, default 
> >>>> it to openssl 1.1.1, and then allow people to gradually move the 758 
> >>>> ports to openssl 3.0.0 as they get to it.
> >>>>
> >>>> Although that is possible with suitable effort, I don’t think that 
> >>>> is workable for many reasons, the most obvious being that we would 
> >>>> then have to modify every single port in some way to find an openssl 
> >>>> installed into a non-default prefix. Creating the PG and adding it 
> >>>> to 758 ports might be work enough, but then finding the right way to 
> >>>> force all 758 ports to build properly against an openssl that is not 
> >>>> in the default prefix is the real horror, and seems like a nightmare 
> >>>> of wasted labours.
> >>>>
> >>>> So it would appear that the same option that was chosen last time, 
> >>>> ie upgrade the default openssl in ${prefix} to the newer openssl, 
> >>>> and then fix the subset of ports that will not build against it to 
> >>>> use an older openssl appears both the better option and lot less 
> >>>> work (assuming most ports do build against openssl 3.0.0, which 
> >>>> seems to be the case so far). Some will disagree, but I put it to 
> >>>> you that it is going to be far less work in the end to force a few % 
> >>>> of the ports to a specific alternate openssl than force all of them, 
> >>>> all the time, forever.
> >>>>
> >>>> Most things I have attempted to rebuild over the past few days have 
> >>>> rebuilt without any issues, but a few things don’t build with 
> >>>> openssl 3.0.0 yet and will need to stay with openssl 1.1.1 for a 
> >>>> while until patched or updated (or forever). That will require both 
> >>>> forcing those ports to find an alternate openssl installation, and 
> >>>> also (the tricky part) forcing them to ignore the openssl in the 
> >>>> default prefix.
> >>>>
> >>>> To support this, there is a new openssl11 port that provides openssl 
> >>>> 1.1.1 tucked away in a subdir, much like we have openssl10, and a 
> >>>> few new options were added to the old_openssl PortGroup to allow 
> >>>> most ports to be forced to the alternate openssl with minimal fuss. 
> >>>> Add the PortGroup, spec the branch, and choose the method, for the 
> >>>> most part.
> >>>>
> >>>> If this plan holds, I would anticipate that we move ports that we 
> >>>> find need to stay on openssl 1.1.1 to openssl11 using the 
> >>>> old_openssl PortGroup soon or now, before we upgrade to openssl 
> >>>> 3.0.0 to minimize fuss. Then once we have done that, upgrading the 
> >>>> default openssl to 3.0.0 and revbumping the deps would occur.
> >>>>
> >>>> So I am asking that anyone with an interest in a port that uses 
> >>>> openssl please try building it against openssl 3.0.0 in the PR 
> >>>> below, and if there is a problem that can’t be solved by an update 
> >>>> or a patch, consider trying to use the old_openssl PortGroup to fix 
> >>>> the build and move it over. As there are so many ports, it would 
> >>>> help if people pitched in with the ones that are important to them.
> >>>>
> >>>> The openssl 3.0.0 upgrade PR is here:
> >>>>
> >>>> https://github.com/macports/macports-ports/pull/12410 <https://github.com/macports/macports-ports/pull/12410> 
> >>>> <https://github.com/macports/macports-ports/pull/12410 <https://github.com/macports/macports-ports/pull/12410>>
> >>>>
> >>>> and my own WIP repo of a few ports that I have found won’t build 
> >>>> with openssl 3.0.0 at present and fixed to use the old_openssl PG is 
> >>>> here:
> >>>>
> >>>> https://github.com/kencu/mybigsuroverlay/ <https://github.com/kencu/mybigsuroverlay/> 
> >>>> <https://github.com/kencu/mybigsuroverlay/ <https://github.com/kencu/mybigsuroverlay/>>
> >>>>
> >>>> the idea is that the ports in that repo would be moved into the main 
> >>>> repo as we work through the main ones that use openssl at least.
> >>>>
> >>>>
> >>>> K
> >>>>
> >>>> PS. One thing I have not yet sorted out how to do is how to support 
> >>>> libressl in the setting of needing to force a port to openssl 1.1.1 
> >>>> using the old_openssl PortGroup. Open to any thoughts on this. This 
> >>>> case wsa basically just ignored last time with openssl10, and we may 
> >>>> have to do the same again unless someone has a bright idea.
> >>>>
> >>>>
> >>
> > 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macports.org/pipermail/macports-dev/attachments/20211004/a02aeb3f/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1930 bytes
Desc: not available
URL: <http://lists.macports.org/pipermail/macports-dev/attachments/20211004/a02aeb3f/attachment-0001.bin>

More information about the macports-dev mailing list