[MacPorts] #51516: MacPorts should use a bundled copy of a newer libcurl and SSL library rather than the OS X version

[MacPorts]

#51516: MacPorts should use a bundled copy of a newer libcurl and SSL library
rather than the OS X version
--------------------------+--------------------------------
  Reporter:  ryandesign   |      Owner:  macports-tickets@…
      Type:  enhancement  |     Status:  new
  Priority:  Normal       |  Milestone:  MacPorts Future
 Component:  base         |    Version:
Resolution:               |   Keywords:
      Port:               |
--------------------------+--------------------------------

Comment (by ryandesign):

 Let me try one more time to explain my position. I still believe that we
 ''should'' bundle libcurl with MacPorts (either on all systems or only on
 older systems that don't get updates from Apple anymore), and that we
 ''should'' bundle a newer libressl or openssl with MacPorts (either only
 on older systems that don't get updates from Apple anymore or only on
 older systems that don't support newer TLS algorithms that many servers
 now require). On newer systems that still get updates from Apple we can
 either use our bundled curl and link it with Apple's security framework,
 or we can continue to use the system's libcurl as we do now.

 To be concrete about it, at this time, I might propose that we use a
 bundled newer libcurl/libressl/openssl on OS X 10.11 El Capitan and
 earlier. I believe El Capitan is no longer receiving security updates as
 of the end of 2018. The problem I mentioned in #51045 was present in El
 Capitan's curl but not Sierra's, and also René recently encountered a KDE
 download server that could not be accessed from El Capitan's curl or
 earlier but worked with Sierra's. On Sierra and later, we could either use
 our bundled libcurl linked with the security framework or else use macOS
 libcurl.


 Replying to [comment:53 neverpanic]:
 > I do not want to ship a library with potentially known and dangerous
 security vulnerabilities to users.

 To what library are you referring? libcurl? openssl? libressl? What are
 the specific known and dangerous security vulnerabilities that you're
 referring to?

 I'm not certain what types of security vulnerabilities do or could exist
 in those libraries but even if there were any I'm not sure how serious
 that would be for us. We would only be using this libcurl for MacPorts
 tasks like downloading known files. We checksum our distfiles and sign our
 packages, so any vulnerability that caused curl to deliver data other than
 that which it should would be noticed by MacPorts immediately and the bad
 data would not be used. And we already don't consider the information we
 send or receive with curl to be confidential—we don't transmit private
 information and many of our mirrors already don't use https. As I have
 said before, it seems likely to me that bundling a newer
 libcurl/libressl/openssl will fix more problems than it causes.

 Users often don't care or even know why something doesn't work, they just
 know it doesn't work and they blame the most obvious source. MacPorts is
 broken on 10.8 and earlier when downloading from many servers, and many
 users have reported this to us. There are some edge cases where our
 buildbot does not mirror distfiles immediately or at all, and sometimes
 it's just busy with other tasks or even offline; during those times, with
 no mirror to fall back on, users can encounter fetch errors. We've even
 had the problem on our own buildbot workers for older OS versions. We've
 identified the cause: outdated OS libcurl/openssl libraries and CA bundle.
 We have a fix: bundle newer ones. This would fix the bug, increase user
 happiness because things will just work for them, and free up the time
 MacPorts developers would otherwise spend responding to user bug reports
 about this problem.


 > > Your objection in comment:3 was that we should not distribute a CA
 bundle. Would bundling an ssl library require us to include CA bundle?
 Doesn't the OS ship with one that we could use? Or is that part of the
 problem—the old OS's CA bundle doesn't contain the information needed to
 trust new sites?
 >
 > The latter, the CA bundle would eventually be too old for new
 certificates to validate.

 Ok, then we could bundle a new CA that MacPorts libcurl/libressl/openssl
 would use, again possibly only for older systems that won't get such an
 update from Apple. We could even think about a way to update it via
 selfupdate that would be separate from MacPorts base releases, if making a
 user wait for a new MacPorts base release might be undesirable. Or we
 could find a way to have MacPorts base optionally use the CA from the
 curl-ca-bundle port.


 > > #51045 was merely the first example of a curl bug that's fixed in
 newer versions that I found to link to this ticket. There's another
 cosmetic issue I found on Leopard that's been fixed. I'm sure there have
 been more bugs fixed in curl over the years.
 >
 > Do you have specific tickets?

 As I recall, curl on Leopard would emit a spurious error message during
 some downloads. As far as I remember this did not affect the ability to
 download the files. I'm not able to find a ticket or other reference to it
 as I don't remember exactly what the error message was so I can't search
 for it now.

 Obviously this bug and the previously mentioned one are trivial but they
 are a reminder that there have probably been many bugs fixed in curl since
 the versions bundled with old macOS versions which our users would benefit
 from if we bundled a newer version. The benefit to us is being able to
 remove code from MacPorts base that works around those old bugs, thus
 simplifying our code and making it easier to maintain. (I currently see 3
 different workarounds for bugs or missing features in various versions of
 curl in our curl.c.) We were able to do the same when we bundled Tcl 8.5
 and could remove the Tcl 8.4 workarounds we had, as I mentioned above
 years ago.


 > I'm not convinced that we should bundle every library we use.

 I'm not trying to propose that. This ticket is only about bundling curl
 and libressl/openssl with MacPorts. I believe this will benefit us and our
 users and I am not convinced that the drawbacks outweigh that.

-- 
Ticket URL: <https://trac.macports.org/ticket/51516#comment:57>
MacPorts <https://www.macports.org/>
Ports system for macOS