[MacPorts] #67186: Set Host header when invoking curl

[MacPorts]

#67186: Set Host header when invoking curl
-------------------------+-----------------------------
  Reporter:  ryandesign  |      Owner:  (none)
      Type:  defect      |     Status:  new
  Priority:  Normal      |  Milestone:  MacPorts Future
 Component:  base        |    Version:  2.8.1
Resolution:              |   Keywords:
      Port:              |
-------------------------+-----------------------------
Description changed by ryandesign:

Old description:

> curl versions from 7.37.0 (2014) up to but not including 7.82.0 (2022)
> fail to handle the situation where a web site's canonical hostname ends
> with a period. See https://github.com/curl/curl/issues/8290. Here is an
> example of the problem:
> {{{
> % /usr/bin/curl --version
> curl 7.79.1 (x86_64-apple-darwin21.0) libcurl/7.79.1 (SecureTransport)
> LibreSSL/3.3.5 zlib/1.2.11 nghttp2/1.45.1
> Release-Date: 2021-09-22
> Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap
> ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
> Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos
> Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL UnixSockets
> }}}
> {{{
> % /usr/bin/curl -I https://pyropus.ca./
> HTTP/1.1 301 Moved Permanently
> Date: Mon, 03 Apr 2023 01:17:16 GMT
> Server: Apache/2.4.56 (Debian)
> Location: https://pyropus.ca./
> Content-Type: text/html; charset=iso-8859-1
> }}}
>
> The memtester port is an example that fails to fetch distfiles because of
> this (if the `master_sites` override is removed from the Portfile). It
> also fails livecheck.
>
> It was pointed out to me by the author of memtester that affect curl
> versions can be made to work by setting the `Host` header properly, as
> in:
>
> {{{
> % /usr/bin/curl -H 'Host: pyropus.ca.' -I https://pyropus.ca./
> HTTP/1.1 200 OK
> Date: Mon, 03 Apr 2023 01:18:35 GMT
> Server: Apache/2.4.56 (Debian)
> Accept-Ranges: bytes
> Vary: Accept-Encoding
> X-Frame-Options: sameorigin
> Content-Type: text/html
> }}}
>
> MacPorts should use a regexp to extract the host out of the URL and set
> the Host header every time it invokes curl for any purpose to work around
> buggy curl versions.

New description:

 curl versions from 7.37.0 (2014) up to but not including 7.82.0 (2022)
 fail to handle the situation where a web site's canonical hostname ends
 with a period. See https://github.com/curl/curl/issues/8290. Here is an
 example of the problem:
 {{{
 % /usr/bin/curl --version
 curl 7.79.1 (x86_64-apple-darwin21.0) libcurl/7.79.1 (SecureTransport)
 LibreSSL/3.3.5 zlib/1.2.11 nghttp2/1.45.1
 Release-Date: 2021-09-22
 Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap
 ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
 Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos
 Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL UnixSockets
 }}}
 {{{
 % /usr/bin/curl -I https://pyropus.ca./
 HTTP/1.1 301 Moved Permanently
 Date: Mon, 03 Apr 2023 01:17:16 GMT
 Server: Apache/2.4.56 (Debian)
 Location: https://pyropus.ca./
 Content-Type: text/html; charset=iso-8859-1
 }}}

 The memtester port is an example that fails to fetch distfiles because of
 this (if the secondary `master_sites` entry is removed from the Portfile
 and `--no-mirrors` is used when fetching). It also fails livecheck.

 It was pointed out to me by the author of memtester that affected curl
 versions can be made to work by setting the `Host` header properly, as in:

 {{{
 % /usr/bin/curl -H 'Host: pyropus.ca.' -I https://pyropus.ca./
 HTTP/1.1 200 OK
 Date: Mon, 03 Apr 2023 01:18:35 GMT
 Server: Apache/2.4.56 (Debian)
 Accept-Ranges: bytes
 Vary: Accept-Encoding
 X-Frame-Options: sameorigin
 Content-Type: text/html
 }}}

 MacPorts should use a regexp to extract the host out of the URL and set
 the `Host` header every time it invokes curl for any purpose to work
 around buggy curl versions.

--

-- 
Ticket URL: <https://trac.macports.org/ticket/67186#comment:1>
MacPorts <https://www.macports.org/>
Ports system for macOS