Mirror unpacks distfile before sending

[macports at raf.org]

Dave Allured - NOAA Affiliate via macports-users wrote:

> On Tue, Apr 2, 2019 at 10:38 PM Ryan Schmidt <ryandesign at macports.org>
> wrote:
> 
> > On Apr 2, 2019, at 23:21, Bill Cole wrote:
> >
> > > On 2 Apr 2019, at 23:45, Dave Allured - NOAA Affiliate wrote:
> > >
> > > [snip]
> > >>
> > >> I have never before seen this sort of fradulent behavior, silent
> > >> unpacking,
> > >> from either an http hosted data site, or the curl command.  Can anyone
> > >> else
> > >> confirm this weird download behavior from that facebook.net mirror?  Is
> > >> there an alternate explanation?
> > >
> > > Yes.
> > >
> > > It sounds like the mirror may have a wrong-ish implementation of HTTP
> > > Compression. (See https://en.wikipedia.org/wiki/HTTP_compression) I've
> > > seen similar oddness dependent on the client request headers.
> > >
> > > This might be something to bring to the attention of Facebook or GNU,
> > > since that's a GNU mirror.
> >
> > I agree, it is a misconfiguration of the Facebook mirror server. Dave,
> > could you please report it to them?
> >
> > Here is what the headers should look like, from ftp.gnu.org:
> >
> > $ curl -I https://ftp.gnu.org/gnu/groff/groff-1.22.4.tar.gz
> > HTTP/1.1 200 OK
> > Date: Wed, 03 Apr 2019 04:32:52 GMT
> > Server: Apache/2.4.7 (Trisquel_GNU/Linux)
> > Strict-Transport-Security: max-age=63072000
> > Last-Modified: Sun, 23 Dec 2018 15:06:58 GMT
> > ETag: "3f2208-57db1d4efd451"
> > Accept-Ranges: bytes
> > Content-Length: 4137480
> > Content-Security-Policy: default-src 'self'; img-src 'self'
> > https://static.fsf.org https://gnu.org; object-src 'none';
> > frame-ancestors 'none'; child-src 'self' https://static.gnu.org
> > https://static1p.gnu.org https://static1p.fsf.org
> > X-Frame-Options: DENY
> > X-Content-Type-Options: nosniff
> > Content-Type: application/x-gzip
> >
> > Here are the headers Facebook's mirror is sending:
> >
> > $ curl -I http://mirror.facebook.net/gnu/groff/groff-1.22.4.tar.gz
> > HTTP/1.1
> > <http://mirror.facebook.net/gnu/groff/groff-1.22.4.tar.gzHTTP/1.1> 200 OK
> > Date: Wed, 03 Apr 2019 04:33:02 GMT
> > Server: Apache
> > Last-Modified: Sun, 23 Dec 2018 15:06:58 GMT
> > Accept-Ranges: bytes
> > Content-Length: 4137480
> > Connection: close
> > Content-Type: application/x-gzip
> > Content-Encoding: x-gzip
> >
> > Note the incorrect "Content-Encoding: x-gzip". That header means that the
> > data has been gzip-compressed for transmission by the server, and the
> > client should un-gzip it before presenting it to the user. But that is not
> > what anybody wants here. We want the client to receive the original
> > unmodified .tar.gz file.
> >
> 
> Bill and Ryan,
> 
> Thanks for the expert diagnosis.  I will report this.  Can anyone recommend
> optimal bug reporting channels for facebook.net and GNU, without requiring
> a Facebook account?

it's wierd. i'm seeing the same Content-Encoding header
but curl doesn't un-gzip the download for me. neither
/usr/bin/curl (7.43.0) nor /opt/local/bin/curl (7.64.1).
neither does wget. i wonder what the difference is.

cheers,
raf