Mirror unpacks distfile before sending

[Ryan Schmidt]

On Apr 2, 2019, at 23:21, Bill Cole wrote:

> On 2 Apr 2019, at 23:45, Dave Allured - NOAA Affiliate wrote:
> 
> [snip]
>> 
>> I have never before seen this sort of fradulent behavior, silent unpacking,
>> from either an http hosted data site, or the curl command.  Can anyone else
>> confirm this weird download behavior from that facebook.net mirror?  Is
>> there an alternate explanation?
> 
> Yes.
> 
> It sounds like the mirror may have a wrong-ish implementation of HTTP Compression. (See https://en.wikipedia.org/wiki/HTTP_compression) I've seen similar oddness dependent on the client request headers.
> 
> This might be something to bring to the attention of Facebook or GNU, since that's a GNU mirror.

I agree, it is a misconfiguration of the Facebook mirror server. Dave, could you please report it to them?


Here is what the headers should look like, from ftp.gnu.org:


$ curl -I https://ftp.gnu.org/gnu/groff/groff-1.22.4.tar.gz
HTTP/1.1 200 OK
Date: Wed, 03 Apr 2019 04:32:52 GMT
Server: Apache/2.4.7 (Trisquel_GNU/Linux)
Strict-Transport-Security: max-age=63072000
Last-Modified: Sun, 23 Dec 2018 15:06:58 GMT
ETag: "3f2208-57db1d4efd451"
Accept-Ranges: bytes
Content-Length: 4137480
Content-Security-Policy: default-src 'self'; img-src 'self' https://static.fsf.org https://gnu.org; object-src 'none'; frame-ancestors 'none'; child-src 'self' https://static.gnu.org https://static1p.gnu.org https://static1p.fsf.org
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
Content-Type: application/x-gzip



Here are the headers Facebook's mirror is sending:


$ curl -I http://mirror.facebook.net/gnu/groff/groff-1.22.4.tar.gz
HTTP/1.1 200 OK
Date: Wed, 03 Apr 2019 04:33:02 GMT
Server: Apache
Last-Modified: Sun, 23 Dec 2018 15:06:58 GMT
Accept-Ranges: bytes
Content-Length: 4137480
Connection: close
Content-Type: application/x-gzip
Content-Encoding: x-gzip


Note the incorrect "Content-Encoding: x-gzip". That header means that the data has been gzip-compressed for transmission by the server, and the client should un-gzip it before presenting it to the user. But that is not what anybody wants here. We want the client to receive the original unmodified .tar.gz file.