Mirror unpacks distfile before sending

[Ryan Schmidt]

On Apr 4, 2019, at 11:45, Dave Allured wrote:

> That *is* relevant.  My Mac is behind an institutional firewall, so this might be aggravating the problem.  Also I am not able to duplicate the headers as Ryan showed using curl -I, not with the facebook.net URL.  I will ask our network admins about this.

What headers are you seeing then? Sounds like your corporate firewall/proxy may be rewriting headers and content. This of course is one of the reasons why so many sites are switching to https -- to prevent such often broken rewriting from taking place.

I still think there is a misconfiguration of the facebook.net mirror. It is sending the "Content-Encoding: x-gzip" header. That means the server is claiming it has gzipped the content prior to sending it and the client should un-gzip the content upon receiving it. So one of two things happened:

either 1. Facebook gzipped a gzip file, sent it, and the client should remove the outer gzipping it to leave you with a gzip file. If this is what's happening, it's stupid because there's no reason to gzip an already-compressed file; Facebook should stop doing that. Perhaps your firewall/proxy is misinterpreting this situation and it is removing all layers of gzip compression, not just the first one.

or 2. Facebook is transferring the original gzip file to you but claiming that it compressed it and that the client should ungzip it. If this is what's happening, it's lying, and I would expect any client to (undesirably, in this case) decompress it before saving it. I can confirm that curl does decompress it before saving it if the --compressed flag is used; I guess curl doesn't react to the Content-Encoding header unless it requested a compressed response in the first place. Since your client (not your your firewall/proxy) made the original request, maybe the firewall/proxy doesn't know (or doesn't keep track of) whether the request was for a compressed or uncompressed resource, and it just looks at the Content-Encoding header to decide, which seems like a reasonable decision. It points out an additional misconfiguration of the facebook.net server: they're sending (or claiming to send) a compressed response even when the client did not request it; Facebook should stop doing that and should send what the client asked it to send.