Thoughts on switching our archive compression method
ryandesign at macports.org
Tue Sep 22 14:42:48 UTC 2020
On Sep 22, 2020, at 07:58, Vincent wrote:
>> It would be nice if we could easily switch our precompiled archives from bzip2-compressed tarballs (tbz2) to better compression methods as they become available. For example, xz-compressed tarballs (txz) would be better today. OS X 10.9 Mavericks and later has built-in support for xz compression (https://trac.macports.org/ticket/56237) so we could use that to avoid needing to use the xz port or needing to bundle a copy of xz (https://trac.macports.org/ticket/52000).
> How much is gained by switching from tbz2 to txz?
It varies of course but here are a few examples:
$ bzcat /opt/local/var/macports/software/texlive-latex-extra/texlive-latex-extra-54584_0+doc.darwin_17.noarch.tbz2 | xz -9 > texlive-latex-extra-54584_0+doc.darwin_17.noarch.txz
$ ls -l /opt/local/var/macports/software/texlive-latex-extra/texlive-latex-extra-54584_0+doc.darwin_17.noarch.tbz2 texlive-latex-extra-54584_0+doc.darwin_17.noarch.txz
-rw-r--r-- 1 macports wheel 565981279 Aug 23 23:16 /opt/local/var/macports/software/texlive-latex-extra/texlive-latex-extra-54584_0+doc.darwin_17.noarch.tbz2
-rw-r--r-- 1 rschmidt wheel 481850300 Sep 22 08:56 texlive-latex-extra-54584_0+doc.darwin_17.noarch.txz
(xz is 85% of bz2 size)
$ bzcat /opt/local/var/macports/software/go/go-1.15.2_0.darwin_17.x86_64.tbz2 | xz -9 > go-1.15.2_0.darwin_17.x86_64.txz
$ ls -l /opt/local/var/macports/software/go/go-1.15.2_0.darwin_17.x86_64.tbz2 go-1.15.2_0.darwin_17.x86_64.txz
-rw-r--r-- 1 macports wheel 124756642 Sep 14 09:09 /opt/local/var/macports/software/go/go-1.15.2_0.darwin_17.x86_64.tbz2
-rw-r--r-- 1 rschmidt wheel 86893084 Sep 22 09:32 go-1.15.2_0.darwin_17.x86_64.txz
(xz is 70% of bz2 size)
$ bzcat /opt/local/var/macports/software/python38/python38-3.8.5_1+universal.darwin_17.i386-x86_64.tbz2 | xz -9 > python38-3.8.5_1+universal.darwin_17.i386-x86_64.txz
$ ls -l /opt/local/var/macports/software/python38/python38-3.8.5_1+universal.darwin_17.i386-x86_64.tbz2 python38-3.8.5_1+universal.darwin_17.i386-x86_64.txz
-rw-r--r-- 1 root wheel 28496603 Sep 6 01:16 /opt/local/var/macports/software/python38/python38-3.8.5_1+universal.darwin_17.i386-x86_64.tbz2
-rw-r--r-- 1 rschmidt wheel 16136624 Sep 22 09:36 python38-3.8.5_1+universal.darwin_17.i386-x86_64.txz
(xz is 57% of bz2 size)
So I think we could save ourselves and our mirror providers, CDN, and users some disk space and bandwidth by switching to xz. bz2 was the best available built-in compression on Mac OS X 10.6 when we started doing binary archives but there are better options now.
More information about the macports-dev