Thoughts on switching our archive compression method

[Ryan Schmidt]

On Sep 23, 2020, at 04:29, Dan Ports wrote:

> Also, bz2 is particularly slow at decompression, so even xz is likely
> an improvement there.

As far as compression/decompression time goes, the normal gzip, bzip2 and xz tools are single-threaded, as far as I know, so there is an opportunity to go faster by using parallel processing.

For bzip2, you can install lbzip2 and configure MacPorts to use it; it then compresses and decompresses in parallel, even bz2 files that were not created with lbzip2. I have been using this on my Mac for years.

For xz, there's pixz. It sounds like xz files cannot be decompressed in parallel unless they were created with pixz. We could install pixz on the buildbot servers and use it to create the archives. This would improve compress/decompress performance on the buildbot servers, and users could choose to install pixz to get improved decompression performance or we can consider bundling it with MacPorts base in the future; it is BSD licensed.