Using xz by default for compression

[Ryan Schmidt]

On Jan 25, 2015, at 3:27 PM, René J.V. Bertin wrote:

> On Sunday January 25 2015 14:29:01 Daniel J. Luke wrote:
> 
>> and how much more time does it take to compress/decompress?
> 
> Decompression is not noticeably slower or faster, compression is. I'd say that shouldn't be an issue for the build bots, and every user can decide for him/herself whether it is locally.
> 
> I'm not working on patches because the support is already there, and requires a point edit in macports.conf to be activated. I'm not familiar with installer development, otherwise I'd propose a patch presenting the user with the choice between xz (slower but more space-efficient compression) and bzip2.

I'm in favor of making xz the default (there is no need to ask the user about this), the trick would be handling existing installations, which have bz2 archives, without problems, and figuring out how best to handle the pre-built binary packages situation (either write a script to recompress everything as xz, or find a way to have MacPorts check for both).

One thing to consider is that we somewhat recently made MacPorts base capable of using pbzip2 if it is installed, a parallel bzip2 implementation which uses more than 1 processor core for compression of bz2 files, and uses more than 1 processor core for decompression of bz2 files created with pbzip2. I worked with the developer of pbzip2 recently to improve its build system and to report an intermittent crashing bug, which got fixed, and this now seems to work stably on my system, where I don't use the binaries from the packages server. However since our buildbot builders do not have pbzip2 active, the archives created by the buildbot builders are not able to be decompressed more quickly by pbzip2 than by bzip2, so it is of limited value. I had considered bundling a copy of pbzip2 with MacPorts to solve this. There seem to be some parallel xz implementations; we should see if any of them is stable enough, and using a suitably compatible license, to consider being included in MacPorts.