"cask" ports just keep on rolling in...

Sat Feb 6 09:46:00 UTC 2021

On Feb 6, 2021, at 02:28, Ken Cunningham wrote:

>>> This kind of port just repackages the DMG into many tgz archives.
>>> 
>>> It’s wasteful. People want them.
>>> 
>>> What we should have instead of this is some kinda tech that
>>> 
>>> 1. downloads the DMG
>>> 2. installs the app
>>> 3. records some way of uninstalling everything
>> 
>> MacPorts already does all these things... The submitted Portfile works fine, presumably. There's no need to reinvent the existing MacPorts functionality that does all these things.
> 
> Well, IMHO there is, but I’ve looked at it quite a bit.
> 
> Look back a month or so and see my post about a “cask” port for SageMath for an idea of why this won’t work (well) in general.

I had not seen that part of the previous discussion because I stopped following it, but going back and searching for "sagemath" in those messages now, I still don't see any reason why what MacPorts can already do would not work for these types of ports.

In that example, the sagemath dmg was 1.5GB and the installed size was 4GB. This is a stupid size for a software package to be, but that's up to its developers. Hopefully there are not more than one or two prospective ports that are so exorbitantly large, so I would not let this extreme example dictate what we do.

The complaint was that MacPorts extracts that 4GB into the work directory, then compresses the contents of the destroot into a package, then activates the package, using unnecessary time and disk space. Let's start by acknowledging that this works just fine, so it does not need to be changed. It is true that this takes a little time and temporarily disk space, but as you know the reason why MacPorts does this is so that the user can deactivate and later re-activate the port as needed.

It was proposed there that MacPorts should instead somehow use the original dmg distfile as the archive, skipping the normal extract, patch, configure, build, destroot, install (create package) and activate (extract package) steps. Based on what I've read so far, that doesn't sound like a great idea, since you'd be replacing a large amount of already-working MacPorts base code with new code that you'd have to test.

You're either requiring the user to keep that distfile around in order to be able to reactivate the port (which would be highly unexpected), or you have MacPorts copy the distfile into the software directory, which could be done, but then the port has to do all of that stuff during the activate phase, has to contain code that knows what files to pull off the dmg and where to put them at activate time, losing the safety net that destroot provides, making development of the port more difficult. When developing a port you want to be able to "sudo port destroot" and then check the contents of destroot to make sure everything is correct, and you want to be able to "sudo port clean" to delete it all with the assurance that nothing outside of the work directory was touched. We already have that today; I don't see how the proposed new mechanism would allow that to continue to be the case.

A point was raised that the original dmg is compressed already, therefore keeping and using it was desirable, but I have to point out that dmgs can be uncompressed, or could be compressed with less-efficient methods than our bz2 (though, granted, these days most of them probably are compressed, some probably with more-efficient methods like lzma). There's also the possibility that the original dmg contains additional files that we don't care about, so requiring the user keep the original dmg around might be requiring them to keep unneeded files around.

Another suggestion was raised there of preventing distfiles from being mirrored. I see no reason why that should be tied to this type of port. On the contrary, if we don't provide packages, then we absolutely want to mirror distfiles (license permitting, of course) for all the usual reasons, such as in case the original site is offline. With most ports, most users use our packages and it doesn't matter if the distfile is inaccessible. But with these ports, if we are talking about not providing our own packages and forcing all users to use the original distfile, then it very much matters if the distfile is inaccessible.

> It’s not that it’s too much work. It’s that these things are very simple, and people submit them but don’t know all these details you mention.

They won't know about the hypothetical new portgroup either unless they read about it or we tell them about it. So it comes down to a matter of keeping our documentation current and readable, and I'm the first to admit that our documentation is not in the best shape that it could be in.

> this won’t work for larger ports too well (see SageMath message for an extreme example)

> we don’t want to run rev-upgrade on them (surprises, surprises, surprises — eg SageMath again)

This might be a valid point. If so, we might enhance MacPorts to provide a way for a port to opt out of rev-upgrade. That would be simple enough. On the other hand, are you saying that sagemath is reported as broken by rev-upgrade, and that either we want to accept that brokenness? (why?) or that rev-upgrade is wrong? (in what way? why don't we fix that?)

It is not inconceivable, in fact it seems likely to me, that a port developer might want to patch a binary so that it finds MacPorts libraries. Given the popularity of Homebrew, someone might distribute a binary that links with Homebrew openssl, for example, and we might want to edit that with install_name_tool to point to MacPorts openssl instead. Or the binary we install might include libraries, and we would want to change their install_names to MacPorts paths. I do this in the oracle-instantclient and libxl ports for example. I would want rev-upgrade to let me know if something is amiss with that.

> we want to be able to run a pkg-installer into some destination…. and nobody except three people know how to do that right

True, some ports might want to install using the installer. macOS includes the `installer` command for doing this. I don't recall the right arguments offhand but I'm sure it's described in the manpage. If it is though that a portgroup that runs installer for you would be an easier interface to this, by all means create one. Or the functionality could be added to base, if that's more appropriate.

It would be nice if MacPorts base "decompress this file" functionality were more generalized (automatically figure out what program to use) and abstracted out to be available to ports at any time. Base already contains two separate implementations, one to extract distfiles and one to extract archives. Adding installer pkg/mpkg files to the list of file types that we know how to extract would be reasonable. But installers can also contain scripts which it might be important to run, and using `installer` would be the way to ensure that that happens.

> we want to be able to uninstall all the cruft the software installs in ~/Library, ~/Preferences, etc easily

I did not see in the messages about sagemath any mention of ~/Library or ~/Preferences cruft. ~/Preferences is where user preferences go; just as with settings files that other ports might install into /opt/local/etc, it would be unexpected for a port to delete them. I would have to take any additions to ~/Library on a case by case basis, but I would assume that any files that any program installed by a port puts into ~/Library after installation are put there intentionally and that it is right and proper for it to be doing so and that it's not up to us to delete them. This would be no different from a user installing an app manually by dragging and dropping. Anything that a port installs during installation would be handled as part of destroot of course, though of course it wouldn't be installing things in user-specific paths like ~.