alternative/leaner installer creation algorithm: GSoC idea?

[René J.V. Bertin]

On Saturday May 13 2017 15:41:30 Rainer Müller wrote:

Hi,


> Hm, but everyone will get this many dependencies when they do the
> regular 'sudo port install rkward'. Why would this only be a specific
> problem for .pkg distribution?

It's my understanding that .pkg installers are meant for situations where you don't want your users to have to do a full-fledged MacPorts install. That's specifically the case here, where rkward is intended to be packaged for easy installation by students.
Also, with a regular `port install` packages are pulled in and installed one by one, you don't have to wait to fetch a huge download.

> Maybe I am missing something, but wouldn't you end up with the exact
> same list of ports by following depends_lib/depends_run as it is done
> now? In which case do you expect a difference with your algorithm?

Maybe you will indeed if you include all runtime dependencies. The difference should come into play when dependencies have variants, and of course in cases where alternative ports are supported.

I *think* that `port mpkg foo` will build all of foo's dependencies with their default variant and/or variants they share with foo. For instance, rkward depends on R with by default installs with +x11 and +builtin_lapack; that's what you'll probably get in the pkg. Rkward will work just fine though without X11 support in R and with +accelerate, which should decrease R's size and get rid of most if not all X11 stuff. By reverse-engineering rkward's (foo's) port dependencies and bundling the ones that are actually installed you should have a better chance to create a snapshot of what you have installed.

You also get the actual dependencies, without any dependencies that port maintainers declared as depends_lib instead of depends_build ... and evidently also dependencies that somehow were missed.

How much difference it really makes remains to be seen in practice, but you should at least lose the depends_build ones if they are currently being included.

I'm not perfectly certain about the handling of runtime dependencies either. Maybe it should be left to the operator to decide which to include, based on a list presented when invoking the package command with `-vy`; in that case it should be possible to list multiple ports for packaging into a single pkg with a single command.

> Taking your example above, you would still go from rkward to kdelibs4 to
> strigi to ffmpeg.

Indeed.

> Side note: please do not create and distribute .pkg installers for
> /opt/local. That will inherently cause conflicts with a regular MacPorts
> installation.

Yes, I know.

> Isn't the whole point of creating installers to be able to install
> software without a full MacPorts installation? If you already have a
> MacPorts installation, why not just use that?

I can think of situations where you want to be able to distribute binary packages without providing "extranet" access or without going through the hassle of setting up a local mirror.

> If I installed such a
> software image without MacPorts and then later want to install MacPorts
> this will not work out.

This is a bit of an orthogonal issue: installing a (compatible) software image tarball as if it came from a MacPorts build bot mirror. It should be possible, I have managed sometimes by putting one of my own images into distfiles/incoming/verified or some such location ... and it's always possible to simulate the destroot phase and unpack an image into `port work foo`/destroot. IOW, as long as you use existing install routines you shouldn't be borking anything. You may end up with missing dependencies, but that should be easy enough to repair.

> There once was the idea to include a minified version of MacPorts inside
> the .pkg to be able to execute the Portfile for post-activate hooks and
> register it ad-hoc in the registry without a full MacPorts installation
> (back in the DarwinPorts days, dp-lite) However, this was abandoned
> because just getting the essential parts was  not possible as there is
> too much complexity.

It's true that installing into an existing install isn't the main problem here, and should be optional ("destination prefix exists AND is MacPorts => use it ELSE just install already").

OTOH, MacPorts base is small enough, certainly when you compare it to the footprint of a typical port with a certain complexity. I know it can be built as an installer (it's how I install my tweaked "LinuxPorts" version on Linux) so it shouldn't be impossible to include the base install in a mpkg. Then, if you implement a way to do `port install <binary-software-image.tbz2>` creating an installer should become a lot faster because it no longer requires building everything.

R.

> Rainer