Breaking long dependency chains

[Dan Ports]

On Sun, Mar 18, 2012 at 07:00:13PM +0100, Freek Dijkstra wrote:
> One of the reasons I see people moving away to package managers like
> Homebrew is that they don't like the many dependencies that come with
> MacPorts.

I agree that this is a problem. Many people have complained about this,
and it's a legitimate gripe -- there's a good chance installing some
port is going to pull in some dependencies you really didn't expect.

The question, of course, is what to do about it... as you note, most
dependencies are there for some reason, so the hard part is finding the
ones that we really can avoid.

> I recall a thread from October 2008, where someone was surprised by the
> depencies of rubber, a small Python script for compiling LaTeX
> documents. At that time it has 142 dependencies. I just re-checked. It
> now has 168 dependencies, including Perl, subversion, openssl, xorg.

Well, let's take a look at rubber's dependencies and see what we can
do. (It, and texlive, happen to be a particular interest of mine...)

Here are my observations:

 - the gtk2 dependency only comes up because poppler requires it, and
   actually it doesn't really need to. I've removed that dependency now,
   thanks to a patch from Rainer. That actually takes 35 dependencies
   out of the picture, some of which are fairly large, so that's a start.

 - around 45 of these dependencies are needed to satisfy build
   dependencies only. If we were installing from binaries, we don't
   actually need those. But does our dependency engine know that we can
   skip build dependencies if archives are available? I'm not sure.

If we exclude those, we're down to 86 dependencies, which is still a
lot, but looking more reasonable (esp. considering that 30-40 of those
are various pieces of texlive itself that happen to be broken into
separate ports).

There are still some surprising things in there, like perl5, python27,
and their dependencies...

Now, I see that rubber depends on a MacPorts python. Does it really
need to? It's evidently OK with that being any version of Python (it
has variants for python24 through 27), and it's not installing or
depending on any Python modules. Is there any reason it shouldn't just
use /usr/bin/python? I'm trying this and it seems to work fine...

I think that reasoning applies to a bunch of other ports that need
*some* version of perl or python. If they don't need any special
modules or otherwise particularly care which version it is, why not use
the system perl/python? For example, I'm wondering whether autoconf
really needs a MacPorts perl. More generally, if something depends on
'perl5' rather than a specific version, we should ask whether the
system perl is OK.

(Some of this might not apply to some really old systems, e.g. whatever
version of Python is provided on Tiger, but we shouldn't put that cart
before the horse...)

> Some suggestions:
> * Reduce the number of dependencies. A good example is Python, which no
> longer depends on GTK (it is no longer required to specify the
> no_tkinter variant)

Sounds good to me, but easier said than done. :-) I've been trying to
keep an eye out for dependencies that should be removed or made
optional; if you have some specific ones in mind, please let us know!

> * Pick the least inclusive variant by default. In particular, do not
> include gtk or x11 dependencies if not required.

I'm not too keen on a solution that involves more variants given that
variant dependencies are a problem for us, but it's possible some
things could be moved into subports.

> Let me think out-of-the box here.
> * Keep track of manually installed and automatically installed packages.
> This ensures that it easier to start over, and that no
> no-longer-required dependencies remain installed

I didn't quite follow this, but if you meant distinguishing ports the
user requested to install from those that were installed to satisfy
dependencies, we can already do this. See `port echo requested`, and
`port echo leaves` for dependencies that are no longer required.

> * provide binary packages for a few packages that are installed
> all-the-time. This includes perl, python, xorg.

We do have binary packages available for some ports, particularly
common ones. For example, in the case of rubber, all but two or three
of its dependencies are available as packages. There are some license
issues to be resolved with the others.

Unfortunately, these packages are currently only available on Snow
Leopard, which is becoming increasingly less relevant. I don't know
when we can expect Lion packages to become available. (This is a source
of some frustration for me.)

> * do depend on Mac-based tools for fetch dependencies and (in some
> cases) build dependencies. While I do think that MacPort should build
> its own library (e.g. openssl, perl version, etc.), I do not think that
> it should build its own tools. E.g. Port uses the Apple-provided
> /usr/bin/tclsh. Why would netpbm need to *compile* subversion just to
> *fetch* netpbm?

We do, in some cases. `port info` and `port deps` don't make this at
all clear, but the system /usr/bin/svn suffices for that subversion
fetch dependency.

There are a couple other cases where we do the same thing; maybe there
should be more. One problem with doing this is that these have a habit
of changing between OS/Xcode versions or even disappearing entirely --
Xcode 4.3 removed bsdmake, autoconf, libtool, and friends.

[I have some other notes about specific dependencies that we ought to
take a look at, but will save those for later.]

Dan

-- 
Dan R. K. Ports              MIT CSAIL                http://drkp.net/