PortGroup directory hierarchy/priority

[René J.V. Bertin]

On Monday April 04 2016 07:34:32 Clemens Lang wrote:

>What kind of information are we talking about here? Compiler flags?
>Paths?

Typically paths defined through variables indeed, or configure settings, plus the occasional pre- or post- block. And dependency info, of course (e.g. declaring a dependency on Qt is supposed to by done by including the Qt portGroup). There may also be variants; if my Qt5 port is installed, the accompanying Qt5 PortGroup declares and makes default a variant for every dependent which ensures users will not install binary packages that were built against the "other" Qt5 port.

>That would potentially cause ports to build different depending on which
>packages you have installed in your environment, which is not
>reproducible.

Yes, but that is already the case when a user replaces a mainstream port with one from an alternative source.
There's only one way to enforce the reproducible build principle all the way, and that's to remove the whole possibility of building from source so that users can only install official, binary packages. If you cannot go there (because it'd mean too many sacrifices) you just have to accept that the reproducible build principle can only go so far in making support easier, and live with that.

>The behavior of a port might even be affected by ports
>that are not in its transitive dependency tree. 

Can you think of an example other than the one below?

>Additionally, this could
>cause a support burden for us…

Yes, but ...

> Reporter: Here's this log, the build of A fails.

... if that log contains a clear indication when it takes a PortGroup from the central location the support request can be redirected easily enough, esp. if the central location is populated with symlinks and the log indicates the target (via [file normalize filename]?).

Counter example and the reason I started this thread: the log from a user who'd followed my installation instructions but not to the point of copying the appropriate PortGroup files into the main tree (because as a newcomer he missed part of my instructions). I'm still not sure I understand exactly what happened, but it sent me off on a wild goose chase for bugs in my code rather than make the instructions more explicit.
In this case that whole support episode happened on the kde-mac ML (or mostly; I did get a constructive remark from Bradley) but it could just as well have come in on trac.

> Reporter: No, the Portfile is pristine and my overlays don't have the
>           port… (but a random other port installs an override for a
>		   PortGroup that is used by this port).

I wouldn't call that a random other port. You can imagine all kinds of scenarios where things go wrong, but that's not taking into account the fact that somewhere there's a developer who maintains this tree, and who would have been bitten by the above random behaviour..
I'd say that the example given here corresponds more to a user who starts playing around with his own custom ports tree, and creates a PortGroup with a name that's already taken. If s/he does that with the current state of things, that PortGroup could end up in the main _resources directory, and then the situation could be much harder to debug.
Worse: at some point such a "rogue" PortGroup might actually be committed. To give an example: I don't have commit access myself, but a while ago I had a few minor but useful changes to a PortGroup belonging to a middleware used by a whole family of ports. The official maintainer was unreachable but I'd been in contact and testing the changes with someone else, who eventually committed the new version. Now imagine that this had happened with a less well thought-out change.

>You might not even be able to get such a bug fixed with the developers
>of such a PortGroup, e.g. because the conflict might be between two
>non-standard port trees. 

Again, that can already happen when unsuspecting users start copying PortGroup files by hand. At least if that goes through MacPorts' install mechanism a conflict error will be raised during activation. Also, a blacklist could be used to prevent a port from installing known central PortGroups that belong to MacPorts.

>The more I think about this, the more I get the
>impression a PortGroup should only be affecting its port tree, and
>nothing else.

To draw an analogy, you prefer a Prohibition-like approach to a more permissive one that provides the tools to achieve do certain things in a controlled way?

The more I think about it, the more I thing that my idea of an additional central priority location has benefits that outweigh the potential risks - even more so if it has to be activated through sources.conf or macports.conf . To refer back to the commit example above: with this idea in place it would have been much easier to put up the port and its PortGroup for testing in the wild by many more users.

And just to be sure this stays under the right lighting: I'm arguing all this with alternative or improved ports and PortGroups in mind that I'd hope to see committed one day sooner rather than later. It's been my experience that there are maintainers of the original ports who might encourage the implementation of new features (if they react at all) but then drop the ball, presumably when they realise they'd have to test things and/or that they'd want to reimplement it their way. That's not to point accusing fingers (I think I've been in similar shoes and can understand what's going on).
Still, it's frustrating when you're sure you have improvements on offer but depend on others to get them committed ... and there's no easy way to show that those improvements aren't just for savvy users but also those who are hardly comfortable following the instructions to add a custom port tree.