Google Summer of Code 2009: Mentors needed!

[Jordan K. Hubbard]

On Mar 11, 2009, at 5:21 AM, Joshua Root wrote:

> This was one of Jordan's crazy ideas IIRC, and there's a reason it's
> classified as very challenging. The idea, as I understand it, is that
> you wouldn't be able to uninstall the old version when something is
> linked against it.

Correct, though I don't know how "crazy" of an idea it is to try and  
make any collection involving hundreds, if not thousands, of software  
packages installed on any machine more robust in the face of version  
skew. :-)

More to the point, there are already scenarios today (and far more in  
FreeBSD's ports collection, where the efforts to address this have  
taken on an increasingly desperate and hacky nature) where you simply  
cannot have two desired ports installed at the same time due to  
conflicting, mutually-exclusive dependencies.  Port Foo depends on  
library Bar, version X.  Port Blah depends on library Bar, version Y.   
X and Y are incompatible with eachother, and Foo cannot be easily  
ported to Bar-Y, just as Blah cannot be easily ported to Bar-X (we're  
talking about some old software in many situations, where there's just  
no one available to fix things "correctly" by adapting to the latest  
revision of Bar).  Even if you do get volunteers to do the heavy  
lifting required once, after it breaks for the 2nd or 3rd time (and  
this is not so rare a scenario where things like Tcl or OpenSSL have  
been concerned, just to cite 2 quick examples) it quickly gets old and  
people are more tempted to walk away and simply let things rot in  
place, which is a bummer if you were one of the users who really liked  
being able to use Foo or Blah.

Naturally, if we had more control over the filesystem lookup layer (or  
could use dyld interpositioning in such a way that every single  
relevant process in a given session got successfully interposed) then  
we could solve this at the "right" layer by simply redirecting Foo and  
Blah's requests for Bar files such that they went to the right place,  
without having to build either Foo or Bar in any particularly special  
way, but that's an even "crazier" suggestion and one likely to be so  
involved that nobody is going to want to even think of tackling the  
problem.  By contrast, simply making sure that everything hooks into  
the depot location rather than looking directly in ${prefix} seems a  
lot simpler.

Rainer also points out the upgrade problem, which is of course an  
important point.  If you want to upgrade Bar (either X or Y) then you  
need to figure out some way of upgrading it in-place or settle for the  
trade-off of simply creating Bar-Z and retrying the builds of Foo and  
Blah against it, hoping that maybe the Bar folks have solved the  
compatibility issues.  This is not a heck of a lot different than  
bumping the version / revision number on a dylib, however, since you  
have to also upgrade *those* in-place in order to get the benefit of  
some security update or other issue important enough to accept the  
risky proposition of upgrading a dylib in-place.  In short, my crazy  
idea does not introduce any issues we do not have already today, and  
solves a number of issues for which we have *no* solution at present.

> This project of course requires the ability to depend on specific
> version ranges of ports (being able to exclude both too old and too
> new), and likewise with variants. There would also need to be a way to
> manage rebuilds when existing software would work with a newer  
> version,
> and you want to get rid of the old one. (And other such interactions
> involving variants etc.)

Yep.  I think what we ultimately need is a tuple containing port,  
version, variant-list which gets looked at in satisfying any  
dependencies, and we make the "binding precedence" follow the same  
order:  First you look for port, then you look for version, then you  
look for (if the depending port even specifies a preference) the  
specific variant.  If the port has declared one or both of the latter  
two items as "soft" dependencies, or has wild-carded them somehow, you  
can also fall back to a non-exact match rather than forcing a build of  
that specific tuple.  Of course, since things also depend on the depot  
locations rather than the bits in ${prefix} in my crazy idea, it also  
means that you can have multiple variants of a given port installed at  
the same time, so all you're burning is some extra disk space and CPU  
time in the interests of a more exact functional match.

- Jordan