Packages Not [was Re: ambivalence about fortran (was Re: numpy & non-Apple gcc?)]

[Jordan K. Hubbard]

On Sep 20, 2010, at 2:09 PM, Jeremy Lavergne wrote:

>> I'm sorry, but archives != packages, so in reality, the port(1) command is not creating packages in that scenario at all.  Archives are missing a bunch of the necessary metadata to quality as packages, the simplest example of which are deletes.  If you upgrade a port simply by extracting newer and newer archives, you will gradually accumulate cruft until such time as certain ports actual fail to work due to legacy bits being detected (and acted upon) when newer versions of the software were written with the assumption that those files were long gone.  You can potentially create a deletion list simply by "diffing" two archives, but that simply leaves you with the next bit of missing metadata, namely the files to explicitly migrate from version X to version X'.  There are also the post install actions to consider (add a role account user, run a special post-install script, etc) which archives do not provide.  Archives are essentially "dumb", and even the über-simplistic {Free,Net,Open}BSD package manager makes its .tar.gz package files "smarter" by including a special manifest file (+CONTENTS) that has a whole bunch of possible actions embedded in it.
>> 
>> You're a long way from binary packages with just archives alone, I'm afraid. :(
> 
> Archives include the build folder, and the Portfile. To say we'd ruin people with archives you'd have to also agree we already ruin people with the way we run MacPorts entirely.

Please don't confuse passive data with active data.  Sure, the archives can and do include the original "port" metadata, but they don't actually include the "package" metadata necessary to make that port work stand-alone in the absence of macports (which, again, is a goal here since macports drags in an entire devtools suite and still puts the burden of building some amount of software on the user).

That part is either not been done, or done in a limited way for a few popular package formats like .pkg, .rpm and .deb (though some of those package formats have bitrotted some and probably need updating).  This translation code between "port metadata" and "package metadata"  is limited since it only knows how to translate a certain number of runtime actions, like foo depends on bar, or foo/blah should be installed with certain permissions, it doesn't know how to translate the actual post-installation actions that many ports have inside those Portfiles.   That is why those package targets have largely bitrotted - they only do the job for a certain subset of the ports collection and nobody quite finished them all the way such that all ports could be truly packageable.

I also never said we were "ruining" people, I simply said that if you were already required to install MacPorts itself (the infrastructure and ports files) then you are providing MacPorts, same as always, and not a public package collection.  Just making the archives available is more of an optimization which the project could, someday, offer to its users as a way of skipping some of the build process for certain popular projects, but they're still going to be calling out to the compiler toolchain and still rebuilding anything that a set of custom variants have been specified for (since you won't have the right "canned archive" available) so it's just an optimization, it's nothing particularly new or innovative.

I'm still waiting for something new, like an actual package collection which does not require anything but a very minimal Tcl runtime which is distributed in the packages itself, making those packages genuinely stand-alone and worthy of the name.

- Jordan