GSoC idea for the binary issue (yet again)

[Anders F Björklund]

Jordan K. Hubbard wrote:

>>> If I'm getting you right, you think the Portfile for the binary archive contains unnecessary information.
>> 
>> Yes. Specifically, a bunch of build-time stuff (and Tcl).
>> 
>> For a binary package format, it would be "better" to have
>> the metadata available in some kind of "markup" instead.
>> (even if it's just a silly text format, and not real ML)
>> That way you can read it from any language, not just Tcl ?
>> 
>> i.e. read +COMMENT/+DESC/+CONTENTS, instead of +PORTFILE
> 
> I have to say that I somewhat disagree with you here.  The appropriate Tcl commands are already available in the Portfile, and extracting this out and/or converting it to another metadata format strikes me as needless additional complexity (and any additional complexity has the potential to outright sink things in this space, considering how long we've been trying to make it happen).

I guess it all depends on the timeframe, and archives vs. packages.

If the scripts are going to remain as-is anyway, a Tcl interpreter
is going to be needed at run-time in order to install the packages.
And since the Portfile contents and the Tcl runtime is there already,
it's "easy" enough to parse that rather than convert to metadata...

Just that I have a few other tools* that work with generic packages,
and that are implemented in Python. If the metadata requires a Tcl
interpreter rather than just reading XML or deb text or rpm headers,
then that makes them less accessible as far as "file formats" go...

* e.g. https://launchpad.net/smart (also at http://smartpm.org)

Currently it supports rpm, deb, pkg, slack, arch, ports formats.
If MacPorts adds binary packages, it could support that as well ?
Including all the query tools and the graphic user interface, etc.
Could still be done even with Tcl +PORTFILE, given enough interest.

But if MacPorts is just a source-based system with binary archives,
rather than a "pizza factory", then the "end product" is not really
suitable for use outside the port(1) tool ? It's *still* a good idea
with binaries, and one could still do a pkg(1) with the same runtime.

I mean something else with the word "package manager", that's all...


* I think that the Portfile should be converted to metadata
* I think that the scripts should be converted to DSL atoms
* I think that the metadata and content need two containers
* I think that signatures should happen on uncompressed data

None of that needs to happen instantly, especially not during
a student assignment... It makes sense to have it happen in 3
"stages", even. First deploy the binary archives, then extend
those "enough" to work with a smaller subset of the runtime.

But as far as theory goes, I think those are worthy end goals.


> Simply archiving the Portfile is extremely simple, the process of executing parts of it selectively (the relevant code being borrowed / imitated straight from ports) somewhat less so but still quite tractable.

No argument about that, which is why I think it was done that way...

> Additionally, I do think that the Portfile in the package has some additional value as "provenance" - you can extract it and get a pretty good idea how the package was built and where its original sources came from.

It's missing the "files" though, so can't really be duplicated as-is.

> It's also quite possible that, over time, MacPorts will add additional actions to the Portfile (I still need to go look to see when and where the "notes" directive is executed) and if we're already bundling it, it should be easy enough to teach pkg to run more stuff out of it vs having to go revisit the intermediate metadata and extend it.

That is true, and has happened before (like with the "license" field)

But I think the current parser chokes on all unknown fields anyway,
so one has to be careful to deploy a new release of "base" first ?
(Since that "PortSystem" version in the Portfile is never increased)
Which tends to make additional port actions a rather rare occurance ?

Though, there is a pending new "test" phase that is dearly needed...

--anders