Alternatives to Tcl [was Re: [GSoC][Binaries support] Architecture]

[Anders F Björklund]

Jordan K. Hubbard wrote:

>> Many of the current Portfiles are trying to do *way* too much in Tcl, that would be much better off integrated into base proper or implemented as an extension. I think that making ports/specs/recipes into programs, rather than markup with a tiny bit of dynamic content allowed is a mistake. But that's another discussion.
> 
> I agree that it's something of a separate discussion, and I might add that the original concept for MacPort was to have Portfiles be in XML and entirely "passive" in the nature of their data, e.g. no active scripts or otherwise un-auditable operations for security / robustness reasons.  Our thinking was that we wanted to be able to roll back all the operations for a given port when encountering a failure or explicit "undo" request, which is kind of hard when it's just executed a shell command which has just done god-only-knows-what to the system, and I think those goals were extremely worthy.  There were a number of reasons we didn't end up doing that, however, some of which I'll note here in order to inform future discussions on the topic:
> 
> 1. It was hard to express conditional / looping behavior without duplicating a lot of data and/or having to express the conditionals in a clumsy fashion.  You can certainly express predicates in XML with AND, OR and NOT prefixes for each subordinate block, for example, but the resulting overall expression looks a lot more like a conditional in FORTH (which I happen to like but many don't ;-) with an RPN syntax driven by the needs of XML.

Yeah, I'm sure that would have scared even more people away... :-)

Not to mention that it gets pretty hairy when you need to extend.
The biggest advantage of having a real programming language and
not something like make or bash is when you do need to extend it.
Even if using XML (like e.g. ant), it would still need "extensions".

> 2. Humans don't think naturally in XML (see above) and the prototype ports we created were not especially easy to read or maintain, meaning we'd need some sort of "port creator" front end that kicked XML out the back end for novices to be able to create ports, which we thought was important (not everyone who's good at porting software is also comfortable with XML or other markup syntaxes).

That was my reason to prefer YAML, as it was much more readable.

But it would still have embedded scripts, for reasons mentioned.

> 3. Port creators like to take short-cuts and otherwise avoid repetitive labor since the process of porting the software itself is difficult enough without having to spend 2X the time just struggling to re-implement an installation script or make(1) procedure which originally came with the port in an entirely passive syntax like XML.  If you can't use shell commands directly, you can't cut and paste from the original software's "make install" procedure and we figured that would be really unpalatable to a lot of folks (and I think we were right in this respect).  By comparison, a lateral move to slightly different shell commands or built-in procedures like xinstall is a lot easier to grasp.

I think most of the Portfile syntax is very simple to read/write.

And I like being able to use system, even if complained upon...

> I also don't think the problem lies with Tcl itself.  If you look at most Portfiles, they're not actually using Tcl as a programming language in at least 99% of the cases (OK, I just pulled that number out of my ass and I admit it) but rather as a declarative syntax.  The port name is foo, the version is bar, and so on, and that was definitely a deliberate design decision on our part.  Where Tcl is actually used to loop over argument lists or glob files, it's the exception rather than the rule.  I agree that it still makes things problematic from a runtime / packaging perspective, however, and we should probably continue that discussion in the other thread since, as you say, this was actually a premature send on your part. :-)

I'm talking about the other 1%, that try to do "clever" stuff.

A lot of that should move elsewhere, so they're simple again ?


But nowadays I don't have a problem with the Portfiles or the
parser of them using Tcl, and I see base using Tcl as an interim
step until something better comes along. It could still use
Tcl for the ports, even if the rest of it uses something else...

--anders