Make options lists instead of strings and make -append more magical

[Rainer Müller]

On 2013-10-08 21:07, Ryan Schmidt wrote:
> Options like configure.args or depends_lib seem in a way magical
> because of the convenience methods MacPorts makes available, but
> they're not as magical as they could be.

> We use commands like configure.args-append to add to options, and
> this makes us feel like options are lists or arrays, but in fact,
> they're just strings. The implementation of handle_option-append is
> quite simple: it just uses Tcl's built-in concat procedure, which
> trims leading and trailing whitespace and combines items with a
> single space. This has undesirable consequences when the option in
> question needs a different type of concatenation.

Depending on the arguments appended, they might be plain string or they
might be lists. I remember we once had problems with {} and "" in
descriptions due to this...

> For example, consider configure.pkg_config_path. Using
> "configure.pkg_config_path-append foo" will append a space and "foo"
> to configure.pkg_config_path, but in fact, pkg_config_path needs to
> be colon-separated, not space-separated. So it becomes incumbent upon
> each maintainer to remember that and to never use
> configure.pkg_config_path-append, and rather to manually concatenate
> onto it using a colon. I'd rather not make maintainers have to
> remember that kind of detail.

I totally agree with this that options should have different semantics.
Clemens already started working on something similar in his base-rewrite
branch by making a difference between a list and a set. For example, the
depends_* options should be treated as sets as it does not make sense to
add the same dependency multiple times. Something like configure.args
should be a list, as the order of arguments can be important or may
contain the same item multiple times.

I guess this commit is best to describe these changes.
https://trac.macports.org/changeset/103064

(Side note: I don't like the overly complex -listsemantics and
-setsemantics modifiers, -list and -set should be enough.)

Adding custom separators for options sounds like another good idea.
However, we would also need to define if and how this separator should
be escaped if it is passed literally.

> Or, consider notes. Notes are supposed to be paragraphs of text. A
> port might define a paragraph or two of notes, then want to provide
> additional notes conditionally, perhaps in a variant. We tend to want
> to separate paragraphs of text by two newlines. But if a port uses
> "notes-append foo" then only a single space and "foo" will be
> appended to the existing notes. There's no way around this: even
> something like "notes-append "\n\nfoo"" doesn't work because concat
> trims the leading whitespace.

Actually, it works for notes because I added special handling for this
case with handle_option_string. It's special because the leading
whitespace of the first line is stripped from all lines. This allows to
indent the notes without indenting the output.

notes {
    Foo
}
notes-append {
    Bar
}
notes-append "\n\nBaz"

This example should work fine and produce the expected output.

However, I noticed there is an edge case where it does not work as you
would expected. That is when the first line is left empty (even no
whitespace):

notes-append {

    Qux
}

This can be fixed by either adding whitespace to indent the empty line,
or add a single notes-append "\n" beforehand.

> I wonder how feasible it would be to treat options as lists instead
> of strings, and then teach MacPorts to use different concatenation
> values depending on which option it is. notes could be concat'd with
> "\n\n", configure.pkg_config_path could be concat'd with ":", if we
> had a configure.path it could be concat'd with ":", and " " could be
> used for others.

I would say such changes demand an increase of the PortSystem version as
it changes the way options are used. This is not a problem as MacPorts
was designed to support multiple different versions of the port system
and every existing Portfile would still work. Furthermore, it would also
remove the need to design this change in a backwards compatible way.

> Trying to include options within a portfile has also been
> problematic. For example sometimes I seem to have to use join or eval
> to get things to work without Tcl inserting incorrect escaping. Would
> be lovely if we could make this process more transparent and
> straightforward. For example, perhaps the hypothetical base procedure
> that would perform this concatenation could be exported for portfiles
> to use when needed.

This depends a lot on where we would do the substitution of the
separator. Either we do it right on -append, but that might make it
harder to -delete the value the same way. The other possibility would be
to do it only if the option is read using the variable ${foo}. Raw
access would still be possible for the port system internally.

Rainer