Tcl removes a level of backslashes when accessing a string as a list

[Rainer Müller]

On 02.02.19 02:52, Ryan Schmidt wrote:
> You'd think I'd understand tcl's lists and strings by now. But no. A list is a string and a string is a list. Or maybe it isn't.
> 
> I'm having trouble with livecheck.regex. Since it's a regular expression, it's pretty common for it to contain backslashes. The backslashes are being removed when I don't expect them to be.
> 
> In src/port1.0/livecheck.tcl MacPorts calls [join ${livecheck.regex}]. Why does it do that? I guess it does that so that we can define a regular expression that contains spaces, without having to enclose it in quotes. MacPorts options are lists, so when we write:
> 
> livecheck.regex this is a valid regular expression
> 
> what we're really setting livecheck.regex to is a list with 6 items. MacPorts helpfully transforms that into a single string for us.

When checking why this uses the join, I noticed I have been on this a
long while ago and tried to "fix" this behavior:

https://github.com/macports/macports-base/commit/961c0ac0bee13e5322c7fd0c997285ed7e44c94e

https://github.com/macports/macports-base/commit/6deef386fea9319b5eb74c98ca4164f6586898c6

I do not really remember the details anymore, but apparently I reverted
back to lists because Portfiles already used an additional level of
escapes... Maybe we should have fixed the Portfiles instead.

> Except that in so doing, it removes one layer of backslashes. Using the tclsh repl we can see that making a string with a backslash works fine:
> 
> $ tclsh
> % set a {hello\.world}
> hello\.world
> % puts $a
> hello\.world
> 
> But if we access it as a list using lindex or join it loses the backslash:
> 
> % lindex $a 0
> hello.world
> % join $a
> hello.world
> % 

lindex or join have to convert the string to a list first. This implies
substitution of backslash escaped characters because they have special
meaning in the string representation of lists.

For example, "x hello\ world y" is the same list as "x {hello world} y",
but in two different string representations. In the first form, the
backslash is used to escape the space character.

Things would be easier if we were always working with list operations,
because these operations will preserve the special characters with
escape sequences when needed. When putting to together a string and only
then converting it to a list, you need to take care of the escaping
yourself.

% set a {}
% lappend a hello
hello
% lappend a {world}
hello world
% lappend a {hello\.world}
hello world {hello\.world}
% puts $a
hello world {hello\.world}
% lindex $a 2
hello\.world
% join $a
hello world hello\.world

Rainer