modern Tcl and correct quoting

[Poor Yorick]

On Thu, Jun 13, 2013 at 02:53:41AM +0000, Poor Yorick wrote:
> On Wed, Jun 12, 2013 at 10:17:10PM -0400, Lawrence Velázquez wrote:
> > On Jun 12, 2013, at 6:18 PM, Poor Yorick <org.macosforge.lists.macports-dev at pooryorick.com> wrote:
> > 
> > > [eval] concatenates its arguments and passes them through the interpreter again
> > > to be parsed and executed as a script.  This results in double substitution.
> > > Here's an example from the patch:
> > > 
> > >    $workername eval "package ifneeded $pkgName $pkgVers {$pkgLoadScript}"
> > > 
> > > The first issue with this is that if, e.g., $pkgName contained a space, the
> > > constructed script would no longer be syntactically correct, as it would pass
> > > too many arguments to [package ifneeded].  The second issue is that simply
> > > putting braces around a substituted variable like $pkgLoadScript is not
> > > guaranteed to result in a well-formed single value.  If $pkgLoadScript contains
> > > a left or right curly bracket, or if it ends in a backslash, the script will be
> > > corrupted.  .  The robust way to write the line above is like this:
> > > 
> > > 	$workername eval [list package ifneeded $pkgName $pkgVers $pkgLoadScript]
> > > 
> > > http://wiki.tcl.tk/1535 has more details.
> > 
> > 
> > Is there a material difference between using list here and doing something like this?
> > 
> >     $workername eval {package ifneeded $pkgName $pkgVers $pkgLoadScript}
> > 
> 
> Yes, in the first case, the variables are resolved in the current scope before
> [$workername eval ...] is invoked, and in the second, they are resolved in the
> scope ofthe $workername interpreter during the execution or [$workername eval
> ...], and those are two entirely different namespaces.
> 

Here's another example from the patch.  The original version looks like this:

	if {[catch {eval curl fetch $verboseflag {$source} {$tarpath}} error]} {
		...
	}

Taking a look at just the [eval] part:

	eval curl fetch $verboseflag {$source} {$tarpath}

The curly brackets around {$source} and {$tarpath} are there to escape
substitution as Tcl prepares to invoke [eval], and this is not necessarily
incorrect, but there are some caveats. $verboseflag is resolved prior to the
invocation of [eval], but $source and $tarpath are passed as literal strings.
If $verboseflag is "yes", [eval] will get exactly five values as arguments:

	curl

	fetch

	yes

	$source

	$tarpath

It will then concatenate these arguments into a script which it will then pass to the interpreter:

	curl fetch yes $source $tarpath

$source and $tarpath have not been resolved yet.  As it prepares to execute
[curl], the the interpreter will resolve them, and [curl] will receive exactly
4 arguments.

If [list] is used instead of the curly brackets, the same thing is
accomplished, but in a different way:

	eval [list curl fetch $verboseflag $source $tarpath]

This time, $verboseflag, $source, and $tarpath are all resolved at the same
time, prior to the invocation of [list], and eval doesn't have to concatenate
its arguments, since it receives exactly one:  a list having exactly five
elements.  Each item in the list has been nicely formatted by [list] such that
it complies with the Tcl syntax rules
(http://www.tcl.tk/man/tcl/TclCmd/Tcl.htm):  All whitespace and other special
characters have been properly escaped.  For example, the list value might look
something like this (For readability, newlines separate list items, but they
would normally be space characters, but otherwise, this is the literal value):

	curl

	fetch

	yes

	{/a/path with/spaces and $ signs in/it}

	/another/such\ path/with\ spaces\ and\ \$\ signs\ in/it

I've intentionally used different quoting mechanisms in the path values to
illustrate that we don't need to know or care exactly how Tcl decides to escape
things to create a properly-formatted list.  What matters is that each argument
is properly escaped and interpreted as a single word when Tcl is preparing to
execute the list as a command.

Using the [list] version yields better performance because it saves at least
two steps.  In the sub-optimal curly-bracketed version of this example, [eval]
has to take the extra measure of concatenating its multiple arguments, and then
prior to interpreting the command, the interpreter has to parse the "curl
fetch..." string into a list.

Under the hood, there is even more optimization that can happen when eval is
passed a single argument which is a "pure list". 

-- 
Poor Yorick