Traversing filesystems [was Re: ASSP out of date]

[Jordan K. Hubbard]

[Adjusting topic to be a bit more descriptive of the current discussion]

On Nov 8, 2008, at 2:53 PM, Ryan Schmidt wrote:

> There is a function fs-traverse in MacPorts designed for this. Grep  
> through the existing portfiles to see how this can be used.

As one of the authors of that function, I'd also be happy to take  
"enhancement requests" for it since it was written to be pretty  
generic and with only a couple of usage cases in mind at the time.   
The code is very simple and not hard to extend, assuming the  
enhancements make any kind of sense.

Regarding your earlier function, however, since I think we should  
always help people learn Tcl where possible.. :-)

proc recursiveDirList { dir } {
   set tmp_list {}
   # Traverse top directory
   set contents [glob -nocomplain -directory $dir *]
   foreach item $contents {
       lappend tmp_list $item
       # Recurse - go into the sub directory
       if { [file isdirectory $item] } {
           lappend tmp_list [recursiveDirList $item]
       }
   }
   # Final result, a list of all files and directories in $dir
   return $tmp_list
}

You say:  "First bad thing is that I do not want the directory in  
there, there is no action I will ever take on a directory.  Second,  
when I print out the list, it is not a clean list, but a nested one,  
and some items have "{" and "}" wrapping the path items, which makes  
no sense to me. Stumped"

First, the directory items are an easy fix.  Just don't lappend the  
item unless it's a file, otherwise recurse.  E.g.:

proc recursiveDirList { dir } {
   set tmp_list {}
   # Traverse top directory
   set contents [glob -nocomplain -directory $dir *]
   foreach item $contents {
       # Recurse - go into the sub directory
       if { [file isdirectory $item] } {
           lappend tmp_list [recursiveDirList $item]
       } else {
	lappend tmp_list $item
       }
   }
   # Final result, a list of all files and directories in $dir
   return $tmp_list
}

That eliminates the extra directory entries.  Now you have directories  
expressed in their own lists, which is actually what you would  
typically want and expect since this lets you pass the resulting  
"uberlist" to some other function which walks through it and could  
benefit from having the nesting level preserved, but OK, you want to  
flatten everything and use absolute paths throughout, no problem.   
Just change:
           lappend tmp_list [recursiveDirList $item]
to
           set tmp_list [concat $tmp_list [recursiveDirList $item]]

Now the lists are flattened.   This brings us to your final point,  
which is that some items have {}'s wrapping them.  This is actually  
not a bug, this is a feature, since Mac OS X filenames can contain  
spaces and braces and other special characters that would confuse the  
heck out of your subsequent pass through the list if they were not  
"escaped" by the braces, e.g. "I am a file with a lot of spaces and  
{crazy chars} in my name!" is going to cause you to look up each  
element in the name otherwise.   That bit of behavior you want to keep.

Finally, you say "In reality, it would be much more ideal to simply  
supply a file extension list to the first function, and return a nice  
clean list of the files"

I suspect you actually already know how to do this, you're just being  
blinded by the obvious.. :)  Final version:

proc recursiveDirList { dir pattern } {
   set tmp_list {}
   # Traverse top directory
   set contents [glob -nocomplain -directory $dir *]
   foreach item $contents {
       # Recurse - go into the sub directory
       if { [file isdirectory $item] } {
           set tmp_list [concat $tmp_list [recursiveDirList $item  
$pattern]]
       } else {
	if {[regexp $pattern $item]} { lappend tmp_list $item }
       }
   }
   # Final result, a list of all files and directories in $dir
   return $tmp_list
}

Voila.  Now you can have, say, a list of all .c or .h files in a  
directory by saying:  recursiveDirList . {.*\.[ch]}

See, no one can say I'm not a believer in that ancient adage:  "Give a  
man a Tcl function and you have educated him for a day.  Teach him  
Tcl, on the other hand, and you have made an enemy for life!"

:-)

- Jordan