[101504] trunk/base/src/port1.0/portextract.tcl

Tue Jan 15 21:10:24 PST 2013

On 01/14/2013 08:32 PM, Blair Zajac wrote:
> On 01/14/2013 05:33 PM, Sean Farley wrote:
>> On Mon, Jan 14, 2013 at 11:09 AM, Blair Zajac <blair at orcaware.com> wrote:
>>> On 01/13/2013 06:46 PM, Ryan Schmidt wrote:
>>>>
>>>>
>>>> On Jan 13, 2013, at 18:53, Blair Zajac wrote:
>>>>
>>>>> On 01/13/2013 03:36 PM, Ryan Schmidt wrote:
>>>>>
>>>>>> https://trac.macports.org/ticket/21117
>>>>>>
>>>>>> I think the problem still remains when you have a non-root MacPorts
>>>>>> installation; that's the configuration I was using when I
>>>>>> initially filed
>>>>>> the ticket, before the privilege-dropping code was added to
>>>>>> MacPorts and
>>>>>> made it an issue for everyone. The way I was thinking of fixing it
>>>>>> was to
>>>>>> exclude those files from copying that we don't need anyway and I
>>>>>> think that
>>>>>> still might be a better way, but I didn't really know how to do
>>>>>> that which
>>>>>> is why I never did anything about the ticket.
>>>>>
>>>>>
>>>>> If you're doing this as non-root, then presumably you have permissions
>>>>> yourself to mount a DMG.  Are you saying that even if you can mount
>>>>> it,
>>>>> there's files in there you cannot see?
>>>>
>>>>
>>>>
>>>> I haven't tried using a non-root MacPorts installation in awhile; it
>>>> just
>>>> seemed to become totally impossible once the privilege-dropping
>>>> branch got
>>>> merged in. Maybe those bugs have been fixed by now.
>>>>
>>>> But here's an example from outside of MacPorts, using the distfile
>>>> for the
>>>> cliclick port:
>>>>
>>>>
>>>> $ open cliclick/2.1/cliclick.dmg
>>>> $ cd /Volumes/cliclick
>>>> $ find . -print0 | xargs -0 ls -ld
>>>> find: ./.Trashes: Permission denied
>>>> drwxr-xr-x  12 rschmidt  rschmidt    476 Sep 26 02:12 .
>>>> -rw-r--r--@  1 rschmidt  rschmidt  12292 Sep 26 02:14 ./.DS_Store
>>>> d-wx-wx-wt   2 rschmidt  rschmidt     68 Sep 26 02:15 ./.Trashes
>>>> drwxr-xr-x   8 rschmidt  rschmidt    272 Sep 26 02:13 ./.assets
>>>> -rw-r--r--   1 rschmidt  rschmidt   5639 Sep 26 02:11
>>>> ./.assets/AppIcon.png
>>>> -rw-r--r--   1 rschmidt  rschmidt     95 Sep 26 02:11 ./.assets/bar.gif
>>>> -rw-r--r--   1 rschmidt  rschmidt   4535 Sep 14 05:24
>>>> ./.assets/diskimage-background.png
>>>> -rw-r--r--   1 rschmidt  rschmidt     67 Sep 26 02:11 ./.assets/li.gif
>>>> -rw-r--r--   1 rschmidt  rschmidt   1193 Sep 26 02:11
>>>> ./.assets/logo.gif
>>>> -rw-r--r--   1 rschmidt  rschmidt   1301 Sep 26 02:11
>>>> ./.assets/style.css
>>>> drwx------   4 rschmidt  rschmidt    136 Sep 26 02:15 ./.fseventsd
>>>> -rw-------   1 rschmidt  rschmidt    209 Sep 26 02:15
>>>> ./.fseventsd/6365733132517f42
>>>> -rw-------   1 rschmidt  rschmidt     36 Sep 26 02:15
>>>> ./.fseventsd/fseventsd-uuid
>>>> -rw-r--r--   1 rschmidt  rschmidt    277 Sep 26 02:11 ./Contact.webloc
>>>> -rw-r--r--@  1 rschmidt  rschmidt   4676 Sep 26 02:11 ./Read me.html
>>>> -rw-r--r--   1 rschmidt  rschmidt    265 Sep 26 02:11 ./Website.webloc
>>>> -rwxr-xr-x   1 rschmidt  rschmidt  89216 Sep 26 02:11 ./cliclick
>>>>
>>>>
>>>> So .Trashes cannot be seen. One solution I thought of was to just
>>>> not copy
>>>> anything beginning with a period. In other words, "cp *
>>>> ${worksrcpath}".
>>>> ("*" expansion does not include items starting with ".")
>>>>
>>>> A more elaborate and more correct solution would be to actually look at
>>>> the permissions of each item and just not copy those we're not
>>>> allowed to,
>>>> but doing that in a single-line command execution, such as is
>>>> required by
>>>> the extract system, was daunting.
>>>
>>>
>>> Instead of tar, you can use find's -perms flag.  I haven't tried
>>> this, but
>>> this may work:
>>>
>>> $ find PATH -depth -perms +r -print0 | cpio -0 -p -d -m -u -v
>>> "$worksrcpath
>>
>> I would agree with Jeremy's suggestion to use rsync. I think that will
>> be the most elegant and not involve any piping or redirection, etc.
>> Though, I haven't tried to formulate the command yet; I'm just
>> thinking out loud here.
>
> That'll work also, but you'll get more errors from rsync on the files
> and directories it cannot read.  Granted, find will also warn on
> directories it cannot descend into.

He who codes gets to choose the technique ;)

I used the find | cpio approach in

https://trac.macports.org/changeset/101649 and
https://trac.macports.org/changeset/101650

I spent a bunch of time trying different approaches and the committed 
approach looks to be the best.  It doesn't ignore unexpected errors, but 
ignores unreadable files, doesn't use more than one fork/exec (unlike 
find's exec()).

Notes from the commit log:

$ cd $dmg_dir; find . -depth -perm -+r -print0 | cpio ...flags... -0 
$destdir

- Use find | cpio to avoid fork()/exec() with very many files to copy;
   its faster.

- Use -depth to go depth first so that cpio will fix permissions on
   parent directories once its finished with all the children.

- Use -print0 and -0 to avoid problems with spaces in the filenames.

- Use -perm -+r to skip unreadable files and directories.  By avoiding
   unreadable files and directories, the error status of cpio does not
   need to be ignored, which it would if cp or rsync were used.

- Use cd $dmg_dir because cpio requires filenames to copy from the
   current working directory.

Blair