[101504] trunk/base/src/port1.0/portextract.tcl

[Sean Farley]

On Tue, Jan 15, 2013 at 11:10 PM, Blair Zajac <blair at orcaware.com> wrote:
> On 01/14/2013 08:32 PM, Blair Zajac wrote:
>>
>> On 01/14/2013 05:33 PM, Sean Farley wrote:
>>>
>>> On Mon, Jan 14, 2013 at 11:09 AM, Blair Zajac <blair at orcaware.com> wrote:
>>>>
>>>> On 01/13/2013 06:46 PM, Ryan Schmidt wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Jan 13, 2013, at 18:53, Blair Zajac wrote:
>>>>>
>>>>>> On 01/13/2013 03:36 PM, Ryan Schmidt wrote:
>>>>>>
>>>>>>> https://trac.macports.org/ticket/21117
>>>>>>>
>>>>>>> I think the problem still remains when you have a non-root MacPorts
>>>>>>> installation; that's the configuration I was using when I
>>>>>>> initially filed
>>>>>>> the ticket, before the privilege-dropping code was added to
>>>>>>> MacPorts and
>>>>>>> made it an issue for everyone. The way I was thinking of fixing it
>>>>>>> was to
>>>>>>> exclude those files from copying that we don't need anyway and I
>>>>>>> think that
>>>>>>> still might be a better way, but I didn't really know how to do
>>>>>>> that which
>>>>>>> is why I never did anything about the ticket.
>>>>>>
>>>>>>
>>>>>>
>>>>>> If you're doing this as non-root, then presumably you have permissions
>>>>>> yourself to mount a DMG.  Are you saying that even if you can mount
>>>>>> it,
>>>>>> there's files in there you cannot see?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> I haven't tried using a non-root MacPorts installation in awhile; it
>>>>> just
>>>>> seemed to become totally impossible once the privilege-dropping
>>>>> branch got
>>>>> merged in. Maybe those bugs have been fixed by now.
>>>>>
>>>>> But here's an example from outside of MacPorts, using the distfile
>>>>> for the
>>>>> cliclick port:
>>>>>
>>>>>
>>>>> $ open cliclick/2.1/cliclick.dmg
>>>>> $ cd /Volumes/cliclick
>>>>> $ find . -print0 | xargs -0 ls -ld
>>>>> find: ./.Trashes: Permission denied
>>>>> drwxr-xr-x  12 rschmidt  rschmidt    476 Sep 26 02:12 .
>>>>> -rw-r--r--@  1 rschmidt  rschmidt  12292 Sep 26 02:14 ./.DS_Store
>>>>> d-wx-wx-wt   2 rschmidt  rschmidt     68 Sep 26 02:15 ./.Trashes
>>>>> drwxr-xr-x   8 rschmidt  rschmidt    272 Sep 26 02:13 ./.assets
>>>>> -rw-r--r--   1 rschmidt  rschmidt   5639 Sep 26 02:11
>>>>> ./.assets/AppIcon.png
>>>>> -rw-r--r--   1 rschmidt  rschmidt     95 Sep 26 02:11 ./.assets/bar.gif
>>>>> -rw-r--r--   1 rschmidt  rschmidt   4535 Sep 14 05:24
>>>>> ./.assets/diskimage-background.png
>>>>> -rw-r--r--   1 rschmidt  rschmidt     67 Sep 26 02:11 ./.assets/li.gif
>>>>> -rw-r--r--   1 rschmidt  rschmidt   1193 Sep 26 02:11
>>>>> ./.assets/logo.gif
>>>>> -rw-r--r--   1 rschmidt  rschmidt   1301 Sep 26 02:11
>>>>> ./.assets/style.css
>>>>> drwx------   4 rschmidt  rschmidt    136 Sep 26 02:15 ./.fseventsd
>>>>> -rw-------   1 rschmidt  rschmidt    209 Sep 26 02:15
>>>>> ./.fseventsd/6365733132517f42
>>>>> -rw-------   1 rschmidt  rschmidt     36 Sep 26 02:15
>>>>> ./.fseventsd/fseventsd-uuid
>>>>> -rw-r--r--   1 rschmidt  rschmidt    277 Sep 26 02:11 ./Contact.webloc
>>>>> -rw-r--r--@  1 rschmidt  rschmidt   4676 Sep 26 02:11 ./Read me.html
>>>>> -rw-r--r--   1 rschmidt  rschmidt    265 Sep 26 02:11 ./Website.webloc
>>>>> -rwxr-xr-x   1 rschmidt  rschmidt  89216 Sep 26 02:11 ./cliclick
>>>>>
>>>>>
>>>>> So .Trashes cannot be seen. One solution I thought of was to just
>>>>> not copy
>>>>> anything beginning with a period. In other words, "cp *
>>>>> ${worksrcpath}".
>>>>> ("*" expansion does not include items starting with ".")
>>>>>
>>>>> A more elaborate and more correct solution would be to actually look at
>>>>> the permissions of each item and just not copy those we're not
>>>>> allowed to,
>>>>> but doing that in a single-line command execution, such as is
>>>>> required by
>>>>> the extract system, was daunting.
>>>>
>>>>
>>>>
>>>> Instead of tar, you can use find's -perms flag.  I haven't tried
>>>> this, but
>>>> this may work:
>>>>
>>>> $ find PATH -depth -perms +r -print0 | cpio -0 -p -d -m -u -v
>>>> "$worksrcpath
>>>
>>>
>>> I would agree with Jeremy's suggestion to use rsync. I think that will
>>> be the most elegant and not involve any piping or redirection, etc.
>>> Though, I haven't tried to formulate the command yet; I'm just
>>> thinking out loud here.
>>
>>
>> That'll work also, but you'll get more errors from rsync on the files
>> and directories it cannot read.  Granted, find will also warn on
>> directories it cannot descend into.
>
>
> He who codes gets to choose the technique ;)

Heh, fair enough :-)

> I used the find | cpio approach in
>
> https://trac.macports.org/changeset/101649 and
> https://trac.macports.org/changeset/101650
>
> I spent a bunch of time trying different approaches and the committed
> approach looks to be the best.  It doesn't ignore unexpected errors, but
> ignores unreadable files, doesn't use more than one fork/exec (unlike find's
> exec()).
>
> Notes from the commit log:
>
> $ cd $dmg_dir; find . -depth -perm -+r -print0 | cpio ...flags... -0
> $destdir
>
> - Use find | cpio to avoid fork()/exec() with very many files to copy;
>   its faster.
>
> - Use -depth to go depth first so that cpio will fix permissions on
>   parent directories once its finished with all the children.
>
> - Use -print0 and -0 to avoid problems with spaces in the filenames.
>
> - Use -perm -+r to skip unreadable files and directories.  By avoiding
>   unreadable files and directories, the error status of cpio does not
>   need to be ignored, which it would if cp or rsync were used.
>
> - Use cd $dmg_dir because cpio requires filenames to copy from the
>   current working directory.

Ah, very nice! I didn't have any experience with cpio, so thanks for
pounding this all out and logging your notes :-)