Filesystem Misinformation [was Re: Homebrew]

[Arno Hautala]

On Tue, May 18, 2010 at 19:45, Jordan K. Hubbard <jkh at apple.com> wrote:
>
> So, so far we have a number of "interesting assertions" here:
> ...
> Sadly, each and every one of those assertions is also false and would have
> been provably false had anyone actually decided to crunch some numbers (or
> "try it") for each assertion.

I decided to check these out:


> 1. Hard links don't play nicely with Time Machine

The don't.  As an example, below is the output from some hard linked
files I created.  First on my system and then on my TimeMachine
Backup.


Local:

$ du -hc
2.1M	.
2.1M	total

$ ls -lah ~arno/test
total 21M
drwxr-xr-x   13 arno staff  442 May 18 20:12 ./
drwxr-xr-x+ 108 arno staff 3.6K May 18 20:17 ../
-rw-r--r--    1 arno staff  82K May 18 20:13 link-test.png
-rw-r--r--   10 arno staff 2.0M May 18 20:07 link0
-rw-r--r--   10 arno staff 2.0M May 18 20:07 link1
-rw-r--r--   10 arno staff 2.0M May 18 20:07 link2
...
-rw-r--r--   10 arno staff 2.0M May 18 20:07 link9


TM Backup:

$ du -hc /Volumes/Backup\ of\
Computer/Backups.backupdb/Computer/Latest/MacHD/Users/arno/test/
 20M	/Volumes/Backup of
Computer/Backups.backupdb/Computer/Latest/MacHD/Users/arno/test/
 20M	total

$ ls -lah /Volumes/Backup\ of\
Computer/Backups.backupdb/Computer/Latest/MacHD/Users/arno/test/
total 21M
drwxr-xr-x+  2 arno staff  442 May 18 20:12 ./
drwxr-xr-x+ 57 arno staff 3.4K May 18 20:17 ../
-rw-r--r--+  1 arno staff  82K May 18 20:13 link-test.png
-rw-r--r--+  1 arno staff 2.0M May 18 20:07 link0
-rw-r--r--+  1 arno staff 2.0M May 18 20:07 link1
-rw-r--r--+  1 arno staff 2.0M May 18 20:07 link2
...
-rw-r--r--+  1 arno staff 2.0M May 18 20:07 link9


Specifically note the 2nd column which lists "10" on my system and "1"
on the TM Backup.  This is the link count and is indicating that on my
system each file is a hardlink with nine other locations linking to
the same data.  On the TM Backup, only one file links to each data
record.  This is not due to anything like excluding the other copies
from the backup.

It's also interesting that 'ls' doesn't seem to be hardlink aware
either as it lists a total of 20 MB for the directory, whereas 'du'
lists the correct 2 MB.

Hardlinks can't span volumes and they aren't retained when copied to
other volumes.  It'd be nice if TimeMachine was smart enough to keep
these (CarbonCopyCloner does [1]), but it currently doesn't.
This should be reported to Apple (re. TimeMachine, ls, others?).



> 2. Hard links are not adequately accounted for by du/Finder/etc in determining the number of blocks allocated to MacPorts

As indicated above, du correctly reports the size, though confusingly
only prints each linked file once.  Ideally, in my view, each file
would be listed, with some indication that it was a hardlink to an
already listed file.

$ du -ch ~arno/test/link*
 84K	/Users/arno/test/link-test.png
2.0M	/Users/arno/test/link0
2.1M	total


The Finder is a different story.  When I select all these linked
files, or their containing folder, in the Finder and Get Info or bring
up the Inspector I get a reported size of 20.9 MB.  (Screenshot
included [2])
This should also be reported (file under FTFF).



> 3. Hard links are slower than symlinks under HFS

Can't speak to this one, but I'll buy that it _was_ an issue and isn't
anymore as has already been stated.  I certainly haven't noticed any
drastic slowdown related to hardlinks.  Apple seems to think they're
fast enough to use with TimeMachine. [3]



[1]
$ ls -lah /Volumes/MacHD-CCC-mirror/Users/arno/test/
total 21M
drwxr-xr-x   13 arno staff  442 May 18 20:12 ./
drwxr-xr-x+ 108 arno staff 3.6K May 18 20:17 ../
-rw-r--r--    1 arno staff  82K May 18 20:13 link-test.png
-rw-r--r--   10 arno staff 2.0M May 18 20:07 link0
-rw-r--r--   10 arno staff 2.0M May 18 20:07 link1
-rw-r--r--   10 arno staff 2.0M May 18 20:07 link2
...
-rw-r--r--   10 arno staff 2.0M May 18 20:07 link9

[2] The discrepancy in the location between the screenshot and the
rest of this message is due to testing the Finder behavior first
before moving the directory out of a TM excluded zone.

[3] It strikes me as odd that Apple would account for hardlinks in
order to save space across redundant backups, but completely ignore
them when creating those backups.


-- 
arno  s  hautala    /-|   arno at alum.wpi.edu

pgp eabb6fe6 d47c500f b2458f5d a7cc7abb f81c4e00
-------------- next part --------------
A non-text attachment was scrubbed...
Name: link-test.png
Type: image/png
Size: 83917 bytes
Desc: not available
URL: <http://lists.macosforge.org/pipermail/macports-dev/attachments/20100518/be447a33/attachment-0001.png>