Filesystem Misinformation [was Re: Homebrew]

[Arno Hautala]

On Tue, May 18, 2010 at 21:32, Jordan K. Hubbard <jkh at apple.com> wrote:
>
> On May 18, 2010, at 6:09 PM, Arno Hautala wrote:
>>
>>> 1. Hard links don't play nicely with Time Machine
>>
>> The don't.  As an example, below is the output from some hard linked
>> files I created.  First on my system and then on my TimeMachine
>> Backup.
>
> I think you may be getting confused by the fact that TM does not create individual file links to your "level zero backup" (assuming files that do not change) if it can also create a directory hard link one level up.  The directory hard link "feature" is unique to time machine and can significantly screw with your existing assumptions if you're used to the way Unix has traditionally done things.  Try modifying an existing file in one of these directories-containing-hardlinks and do another backup to see how this is dealt with in turn.

No, look at my examples.
I know that if a directory hasn't changed between backups TM doesn't
re-copy or link every file (as does rsnapshot) to save space.  It
links the directory.  And a modified file will copy that file, link
the unchanged files, and carry on.
I'm talking about hardlinks on the source volume that TM should be
backing up as consistent hardlinks.

MacPorts, in order to save space, installs into
/opt/local/var/macports/software/  It then creates hardlinks to the
intended destinations, such as /opt/local/bin as in:

$ ls -lahi /opt/local/bin/mplayer
/opt/local/var/macports/software/mplayer-devel/31034_0+darwin+dts+esd+live+macosx/opt/local/bin/mplayer
28742887 -rwxr-xr-x 2 root admin 9.2M May  2 01:14 /opt/local/bin/mplayer
28742887 -rwxr-xr-x 2 root admin 9.2M May  2 01:14
/opt/local/var/macports/software/mplayer-devel/31034_0+darwin+dts+esd+live+macosx/opt/local/bin/mplayer

When TM runs over these two files, they are backed up as separate
regular files instead of hardlinks to the same data.  Subsequent
backups will link the containing folders, but that first backup
created 2 unique inodes where 1 would do the job.

I'm excluding /opt/local/var/macports from my TM backups, so accept
the following familiar example instead:

$ ls -lahi /Volumes/Backup\ of\
Computer/Backups.backupdb/Computer/Latest/MacHD/Users/arno/test/
total 21M
108469819 drwxr-xr-x+  2 arno staff  442 May 18 20:12 ./
108426565 drwxr-xr-x+ 57 arno staff 3.4K May 18 20:17 ../
108469820 -rw-r--r--+  1 arno staff  82K May 18 20:13 link-test.png
108469821 -rw-r--r--+  1 arno staff 2.0M May 18 20:07 link0
108469822 -rw-r--r--+  1 arno staff 2.0M May 18 20:07 link1
108469823 -rw-r--r--+  1 arno staff 2.0M May 18 20:07 link2
108469824 -rw-r--r--+  1 arno staff 2.0M May 18 20:07 link3
...
108469830 -rw-r--r--+  1 arno staff 2.0M May 18 20:07 link9

The TM backup is not correctly backing up the hardlinks; note the
differing inodes.

If it was, the above would look like this:

$ ls -lahi /Volumes/Backup\ of\
Computer/Backups.backupdb/Computer/Latest/MacHD/Users/arno/test/
total 21M
108469819 drwxr-xr-x+  2 arno staff  442 May 18 20:12 ./
108426565 drwxr-xr-x+ 57 arno staff 3.4K May 18 20:17 ../
108469820 -rw-r--r--+  1 arno staff  82K May 18 20:13 link-test.png
108469821 -rw-r--r--+  10 arno staff 2.0M May 18 20:07 link0
108469821 -rw-r--r--+  10 arno staff 2.0M May 18 20:07 link1
108469821 -rw-r--r--+  10 arno staff 2.0M May 18 20:07 link2
108469821 -rw-r--r--+  10 arno staff 2.0M May 18 20:07 link3
...
108469821 -rw-r--r--+  10 arno staff 2.0M May 18 20:07 link9


> I think you're also getting fooled into thinking that TM's job is to recreate your filesystem hierarchy exactly in another location - it's not.  TM's job is to provide a *backup* of your files in such a fashion that they can be restored back exactly the way they were, hardlinks and symlinks included, and it does.

I understand your point about providing backups, but it's just not
true that TM "can be restored back exactly the way they were,
hardlinks and symlinks included".  I am confused as to what you mean
when you say that TM's job isn't to recreate the filesystem exactly in
another location, but it is supposed to restore in exactly the way it
was.
TimeMachine does not correctly backup hardlinks.  If you create a 100
MB file and hardlink it 10 times, the resulting TimeMachine backup
will take up 1000 MB of space.  Restoring from this backup will take
1000 MB.  No data will be "lost", the hardlinks are broken, but you
also definitely don't have the same files restored "exactly the way
they were".

This is the issue that was mentioned regarding MacPorts and TM.  It's
not a big problem, but it's there.


> No, they can't span volumes since each volume obviously has its own inode table, but a number of archivers / file copy utilities on MacOSX will happily preserve hard links

Agreed, CCC does, others might, TM doesn't.


-- 
arno  s  hautala    /-|   arno at alum.wpi.edu

pgp eabb6fe6 d47c500f b2458f5d a7cc7abb f81c4e00