rsync strange behaviour

[Ryan Schmidt]

On Oct 26, 2017, at 04:28, pagani laurent wrote:

>> My first guess would be that some items on your internal drive are hardlinked, and that when you rsync, those items are being created on the external drive as separate files with each copy taking up space. And I can confirm by inspection that there are many hardlinked items inside the Xcode 9 app, for example. There's a flag you can supply to rsync ("--hard-links") to tell it to detect and recreate hard links on the destination drive, which should eliminate this reason for the size difference.
> 
> Did not work. I erased Xcode again and rsync’ed it with —hard-links. Size remains the same. 12 Gb instead of 7.8 local.

I can only confirm your observation. My original Xcode.app is "10,581,631,771 bytes (5.76 GB on disk)" according to the Finder's Get Info window, while an rsync'd copy with --hard-links is "10,511,782,504 bytes (11.12 GB on disk)". As a result, I now question whether it's possible to create an accurate copy of a macOS disk using rsync.

The rsync manpage mentions --hard-links not being a default because finding hard links is expensive. Maybe there is an upper limit to the number or size of files rsync is willing to analyze to find hard links; if so, maybe what we're doing exceeds that limit.

Maybe Xcode and other things on your original disk are making use of directory hard links, a feature Apple implemented so that Time Machine could be more efficient, but which rsync doesn't know about. I read about this here:

https://arstechnica.com/civis/viewtopic.php?f=20&t=1108222

It's also mentioned there that you can use "cpio -pdl" somehow to reunite inadvertently duplicated files with one another as hard links, though I'm not sure what it will do if you have any identical files that are supposed to remain unlinked.