registry backup (and upgrade --force)

[René J.V. Bertin]

On Sunday April 03 2016 14:41:06 Clemens Lang wrote:

Hi

>> Subsequent port commands hung without even a line of debug output,
>> which I finally presumed due to registry corruption.
>
>How did you presume that? Are you sure your assumption is correct?

The only sensible feedback I got was from the CrashReporter after I sent a SIGTERM (^]) which showed a backtrace into sqlite fsck functions. I also think I had something similar before, and like the previous time, restoring the registry.db file from backup unblocked the situation.
Now, maybe the file wasn't "corrupt" but in some other state, but as far as I'm concerned the net result was the same ;)

>That would still leave a time window where your registry doesn't match
>what's on disk. You wouldn't loose all data, but your data would now be
>faulty. I'd argue that's not much of an improvement.

Does it? If the operation that would introduce new data fails and leaves the file in some unknown state, can you really say new data has been recorded? It's not unlike what journaled and COW filesystems do (as far as I understand those). Commit the new state to some transitional registry, and make that current when you know the commit has succeeded. If that operation fails, you only lose whatever you were trying to change, just like in the scenario I had in mind - and in our case that should typically be an operation that'll be easy enough to repeat.

>We do, however, have signal handling code on trunk that should
>gracefully handle interruptions in the critical phases that touch the
>registry and avoid these problems in the future. Until then:
>Interrupting MacPorts during activation/deactivation isn't safe. Don't
>do it.

Yeah, I was thinking of that too. The problem is of course that's something you think about *after* hitting ^C when you realise you just launched something that also carries the risk of borking your whole install. Not easy to time that ^C (maybe I should get used to using ^Z instead).

>Only if the contents are the same, which isn't the case in general when
>the backup is a day old. Doing this kind of recovery automatically would
>silently introduce corrupted data -- a very bad idea.

I was very specifically thinking of a mechanism that maintains a backup of the last known valid state.
I know I mentioned an offline procedure, but that was with launchd's file-watching feature in mind which would trigger the copy after each change to the file. But there are other solutions of course, like forking a script that does the copy in the background.

>I agree, upgrade --force should imply only rebuilding the given ports. A
>patch would be nice.

Is that a hint? :)
Any idea if that's going to require a lot of changes or rather something that's as trivial as it sounds like it should be?

R.