MacPorts on COW filesystem

[René J.V. Bertin]

On Saturday July 11 2015 11:00:37 Clemens Lang wrote:

>OK, that sounds like something you should also report to the developers of SQLite.
>They may not be aware of the performance of SQLite on ZFS. However, I'm not sure

Erm, you do know that ZFS is used as the default fs on server-oriented OS flavours that are unlikely NOT to see sqlite-based applications?

>It should disappear when the database is closed, which happens when MacPorts
>terminates.

OK, thanks.

>> filesystem into account. My experience with the apt/dpkg registry on Linux
>> seems to confirm that, as well as today's observations with the pool mentioned
>> above.
>
>Yes, that's possible. I'd expect the developers of these databases to be interested
>in any help you can offer.

The apt/dpkg registry is built with xapian methinks, but other than that, which "developers of these databases"? The people who develop the application, or those who develop the technology behind the application (sqlite/xapian/... devs)?

I've seen similar intermittent issues when I still used an sqlite3-based database with KDE PIM's akonadi, on my Linux rig that runs off ZFS. From time to time, operations would take much longer than usual, so much so I'd be tempted to restart the application(s) (which wouldn't help a bit). It became better when I moved the database to mysql (akonadi has support for sqlite, mysql and postgresql) but it wasn't until I tweaked some of the server parameters in the dedicated mysql.conf file that those issues disappeared (or mostly). Sadly that was with the help of a DB expert on the kdepim ML, and I can't say I learned much from it other than the general idea that db applications like this are rarely designed with the peculiarities of something like ZFS in mind.

I doesn't NOT make sense though: I guess one can think of a database file as something akin to a disk image, which can be subject to symptoms like (or homologous to) fragmentation that require intermittent maintenance (which would probably be hidden from the user). Thinking aloud that could also mean that there's little to tune in terms of parameters governing standard operations.

>Edit src/cregistry.c, function reg_attach; add a new SQLite query after
> ATTACH DATABASE '%q' AS registry
>that uses the PRAGMA command to choose the journal mode.

OK, I hope I'll remember the next time I run into weirdness that persists long enough to start rebuilding and tweaking MacPorts innards.

R.