sqlite3 database in macports

[Mihir Luthra]

Hi,

>
> The database lives on disk in ${portdbpath}/registry/registry.db. It has
> to, or we wouldn't have the D in ACID [1].
>
>
Probably then it should make a huge difference in hdd or fusion drives. All
tests I made were on ssd.
Also I tried connecting an external hdd, but when I tried installing
MacPorts base on it, while configuring
it gave me errors saying something like specify —build, —target and —host.


> An SQL database is likely not optimal for the access pattern of trace
> mode. Using another way of looking up file ownership would be a good
> direction to explore. Perhaps also look at doing more caching if
> possible, of negative results as well as positive.
>

In general what I found is, like in port gettext, out of 400,000 calls to
the sever, 50,000 calls need to get queried through registry.
Remaining 350,000 are always allowed or always denied prefixes such as /bin.
Till now I was simply caching any path returned from
__darwintrace_sandbox_check(). Before I never noticed 350,000 times server
is asked only for prefixes. This led to many cache miss in case of prefix
searches.
When I do caching of all this data, making shared memory capable of
handling prefixes also. only “300” calls actually go to server. This
actually means most of the time socket doesn’t even need to get setup.
Doing this, gettext took (real:11m.9.701s, user:12m47.990s,sys:6m51.499s)
which is pretty much an improvement compared to last time.

Although due to __darwintrace_setup(), being called every time before path
lookup, setup gets done every time for no reason. Also path normalisation
code is mixed up with path lookup . I guess I should try and modularise
them somehow before next testing.


>
> And most importantly, do some profiling so you know what is actually
> taking the time. Even if you just print out a bunch of timestamps at
> different places you at least know something.
>

That really helped.


>
> - Josh
>
> [1] https://en.wikipedia.org/wiki/ACID_(computer_science)


Thanks for the help,
Mihir
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macports.org/pipermail/macports-dev/attachments/20190613/9671db3a/attachment-0001.html>