Port update outdated problem.

[Ryan Schmidt]

On Nov 20, 2014, at 4:30 AM, René J.V. Bertin wrote:

> On Thursday November 20 2014 11:11:05 Clemens Lang wrote:
> 
>>> What is the point of all those scans, if I may ask?
>> 
>> We're not the ones doing them, SQLite is. We don't really care how SQLite does
>> things, as long as it's fast.
> 
> OK. BTW, isn't SQLite a protocol that doesn't really scale well? I'm far from an expert on this kind of subject, but I recently followed advice on the kdepim-user ML to migrate my akonadi db to mysql (mariadb, as is the default for port:akonadi), and performance has become way better.
> The usage scenario is probably quite different though.

SQLite is nice in that it does not require a separately running server process. MySQL (MariaDB) and PostgreSQL are full database servers that require a database administrator to manage them. We don't want every MacPorts user to have to become a database administrator.


>>> I wonder if this also explains the memory exhaustion problems I've seen occur
>>> during lengthy port upgrades (i.e. many ports at once).
>> 
>> That's very unlikely. I also haven't seen any problems like these.
> 
> I understand that 10.6 isn't concerned by those db issues.

The slow SQLite behavior is new for 10.10 Yosemite.


> Just to clarify though: I typically tend to run selfupdate/upgrades when I start getting the warning about the repo being outdated, and then set the process to a very low priority (but keep the number of jobs equal to the number of cores I have). And do other stuff that usually doesn't require a lot of resources.
> When I still had "only" 8Gb of memory I often saw messages about emergency pages being used during the upgrade process (in kernel.log). The other day I had a parallels VM with 4Gb RAM open, and at some point got low disk warnings because 19Gb or so of swap files had consumed most of my remaining free space.
> That's on 10.6, where I have indeed seen clang-3.4 consume huge amounts of memory on a single file, AFAIK that compiler wasn't being used. However, if port upgrade outdated uses a single tclsh process for the whole procedure, and it's this process that handles (inefficient) db stuff, that could cause its memory footprint to explode.

It's certainly possible there is a memory leak in MacPorts base somewhere.