circular dependencies

[Clemens Lang]

----- On 23 Feb, 2015, at 23:34, René J.V. Bertin rjvbertin at gmail.com wrote:

> It intercepts all (file open and similar) system calls and then checks against
> the installed files database, possibly for each and every header file included
> (which often means the same header file multiple times), and you see no
> significant impact? I find surprising, esp. if the database operation is run
> through tcl (not saying *in* tcl :)).

You are describing why it isn't enabled by default yet, and this is something
that will have to be solved before it does get enabled by default. I do have a
model in mind about a non-blocking shared-memory cache data structure that will
avoid multiple lookups, but I did not have time to implement it yet. Also note
that the disk access to read a file will be orders of magnitude slower than a
lookup in this cache.

Note that this is about the non-functional aspects of the system. The
functional aspects (which I assume were what Jeremy meant) have close to no
impact whatsoever on the execution of the builds, but do provide a huge
improvement (namely reproducible builds in various environments). For example,
trace mode, if implemented correctly, can make the requirement of the tedious
migration process after OS updates go away. As of this moment, I have been
exclusively compiling ports with trace mode enabled for probably well over a
year and so far I am only aware of two ports that do not build in trace mode
due to functional changes or problems that are caused by trace mode, and those
are:
 - go, because the OS X loader uses malloc() when DYLD_INSERT_LIBRARIES is set
   and that breaks a very low-level step in the build process of go.
 - openjade, which crashes during the build with what I can only conclude to
   be a buffer overflow in the build system of openjade (which is one of the
   downsides you get for re-using the address space of a program).
Clearly, we'll need a system in place to selectively disable trace mode for
these critical steps, but other than that the *vast* majority of our ports
build fine in trace mode or fail because of bugs in the port.

P.S.: The database lookup does not use Tcl at all, for precisely the reasons
you mentioned. It does use IPC, which comes with context switch overhead,
which is why a cache should speed it up considerable. Still, the overhead is
currently only about 80 %, which I think is pretty good considering no work
has been put into optimization.

-- 
Clemens Lang