The static library discussion

[Marin Saric]

Hi everyone,

An issue has been raised in a ticket recently of whether the static
libraries should be (if possible) dealt away with entirely in
MacPorts.

Here's some examples of cases against doing away with them, in no
particular order of importance:

Example 1: ABI incompatibilities

x264 is a great example of where having it in a static form removes
just the rebuilding problem that occurred now. x264 does not have a
very stable ABI. It's source level API is consistent, but depending on
what the version of x264 you download (new "snapshots" on something
like a daily basis), your program is going to link against different
symbols. In the newest version of x264 the ABI broke again.

Assume x264 is dynamic. Now assume a set of dynamic libraries H export
symbols against an older ABI version of x264. Binaries depending on H
and binaries dynamically linking against libraries depending on H all
break once x264 is replaced with a new version. The old symbols are
not found or, even worse, they are found but their semantics or
interfaces have changed in a subtle way causing malfuction. Hence the
"bump_dev_revs.sh" script, forcing a recompile of everything that
depends on x264. If MacPorts were a binary-only distribution, we'd be
in trouble.

This can be circumvented with "properly" versioning x264, but this is
something outside of MacPorts control (IMHO too much involvement
upstream).

One "bad apple" can wreak havoc on the whole system. Note that this is
different than the symbol-collision problem fixed by two-level
namespaces.

Example 2: "Feature freeze"
If you want your binary to just depend on a specific version of a
library that you have validated against instead of the newest latest
and greatest, you will link against a static version of it. On MacOS X
you can put stuff in your own framework or have it live in an
application bundle. Outside the world of MacOS X, linking against a
static library is a way to do it.
In a perfect world, you'd trust the library developer not to break the
desired behavior by adhering to a consistent versioning. You might
also expect the newer versions of the library never to introduce a
regression. Or you might just say, OK, I am happy with the
functionality provided by the current version of the library and I am
OK with statically linking against it.

Example 3: Building plugins - avoiding versioning conflict.
This is a test case I am dealing with right now.  I have binaries that
open my plugin through the dlopen API. These binaries are outside of
MacPorts and they depend on their own private versions of freetype and
some other libraries. Unfortunately they work in a flat namespace
model and thus things break. Static library is a simple fix. I am not
dependent on the application developer to "fix" their program or to
resolve the conflict in any way. My plugin is a dylib (or on Linux a
.so), but it links statically against all the dependencies, so loads
and runs without a problem in the application that uses different
versions of dependent libraries. Other than standard system depencies
on stdc++ library and the usual MacOS frameworks), no symbols are
exported, so there's no collisions or user-unfriendly load-time plugin
failures.

Example 4: Buidling libraries and plugins - deployment
You can use the MacPorts provided libraries, yet provide a dynamic
library or a plugin that does not have extra dependencies. You will
find examples already of software inside MacPorts building "private"
versions of library for the very same reasons.

Example 4.2: "Convenience" libraries
This is a well-known subcase of Example 4. Static "tool" libraries
that are linked into dynamic libraries. This is pretty common across
open-source projects.
http://www.freesoftwaremagazine.com/articles/building_shared_libraries_once_using_autotools

Example 5: Binary deployment.
Statically built binaries with MacPorts become trivially deployable
outside of MacPorts. No extra libraries to bundle with or ask the user
to compile, etc., no need for an installer, etc.

Example 6: Load time
Statically built binaries load very quickly. This comes at a price of
an increased cost in RAM. This used to be an issue when machines came
with an order of magnitude less RAM then they do today. The incurred
cost is small in absolute terms nowadays (end of 2011)
--

MacOS X offers some more advanced solutions to remedy some problems
pointed out by the examples above, but since MacPorts focuses a lot on
bringing projects from the UNIX world into MacOS X, supporting static
libraries means supporting cross-platform solutions to the above
problems. A developer of an open-source package does not need to
special case the MacOS X deployment, but can rely on the standrad
static linking solution.

   Marin