[GSoC][Binaries support] Architecture

[Anders F Björklund]

Felipe Tanus wrote:

> I started to think about an architecture for the binaries support
> project, and I would like to know your opinion at some points. I think
> it's easy to notice that the software has two distinct parts, the
> server and the client. The server is the build and distribution
> solution, and the client is the port modification and maybe the
> binary-only version. I'm splitting my ideas in these two areas.

I think that's a valid distinction, and offering my own opinions:


> About the server:
> 
> * Failures report: It might be obvious, but since it's not on the
> description in idea's list, I think the best way of notification is by
> e-mail. A public mail list can be created to spread this info, maybe
> at macosforge.

Usually it would email the maintainer and keep the build log,
but that's up to the build server setup and administrator...

Like for example http://pointyhat.freebsd.org/errorlogs/ for
FreeBSD, but I'm not sure what the current MPAB/MPWA status is ?

> * Compressing packages: I think it's a good idea use liblzma, like
> slackware and arch linux are doing. It has a great compression and low
> decompression time, and cost only a little more for compressing than
> usual.

I fully agree, using xz is better than bzip2. There was some
concern about expanding them, but there's both a stand-alone
XZ package* as well as a "xzdec" extractor (a mere 50K or so)
* Universal Binary: http://tukaani.org/xz/xz-5.0.0-macosx.dmg

Support for .txz archives is already included in MacPorts:
http://trac.macports.org/changeset/51232 (since at least 1.8)
I think it's mostly embedded concerns that they are not in BSD,
and like you say a lot of Linux distributions use them already.

> * Signed packages: I thought of it as signing at build time, with an
> GPG key. Not sure if it's the best.

I think it currently uses OpenSSL, since GnuPG is not included ?
But yes, it's about signing at build time with a "known" key...
i.e. one that is known to /opt/local/etc/macports/pubkeys.conf
(might be easiest to look at the base code, to see how it works)

Signing the compression container and not the files content is
not optimal, but "good enough". Another approach in the long run
is including a checksum in the index, and just sign that (once).
The important part is that it at least has a signature somewhere.

> *Others:
>     *What universal binaries and multiple variants on ideas list
> mean? I don't get the idea.

Universal binaries means the +universal variant, that would for
instance build packages with both i386 and x86_64 architecture
so you can use them for 32-bit software too on a 64-bit system.
Other similar variants would be +x11 versus +quartz, and so on.

So the build system could build both foo and foo+universal,
since those could both be requested. The list of variants
should be kept very short though, since it adds exponentially
to the number of archives created... We also need several OS.

e.g.
darwin_9/i386/foo/foo-1.2.3_0
darwin_9/i386/foo/foo-1.2.3_0+universal
darwin_10/x86_64/foo/foo-1.2.3_0
darwin_10/x86_64/foo/foo-1.2.3_0+universal

Otherwise the packages should be built with the *default* variants.
If the variants don't match, it will normally be built from source.

>     *The language must be TCL? Shouldn't we think about using C
> instead in the distribution service to avoid performance issues? The
> compiling itself might be in any language, I think, but since we
> already have MPAB let's keep in TCL.

MacPorts is written in Tcl, with C extensions where it matters.
If there are performance issues with using shell or tcl scripts,
those bottlenecks should be rewritten in C. Eventually it might
_all_ be rewritten in Objective-C, but that's another project...

> About the client:
> 
> There is not much buzz about this part. The suggested extension is
> very attractive IMHO.
> 
> best regards,

The biggest client change eventually is the addition of binary
packages to the MP GUI, so that it can display and install both
source and binary packages. Currently it uses the latest versions,
which in practice would mean something like "binaries if available"

--anders