Report from the Reproducible Builds World Summit 2016

[Clemens Lang]

Hello fellow MacPorters!

>From December 13th to 15th, I attended the 2016 edition of the
reproducible builds world summit [1] to represent MacPorts. This was
this year's follow-up event to the first summit in December 2015 in
Athens (which I have previously covered in [2]).

The number of attendees increased compared to last year's meeting. There
seems to be a growing interest across many projects in reproducible
builds. Unfortunately, while the various BSDs seem to have a continued
interest, I was the only attendee interested in macOS.

The workshop was again organized in an unconference style with different
parallel sessions. A detailed report will be available next year. I'll
cover a number of topics that might be relevant for the MacPorts
developer community below.

Diffoscope & Reprotest
======================
Tooling to help achieve reproducible builds is seeing continued
interest. A number of sessions dealt with the future of these tools and
several improvements have been developed and merged during and after the
summit. Specifically, the multi-file-format diff utility `diffoscope`
(which is also available from MacPorts [3]) was covered in multiple
sessions. I'd like to point out the https://try.diffoscope.org web
service, which may be helpful if you want to avoid the numerous optional
dependencies or a local installation of `diffoscope`.

With `reprotest`, a new tool to help vary different parameters between
builds is taking shape. While it will probably not work out of the box
on macOS, it can be a valuable resource to find out how to set up
automatic testing for reproducibility.

For more information on tools, see [4].

Documentation & Defining 'Reproducible Builds'
==============================================
A big topic of this year's summit certainly was documentation. There
were numerous sessions with ideas, plans and commits to improve the
documentation for reproducible builds that is currently available on the
reproducible builds website [5]. A very specific outcome of the
documentation-related sessions is a definition of what the reproducible
builds effort means with the term 'reproducible builds'. The definition
was published a few days after the summit and is now available at [6].

End-user verification & policies
================================
Several people at the summit were talking about ways to make it easier
for end users to verify reproducibility and changes required in package
managers to ensure only reproducible packages are installed without
interrupting users' workflows. While I did not have time to participate
in these sessions myself, I've seen some good first results in the
reports. If you are interested in these, I encourage you to read the
final report once it is published.

Bootstrapping
=============
One issue that was seen as critical by a number of attendees was
bootstrapping of self-hosted compilers so that trust chains to
precompiled binary compilers are at least documented, if not
reproducible from source. There were signs indicating the formation of a
new movement to document bootstrapping best practices and linking good
examples.

Various other topics
====================
Given the size of the meeting, there were usually about 6-8 parallel
sessions at any given time. I did obviously not have the opportunity to
take part in all of them. Topics that have been covered, but I could not
follow:

 - Buildinfo files [7]: What to do with them and where to go from here?
 - Reproducible RPMs: Which problems exist and how can they be solved?
 - Use cases for reproducible builds: Several improvements to the
   documentation [8] were written and merged.
 - SOURCE_PREFIX_MAP, a draft specification to allow reproducing despite
   varying build paths

On a related note, I tried to reproduce a simple binary using clang(1)
during the summit and noticed that Apple's ld64 stores the path of the
object files it links and their modification time in the debug symbols
of the generated binary, while not supporting -fdebug-prefix-map. This
leads at least to a build path leak into all generated debug symbols for
linked object files, and to complete non-reproducibility for files
compiled & linked in a single call to clang, because clang uses a random
output file name in $TMPDIR for an intermediate object file.

This build path leak seems to be deliberate in [9] in
OutputFile::synthesizeDebugNotes with
| objStab.string = assureFullPath(atomFile->path());
| objStab.value = atomFile->modificationTime();
This could be considered a bug, and we should report it to Apple so they
can fix it.

Acknowledgments
===============
My time, travel and accommodation costs have been sponsored by my
employer BMW Car IT GmbH.


Clemens Lang

[1] https://reproducible-builds.org/events/berlin2016/
[2] https://lists.macports.org/pipermail/macports-dev/2015-December/032016.html
[3] https://github.com/macports/macports-ports/blob/master/sysutils/diffoscope/Portfile
[4] https://reproducible-builds.org/tools/
[5] https://reproducible-builds.org/docs/
[6] https://reproducible-builds.org/docs/definition/
[7] https://wiki.debian.org/ReproducibleBuilds/BuildinfoFiles
[8] https://reproducible-builds.org/docs/buy-in/
[9] https://opensource.apple.com/source/ld64/ld64-274.1/src/ld/OutputFile.cpp.auto.html