Build Reproducibility Workshop Report

[Ryan Schmidt]

On Dec 10, 2015, at 17:33, Clemens Lang <cal at macports.org> wrote:
> 
>> On Wed, Dec 09, 2015 at 05:21:09PM -0800, Ryan Schmidt wrote:
>> Apple hardware is available. Virtualizing OS X on Apple hardware is no
>> problem; we do it today with our existing buildbot builders.
>> [...]
>> I'm open to changes in the buildbot builder setup or replacing it, if
>> there's something better.
> 
> That's good news, but my impression is that our buildbot setup is
> currently already quite heavily loaded. Do you think it could handle
> building every port twice? How much headroom do we have?

Let's assume that getting more CPU power would not be a problem. 

> 
>>> To fix the timestamp issues, I am looking for a suitable value to use as
>>> SOURCE_DATE_EPOCH and then add a find statement before creating the
>>> archive that will put an upper mtime limit on all files to be packaged.
>>> I am not yet sure what a good (reproducible!) timestamp might be:
>>> - The Portfile mtime would be perfect, but is not preserved by
>>>  Subversion, so we cannot rely on it. It is preserved by our rsync
>>>  sync, but the mtime in that is probably meaningless since it's the
>>>  one generated on the rsync server during svn update.
>> 
>> And it differs between different rsync mirrors, because they mirror at
>> different times. We should examine our mirroring strategy and fix it
>> so the mirrors are true copies, including metadata like timestamps.
> 
> Yes, mirrors should sync with rsync -t. Isn't this standard practice for
> most mirrors already?

There were never standard instructions for how to set up a mirror. Each mirror admin did it however they thought they should. 


> 
> 
>> I think the problem is that the rsync server is where the portindex is
>> generated. Since use-commit-times=yes is such an obvious Subversion
>> feature, there must be a good reason why it hasn't been enabled on the
>> server yet.
> 
> Yes, for this to work correctly use-commit-times=yes would have to be
> enabled on the rsync server. My guess is that it isn't enabled because
> it isn't the default and nobody bothered changing settings from the
> default.

But it thought we were saying the portindex wouldn't be generated correctly if use-commit-times=yes. I certainly remember problems along those lines years ago...