[MacPorts] #59286: Mirror distfiles immediately after a commit

[MacPorts]

#59286: Mirror distfiles immediately after a commit
----------------------------+---------------------
  Reporter:  ryandesign     |      Owner:  admin@…
      Type:  enhancement    |     Status:  new
  Priority:  Normal         |  Milestone:
 Component:  buildbot/mpbb  |    Version:
Resolution:                 |   Keywords:
      Port:                 |
----------------------------+---------------------

Comment (by ryandesign):

 Replying to [ticket:59286 ryandesign]:
 > 2. Files are not mirrored until right before they're built. This is a
 problem if all of the builders for 10.9+ are busy. For example, right now,
 all builders except one have been busy for days building all the p5.30
 ports. Commits for other port updates happened after that, but they are
 waiting, and so their files have not been mirrored. Meanwhile, users of
 10.6-10.8 will learn of those updates when they `port sync`, but they
 might not be able to upgrade their ports if upstream uses restrictive SSL.
 Our 10.6-i386 builder is behind the others and has not yet reached the
 p5.30 ports commit, so it is still working through older updates. But
 because the buildbot updates the ports tree to current before each build,
 it already knows about the sqlite3 update, and tries to build it as a
 dependency of other ports, but fails to fetch it, so I have had to take
 that builder offline.

 In fact it's worse than this: the files might never get mirrored at all.

 Consider: During the days that the p5.30 ports were being built, sqlite3
 was updated. After the p5.30 builds finished on the 10.9 builder, a build
 of another port started, which had sqlite3 somewhere in its dependency
 chain, so sqlite3 got built and deployed—but not mirrored, because it
 wasn't in the list of portfiles modified in that commit. Later, when the
 builder got around to
 [https://build.macports.org/builders/ports-10.9_x86_64-watcher/builds/28251
 the commit that actually updated sqlite3], it saw that sqlite3 had already
 been deployed and
 [https://build.macports.org/builders/ports-10.9_x86_64-watcher/builds/28251/steps/subports/logs/stdio
 excluded it from the list]. The distfile would not have been mirrored at
 all, were it not for the fact that the sqlite3 port happens to have other
 subports that still needed to be built.

 If we mirrored distfiles right away, before scheduling builds, we would
 avoid that problem as well.

-- 
Ticket URL: <https://trac.macports.org/ticket/59286#comment:4>
MacPorts <https://www.macports.org/>
Ports system for macOS