[MacPorts] #57343: mpbb mirror-distfiles repeatedly downloads distfiles with checksum mismatches

[MacPorts]

#57343: mpbb mirror-distfiles repeatedly downloads distfiles with checksum
mismatches
---------------------------+---------------------
 Reporter:  ryandesign     |      Owner:  admin@…
     Type:  defect         |     Status:  new
 Priority:  Normal         |  Milestone:
Component:  buildbot/mpbb  |    Version:
 Keywords:                 |       Port:
---------------------------+---------------------
 Consider [https://build.macports.org/builders/jobs-mirror/builds/46329
 this run of mpbb mirror-distfiles] which ran after the qt58 port was
 modified to fix a typo. Upstream may have done a stealth update, because
 31 of the subports' distfiles had checksum mismatches. But mpbb downloaded
 each files 12 times, because mpbb tries mirroring separately for each of
 our platform versions and archs. The 31 files totaled 339MB, downloaded 12
 times is 4GB, which took 53 minutes, during which time no other builds
 could be started. And because we still haven't optimized our buildbot
 setup to trigger the mirror task only once per batch of commits, and
 instead we trigger a mirror task from each of the 12 portwatchers for each
 batch of commits, this 4GB 53-minute download was repeated 12 times.

 The reason why we impersonate each platform while mirroring is so that we
 mirror files that a port only uses on a particular platform. But if a file
 fails to download or checksum properly for one platform, we should
 probably remember that and not try downloading that same file again for
 another platform during this mirroring run.

-- 
Ticket URL: <https://trac.macports.org/ticket/57343>
MacPorts <https://www.macports.org/>
Ports system for macOS