[MacPorts] #50875: mirror-macports.sh runs daily but takes > 24 hours to complete

[MacPorts]

#50875: mirror-macports.sh runs daily but takes > 24 hours to complete
----------------------------+---------------------
 Reporter:  ryandesign@…    |      Owner:  admin@…
     Type:  defect          |     Status:  new
 Priority:  Normal          |  Milestone:
Component:  server/hosting  |    Version:
 Keywords:                  |       Port:
----------------------------+---------------------
 The browser:trunk/base/portmgr/jobs/mirror_macports.sh script runs every
 day at 2:15 am on rsync-origin.macports.org, but because of the number of
 ports we have and because it tries to mirror each port for 8 different OS
 X versions (darwin 8 through darwin 15), it now takes more than 24 hours
 to complete.

 When a file is not found or the checksum doesn't match, the fetch is
 attempted 8 times, one for each OS X version, wasting time and bandwidth.

 It mirrors for each version of OS X separately because a small subset of
 ports declare different distfiles for different OS X versions. However the
 vast majority of ports don't do that.

 I think it's also supposed to be mirroring for each variant, since some
 variants might add distfiles.

 We should find a way to rewrite the script so that it gets a list of URLs
 to mirror for all permutations of the port, but only attempts to fetch
 from each URL once.

 The script on the server has been updated quite a bit from the version
 currently in the repository, and there are changes in the repository that
 have not been reflected on the server yet.

-- 
Ticket URL: <https://trac.macports.org/ticket/50875>
MacPorts <https://www.macports.org/>
Ports system for OS X