[mpbb] branch master updated: Add smarter distfile mirroring script

[Joshua Root]

On 2018-3-10 16:12 , Ryan Schmidt wrote:
> 
> Well I did cancel it... Looking at the log, the list of ports isn't being deduplicated. Mirroring ncurses was attempted 506 times, gperf was attempted 687 times, etc.
> 
> https://build.macports.org/builders/jobs-mirror/builds/1212/steps/mirror/logs/stdio
> 
> The next run looked better, with 21 more ports being marked completed in the mirrorcache within the first 3 minutes. But then nothing further for 20 minutes. I cancelled that too so I could look at the log:
> 
> https://build.macports.org/builders/jobs-mirror/builds/1213/steps/mirror/logs/stdio
> 
> It's still mirroring the same ports multiple times, even ports like libiconv that are marked in the mirrorcache.
> 
> I'm going to cancel the remaining mirroring jobs for this commit. We'll be able to debug this better looking at logs from commits of fewer ports.

That should be fixed. It was again only a problem in the case of
failures, which are apparently going to be common. Stuff like this is a
real problem for figuring out if a port should be considered
successfully mirrored or not:

--->  Fetching distfiles for clang-3.4
Error: clang-3.4 is not supported on macOS Sierra or newer.
Error: Failed to fetch clang-3.4: unsupported platform

--->  Fetching distfiles for xattr
Error: xattr cannot be installed for the configured universal_archs
'x86_64 i386' because it only supports the arch(s) 'i386 ppc'.

- Josh