Mirror size & completeness of binaries

[Ryan Schmidt]

On Mar 27, 2018, at 06:56, Mojca Miklavec wrote:

> I don't know about the best way to do it, but I would like to suggest to provide macports mirorring in two different sizes: a small one and a complete one.
> 
> While I'm a heavy supporter of providing support for legacy systems, I see no reason to mirror files for them on all of our mirrors and cause troubles to them. I would suggest to mirror by default just the latest version of any given source and binary and only support the latest three OSes there. Then we could have additional files to support older systems on a smaller set of mirrors, just on those where it would not cause any additional troubles to them. Since the number of users of legacy systems is much smaller, this should not have a heavy impact on bandwidth to that smaller number of mirrors either.

I do not wish to create separate small and complete mirrors. I wish to keep a single complete mirror at our primary rsync.macports.org server, and leave it up to each individual mirror operator to decide for themselves whether they want to mirror all of it or whether they want to exclude some older systems, and if so which systems they wish to exclude. It is my assumption that it should be easy to specify exclusions in the rsync command. We would update the Mirroring wiki page with those instructions, along with our recommendations for which older OS versions to consider excluding.

At the same time, we need to consider how MacPorts base will respond to the situation. Currently, it assumes all packages are available on all mirrors. Therefore, it tries to find a package on three mirrors before giving up. If many mirror operators choose to exclude an OS version, this could leave a user building from source, even when a binary exists on a more-distant mirror; I want to avoid that. One workaround is to recommend that mirror operators who exclude some packages should also add http redirects for excluded packages to their web server configuration, redirecting those requests back to the main (full) mirror. We can update our sample configurations on the Mirroring wiki page to reflect this. 


> I'm not saying this should be implemented immediately, but I would certainly start thinking about that before we add additional four mirrors (three legacy ones and 10.14).

As I understood our libstdc++ to libc++ transition plan for 10.6-10.8 systems, we will not create new builders. We will instead repurpose existing builders for libc++, and delete the old libstdc++ archives for 10.6-10.8 system when we do so.