slow distfiles mirror

[Ryan Schmidt]

> On Aug 8, 2016, at 1:18 PM, Daniel J. Luke <dluke at geeklair.net> wrote:
> 
> On Aug 8, 2016, at 1:58 PM, Ryan Schmidt <ryandesign at macports.org> wrote:
>> On Aug 8, 2016, at 12:49 PM, Daniel J. Luke wrote:
>>> Ryan said: "it's possible for a server to respond quickly to pings but slowly to actual file transfers"
>>> 
>>> I'm interested in what prompted that.
>> 
>> I just imagine that on some networks, it might be possible that the small amount of data used for pings could transfer quickly, while nevertheless that the larger amount of data used to actually transfer files might be slower.
> 
> I'm not sure how that would work (or, in fact, what 'faster' and 'slower' means in this case).
> 
> In the specific case of our mirrors, they all should have adequate bandwidth that the 'best' server for any individual is going to be the one that is closest (in network latency) to the end user.
> 
> The 'ping test' is an attempt to figure out which host is closest.

Yes.


>> I thought this was something that had happened to me on some network at some point in the past.
> 
> actual details of any failures of MacPorts mirror selection would be useful so the system could be improved.

I don't have any details for you; it was years ago.


>> It's also possible that all pings failed, in which case MacPorts does not know which server is best. In that case, I'm not sure whether MacPorts tries servers in the order listed in the tcl file or alphabetical order or what.
> 
> fetch_common.tcl sets a pingtime of 10000 for hosts that fail [comment says "ping failed, so put it last in the list (but before the fallback mirrors)"]

Yes. And if all mirrors have a pingtime of 10000 because for whatever reason ping didn't work, then they will all be considered equally good and MacPorts will use some other method to pick the order in which the servers are contacted.


>> In any case I'm not an expert on network infrastructure. I just know that our mirrors usually work fine, so when a user reports a problem with one of them, it's usually because of something on the user's system like a virus scanner or firewall, or something on the user's network, or some problem with the user's ISP, or a problem somewhere between the user's ISP and the network the mirror is on.
> 
> If there are cases where MacPorts mirror selection isn't working properly, we should try to address them. If it is working fine, we should avoid unnecessarily telling people that it might be broken.

The intention in all of my replies was not to tell people that MacPorts is broken; my intention was to tell the user that their network is broken. You asked for reasons why that might be, and I tried to come up with some possible explanations.

If you have suggestions for improving the mirror selection mechanism we can certainly discuss it. Using ping time has always seemed indirect and possibly inaccurate to me, but it was easy to program and has worked well enough. Years ago I brought up possibly using geolocation, but even if we had the servers' locations and the user's location, there's no guarantee that geographic proximity equals faster network speed.