slow distfiles mirror

[Ryan Schmidt]

On Aug 8, 2016, at 10:25 AM, Daniel J. Luke wrote:

> On Aug 7, 2016, at 5:22 PM, Ryan Schmidt wrote:
>> It decides this based on lowest ping time. This is not necessarily 100% reliable: it's possible for a server to respond quickly to pings but slowly to actual file transfers.
> 
> If this is true for any of our mirror sites, they shouldn't be mirror sites.
> 
> Mirrors should have adequate bandwith to push data, so the determining factor in transfer speed is going to be the rtt (see also tcp bandwith-delay product).

As I said in my next sentence:

>> The UWaterloo server is often the one chosen for me, and it's been fast for me, but it depends on your unique network situation.


The xorg-randrproto distfile that was mentioned earlier is 139K which is smaller than most web sites and should transfer instantaneously on most any connection.

I tested downloading a somewhat larger file using my current Internet connection (100mbps Time Warner Cable in San Antonio, TX), which transferred at 521kbps. For comparison, I tried our CDN, which in my case was reaching the Dallas CDN datacenter, and that was faster at 951kbps. I also checked pings, and the CDN had lower ping times, so in my case, of those two server, MacPorts would have tried to use the CDN first. But these results are specific to my Internet connection and the results would be different for other users.


$ curl -O http://ykf.ca.distfiles.macports.org/MacPorts/mpdistfiles/mongodb/mongodb-src-r3.2.8.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 27.1M  100 27.1M    0     0   521k      0  0:00:53  0:00:53 --:--:--  537k


$ curl -O https://distfiles.macports.org/mongodb/mongodb-src-r3.2.8.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 27.1M  100 27.1M    0     0   951k      0  0:00:29  0:00:29 --:--:-- 1010k


$ ping -c 4 ykf.ca.distfiles.macports.org
PING mirror.csclub.uwaterloo.ca (129.97.134.71): 56 data bytes
64 bytes from 129.97.134.71: icmp_seq=0 ttl=44 time=64.722 ms
64 bytes from 129.97.134.71: icmp_seq=1 ttl=44 time=69.800 ms
64 bytes from 129.97.134.71: icmp_seq=2 ttl=44 time=65.103 ms
64 bytes from 129.97.134.71: icmp_seq=3 ttl=44 time=64.210 ms

--- mirror.csclub.uwaterloo.ca ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 64.210/65.959/69.800/2.240 ms


$ ping -c 4 distfiles.macports.org
PING distfiles.macports.netdna-cdn.com (198.232.124.36): 56 data bytes
64 bytes from 198.232.124.36: icmp_seq=0 ttl=114 time=17.169 ms
64 bytes from 198.232.124.36: icmp_seq=1 ttl=114 time=20.108 ms
64 bytes from 198.232.124.36: icmp_seq=2 ttl=114 time=29.649 ms
64 bytes from 198.232.124.36: icmp_seq=3 ttl=114 time=23.775 ms

--- distfiles.macports.netdna-cdn.com ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 17.169/22.675/29.649/4.657 ms


$ curl http://198.232.124.36
You are hitting the MaxCDN Dallas Datacenter<br>
<img src=netdna.gif?city=6 >