Should preferred_hosts honour relative ordering within the line?

[Saj Goonatilleke]

On Tue, 24 Dec 2024, at 03:59, Ryan Carsten Schmidt wrote:
> Normally, MacPorts tries hosts in ping order. Without having looked at 
> the code, I assume the same applies to preferred hosts: that they are 
> tried in ping order, before any of the other hosts.

The ordering of the preferred_hosts subset does not appear to correlate
with RTT nor mirror server response time.

-- 8< --
% ping -c 2 aarnet.au.packages.macports.org
PING mirror.aarnet.edu.au (202.158.214.106): 56 data bytes
64 bytes from 202.158.214.106: icmp_seq=0 ttl=53 time=24.106 ms
64 bytes from 202.158.214.106: icmp_seq=1 ttl=53 time=26.713 ms

% ping -c 2 mirror.fcix.net
PING mirror.fcix.net (23.152.160.16): 56 data bytes
64 bytes from 23.152.160.16: icmp_seq=0 ttl=56 time=166.605 ms
64 bytes from 23.152.160.16: icmp_seq=1 ttl=56 time=155.940 ms

% cat curltime 
#!/bin/sh
exec curl -w @- -o /dev/null -Ss "$@" <<'EOF'
    time_namelookup:  %{time_namelookup}\n
       time_connect:  %{time_connect}\n
    time_appconnect:  %{time_appconnect}\n
   time_pretransfer:  %{time_pretransfer}\n
      time_redirect:  %{time_redirect}\n
 time_starttransfer:  %{time_starttransfer}\n
                      ----------\n
         time_total:  %{time_total}\n
EOF

% ./curltime http://aarnet.au.packages.macports.org/pub/macports/packages/cowsay/cowsay-3.7.0_0.any_any.noarch.tbz2.rmd160
    time_namelookup:  0.002164
       time_connect:  0.026148
    time_appconnect:  0.000000
   time_pretransfer:  0.026356
      time_redirect:  0.000000
 time_starttransfer:  0.062440
                      ----------
         time_total:  0.062725

% ./curltime https://mirror.fcix.net/macports/packages/cowsay/cowsay-3.7.0_0.any_any.noarch.tbz2.rmd160
    time_namelookup:  0.129084
       time_connect:  0.336007
    time_appconnect:  0.664021
   time_pretransfer:  0.664135
      time_redirect:  0.000000
 time_starttransfer:  0.968194
                      ----------
         time_total:  0.968460
-- >8 --

Empirically, at least, it would seem that something else is going on.

https://github.com/macports/macports-ports/blob/f93beda82912cd4d525e0ec83d0af2447902e34b/_resources/port1.0/fetch/archive_sites.tcl#L48-L49

What if I swap these around?

set portfetch::mirror_sites::sites(macports_archives) [lsearch -all -glob -inline -not "
    ${fastly}://packages.macports.org/:nosubdir
    ${nue.de}://nue.de.packages.macports.org/:nosubdir
    ${aarnet.au}://aarnet.au.packages.macports.org/pub/macports/packages/:nosubdir
    ${fcix.net}://mirror.fcix.net/macports/packages/:nosubdir
    ...

-- 8< --
% sudo port install cowsay
--->  Fetching archive for cowsay
--->  Attempting to fetch cowsay-3.7.0_0.any_any.noarch.tbz2 from http://aarnet.au.packages.macports.org/pub/macports/packages/cowsay
--->  Attempting to fetch cowsay-3.7.0_0.any_any.noarch.tbz2.rmd160 from http://aarnet.au.packages.macports.org/pub/macports/packages/cowsay
--->  Installing cowsay @3.7.0_0
--->  Activating cowsay @3.7.0_0
--->  Cleaning cowsay
--->  Scanning binaries for linking errors
--->  No broken files found.                             
--->  No broken ports found.
-- >8 --

That works.  The aarnet mirror was hit in three successive tests.

If the order in archive_sites.tcl is indeed authoritative at the moment,
would it make sense to allow users to tailor this ordering in macports.conf?
(I can go file a feature request and wait, or learn TCL.)

Separately, you may want to bump the aarnet mirror down the list if most
of your users are based in the Americas and Europe.  This mirror is
great for the Aussies, but probably not so great for everyone else.

> CDN throughput is poor everywhere. I need to figure out if there is a 
> change I can make to the CDN settings to improve it. It has been a few 
> years since I asked them about it. Maybe it's time to ask again.

Ah, OK.  In case it helps, ~350 KiB/s was fairly typical from cache-syd*.
I did not take a packet capture, but pv(1) would seem to indicate the
data comes in fits and bursts, often dropping to a rate of zero.

Let me try this...
http://packages.macports.org/packages/llvm-17/llvm-17-17.0.6_1.darwin_24.x86_64.tbz2

112,411,514 bytes in almost exactly five minutes ~= ~366 KiB/s (3 Mbps)