fetching from closer mirrors?

[Ryan Schmidt]

On Sep 22, 2007, at 17:56, paul beard wrote:

> On 9/21/07, Ryan Schmidt wrote:
>
>> On Sep 18, 2007, at 21:49, paul beard wrote:
>>
>> > as I watch my port upgrade process scroll by (as much as  
>> progress on
>> > an 800 MHz G4 can be called scrolling), I wonder if there is  
>> some way
>> > to fetch from geographically nearer mirrors. Lyon, FR, is a long  
>> way
>> > from here, and even if they have oodles of bandwidth, avoiding
>> > trans-oceanic hops is probably a good idea.
>> >
>> > I know I can hack the order of mirrors but that's not persistent.
>> > Would there any way to add a list per continent or region with that
>> > region specified at install time?
>>
>> As I recall you've brought this up several times, but I think the
>> idea needs to be fleshed out more and more-precisely defined: exactly
>> how would this feature work, how would it behave, how would it be
>> implemented, would there be any downsides, etc. Developer time is
>> limited, and we have very few people who are comfortable coding in
>> base, so a much more precise specification of the feature is needed.
>
> several times? Yikes, I sound like a nag ;-)

Not at all; I just meant that, while I agree in principle that it  
would be more efficient to fetch from closer sources, how to  
accomplish that in a way that makes sense for all of MacPorts is not  
yet clear to me. Or, I could think of a few possible solutions, but  
haven't worked through them completely to see what issues there might  
be.


> I think something as simple as a per continent or region list of  
> mirrors is all that's needed. North America-based installs would  
> use mirrors on that continent, European-based folks would use  
> theirs, etc.

Issues I see here:

1. How do we determine what continent a user is in? Automatically?  
(How?) Or the user specifies it manually? (Where, how?)

2. Not all ports fetch their distfiles from the same servers. Far  
from it. See:

http://trac.macports.org/projects/macports/browser/trunk/base/src/ 
port1.0/resources/fetch/mirror_sites.tcl

Some ports fetch from gnome's servers (there's your Lyon, France  
server), some fetch from gnu servers, some fetch from sourceforge,  
and so on. And many, many ports do not use any servers in this list;  
they just specify a master site URL (or several URLs) directly.

Your "per continent or region list of mirrors" wouldn't help those  
ports that fetch from specific URLs, but let's look at it anyway. How  
would you do it? Would you duplicate mirror_sites.tcl per continent  
and reorder the URLs in each? That would make it inconvenient to ever  
make any changes (add, remove or change a URL; add or remove a  
category) as the change would have to be made in several files at the  
same time. It would be easy for someone to forget, and for the files  
to become out of sync. I think it's a bad idea. It would be better to  
keep a single list, but to add some kind of geographic information to  
each URL, be that country or continent or whatever.


Here is a previous thread you started on this topic:

http://lists.macosforge.org/pipermail/macports-users/2007-June/ 
003826.html

I attempted to direct the thread to the macports-dev list (since it  
relates to the development of MacPorts, not its use) with a lengthy  
reply with several ideas which expand on the above:

http://lists.macosforge.org/pipermail/macports-dev/2007-June/001857.html

There wasn't a whole lot of discussion after that though. Maybe there  
will be more input this time around.


I will attempt the same redirection with this thread now: please  
reply only on the macports-dev list (edit "macports-users" out of the  
Cc list when replying).