MacPorts caching of distfiles

[Jordan K. Hubbard]

On Feb 23, 2008, at 8:29 AM, Ryan Schmidt wrote:

>
> On Feb 23, 2008, at 10:05, William Siegrist wrote:
>
>> I didnt mean to add any unnecessary work for MP. My main point was  
>> that per-commit versus daily is the same work for me. I dont have  
>> any hard stats, but I suppose most Portfile changes result in some  
>> sort of distfile change (most are version bumps?), so we can  
>> probably just make it work with a "port fetch". We already go  
>> through the trouble of pulling aside a fresh copy of the Portfile  
>> during post-commit for linting, so we can just add in the fetch at  
>> that point.
>>
>> So whats left? I think you'll want a special hostname for these?  
>> Maybe http://distfiles.macports.org/<group>/<port>/<file>?
>
>
> Is this going to be a distfiles backup in case the original site  
> goes away? Or is this going to be a primary fetch location, before  
> the original master_sites? I assume the former, since we wouldn't  
> want to stop using the lovely global distributed server network that  
> for example sourceforge provides, would we?

"Depends."   For the sourceforge case, I can see merit to the argument  
that it should be last in the search path.  For almost everything  
else, however, there's merit to the argument that it should be first  
since few other individual distribution points can compare to Apple's  
mighty multi-gigabit bandwidth and peering points.

Looking at the percentages, we see:

$ find dports/ -name Portfile|xargs egrep master_sites[[:space:]] 
+sourceforge|wc
      891    1822   51961

891 ports out of 4500.  I'd say that's a strong argument for it being  
a primary fetch location.

> What, if anything, will we do with MacPorts-hosted distfiles that we  
> currently have in the repository? The repository has never been a  
> great place for distfiles to live IMHO.

Agreed.  I see no reason they couldn't also be hosted in the same way.

- Jordan