How distfiles mirrors know to fetch new files

[Ryan Schmidt]

Dear admins of the European MacPorts distfiles mirrors,

It appears that the files on the mirrors are going stale, that is,  
files for newly-added or updated ports are not appearing on your  
mirrors. Do you have any mechanism in place to periodically update  
your mirrors? William Siegrist suggests that you should poll for  
changes or just run "port mirror all" from time to time.

The California mirror is kept up to date via a post-commit script  
that runs as soon as anything is committed to the repository. I  
imagine this is possible because the repository server and the mirror  
server are near one another, but perhaps there would be a more  
general way to have the repository server ping all the mirrors to  
tell them it's time to re-fetch a given port, for example by simply  
accessing a URL like

http://<mirror>/refetch.php/<port>

refetch.php could be written to only accept connections from the  
repository server to prevent abuse.


On Feb 10, 2009, at 20:07, William Siegrist wrote:

> On Feb 10, 2009, at 4:58 PM, Ryan Schmidt wrote:
>
>> We have three distfiles mirrors, in the US, Norway and Sweden:
>>
>> US: http://distfiles.macports.org/
>> NO: http://trd.no.distfiles.macports.org/
>> SE: http://arn.se.distfiles.macports.org/
>>
>> In /graphviz the US mirror has distfiles up to the current port  
>> version, graphviz-2.21.20090209.0545.tar.gz.
>>
>> But Sweden only has files up to graphviz-2.21.20090106.0545.tar.gz.
>>
>> Norway only has files up to graphviz-2.21.20081216.0545.tar.gz.
>>
>> I believe the US mirror, which runs right next to the Subversion  
>> repository, gets updated in a post-commit hook, so as soon as  
>> someone commits a new version of a port, the mirror fetches the  
>> new file.
>>
>> How do Norway and Sweden get informed that they should fetch new  
>> files? I fear they're not getting informed at all.
>
> You might want to CC the mirror owners, but in any case, they have  
> to poll for changes (via RSS, mailing list, whatever) or just  
> perform full "port mirror <port>" runs periodically.