How distfiles mirrors know to fetch new files

[Ryan Schmidt]

On Feb 21, 2009, at 09:05, David Bannister wrote:

> I update everyday at 2100 EST from the california site, the update  
> is via wget and rysnc.

That sounds like it should work. Any ideas why not all of the files  
from the California mirror are appearing on yours then? See for  
example the directories dcraw, graphviz, lighttpd, xorg-libX11...

> The server is based on FreeBSD 7.1, if someone would like to clean  
> up the macports package to compile successfully on that platform  
> then I would be happy to run 'port mirror all.'

I wasn't aware we had problems compiling on FreeBSD. Are tickets  
already filed for the problems you're encountering? If not, could you  
go into more detail? As you know, the vast majority of MacPorts users  
and developers have Macs and don't have access to other systems like  
FreeBSD to test on.


> On Feb 21, 2009, at 2:50 AM, Ryan Schmidt wrote:
>
>> Dear admins of the European MacPorts distfiles mirrors,
>>
>> It appears that the files on the mirrors are going stale, that is,  
>> files for newly-added or updated ports are not appearing on your  
>> mirrors. Do you have any mechanism in place to periodically update  
>> your mirrors? William Siegrist suggests that you should poll for  
>> changes or just run "port mirror all" from time to time.
>>
>> The California mirror is kept up to date via a post-commit script  
>> that runs as soon as anything is committed to the repository. I  
>> imagine this is possible because the repository server and the  
>> mirror server are near one another, but perhaps there would be a  
>> more general way to have the repository server ping all the  
>> mirrors to tell them it's time to re-fetch a given port, for  
>> example by simply accessing a URL like
>>
>> http://<mirror>/refetch.php/<port>
>>
>> refetch.php could be written to only accept connections from the  
>> repository server to prevent abuse.
>>
>>
>> On Feb 10, 2009, at 20:07, William Siegrist wrote:
>>
>>> On Feb 10, 2009, at 4:58 PM, Ryan Schmidt wrote:
>>>
>>>> We have three distfiles mirrors, in the US, Norway and Sweden:
>>>>
>>>> US: http://distfiles.macports.org/
>>>> NO: http://trd.no.distfiles.macports.org/
>>>> SE: http://arn.se.distfiles.macports.org/
>>>>
>>>> In /graphviz the US mirror has distfiles up to the current port  
>>>> version, graphviz-2.21.20090209.0545.tar.gz.
>>>>
>>>> But Sweden only has files up to graphviz-2.21.20090106.0545.tar.gz.
>>>>
>>>> Norway only has files up to graphviz-2.21.20081216.0545.tar.gz.
>>>>
>>>> I believe the US mirror, which runs right next to the Subversion  
>>>> repository, gets updated in a post-commit hook, so as soon as  
>>>> someone commits a new version of a port, the mirror fetches the  
>>>> new file.
>>>>
>>>> How do Norway and Sweden get informed that they should fetch new  
>>>> files? I fear they're not getting informed at all.
>>>
>>> You might want to CC the mirror owners, but in any case, they  
>>> have to poll for changes (via RSS, mailing list, whatever) or  
>>> just perform full "port mirror <port>" runs periodically.