MacPorts caching of distfiles [was Re: ntfs-3g 1.1120 source missing]

[Anders F Björklund]

Jordan K. Hubbard wrote:

> On Feb 22, 2008, at 7:11 PM, Rainer Müller wrote:
>
>> One of these things would be how we would push files on the mirror?
>> If we keep master_sites we could use something like `port fetch all'
>> in a cronjob or in a post-commit hook. But that will have the
>> problem that it currently port fetch will not remove old distfiles.
>> So this would need improvement.
>> Alternatively committers would have to upload there files manually
>> which would be more work for them and also more error-prone.
>
> 1. I think doing a "port fetch all" is just fine - it will skip the
> fetch step for ports which already have their distfiles in place (we
> can even add a little extra logic to stick them somewhere other than
> in ${prefix}/var/macports/distfiles for the mirroring case) and
> doesn't involve a lot of extra work on our part.  The only thing we'll
> need to make sure of is that our timeout logic is genuinely robust
> since we don't want a single hanging fetch to scrog the entire mirror
> run, ditto for our failed fetch logic - we don't want partial
> distfiles to get left lying around, but both of those are things we
> should have working well anyway.

Before fetching all the files, it would probably nice to fix the
timestamps so that they correctly reflect the upstream distfile ?

http://trac.macports.org/projects/macports/ticket/12629

The --remote-time feature is added, but disabled due to the
old and buggy version of libcurl that is present in Tiger...

--anders