Retry on checksum failures?

[Bryan Blackburn]

On Fri, Oct 31, 2008 at 11:14:17AM -0700, David Evans said:
> After updating gimp2 to 2.6.2 last night, an incidence was reported this
> morning of checksum failure
> during a fetch from one site out of many on the port's master_sites
> list.  Fetches from other sites on
> the list were successful and so it appears that this one site had a
> bogus copy of the distribution on
> its server.  Nonetheless, the attempt to fetch failed with no recourse
> for the user other than to
> remove the offending server from the Portfile.  Not something that the
> casual user of MacPorts
> should be expected to do.
> 
> I wonder if it would be a difficult thing to modify port to retry with
> other sites on the list until
> a successful fetch occurs or all listed sites are exhausted?  Isn't this
> the behavior when a fetch
> from one of the master_sites fails for other reasons? (timeout, file
> doesn't exist, etc)?
> 

In theory, trying other servers on a checksum mismatch makes sense, but
there are a few areas where this would be really annoying.  Take, for
example, the texlive_texmf-docs port whose distfile is 255M; for large files
like that, I'm not sure we'd possibly want to re-download it multiple times.

I can think of a couple ways to improve the fetching: adding a distsize key,
and making use of HTTP's HEAD.

With distsize, port can check to see if what it downloaded is close to
what's expected when checksums don't match, and if if it's a really small
file (eg, HTML error page and not the distfile) try elsewhere.  Of course,
this still doesn't really fix what happens on either a stealth upgrade or
corruption.

With using HEAD, port could test first for things like size, maybe even
checking Last-Modified to try and be smarter about what it may be
downloading.

In the end, of course, any possible solution will need someone to implement
it as well...

Perhaps there's another, simpler, solution I'm not thinking about?

Bryan


> Seems that this would make fetches more reliable to the end user without
> compromising verification of the
> validity of the distribution.
> 
> Dave