Download sources from servers that don't respond to ping

[Ryan Schmidt]

On Oct 4, 2009, at 23:16, Eric Hall wrote:

> 	Instead of using curl, why not use an internal socket
> connection on "the right port(s)" for the service used to
> deliver distfiles?  That will be tcp/80 for most ports,
> some will have different ports (svn or ftp for example).

svn/git/cvs type connections need not be checked. Either the server  
exists and will work or it doesn't and the port will fail to fetch.  
There are no backup distfile servers for ports that check out from a  
repository and thus don't use distfiles.

> Start a tcp connection, once you get the socket completed,
> check the timing.

That could be a possibility. I don't do this type of network  
programming so I don't know what would be involved.

> That also lets you know if the service
> you want to use is available.

Additional checks to see if the service is available aren't strictly  
needed since we already fail gracefully in the fetch phase (proceeding  
to the next server in the list) if it isn't. Though detecting this  
earlier would allow us to exclude those servers from the list entirely.

> 	In fact, if this was done "at the right place" (TM),
> the "fastest" connection could be the connection used for
> the actual download, and the other connections can be dropped.
> Note that I did not look at the code that implements the
> downloads, it may not be practical to insert timing checks
> and connection-drop logic there.

We use curl to actually download the distfile; if we wanted to re-use  
the timing check connection we would have to implement the downloading  
code ourselves which is probably nontrivial to get correct for all  
possible HTTP/HTTPS/FTP servers...