livecheck checking Portfile's homepage

[Ryan Schmidt]

On Jun 14, 2014, at 7:58 PM, Kurt Hindenburg wrote:
> On 6/14/14, 8:31 PM, Ryan Schmidt wrote:
>> On Jun 14, 2014, at 4:55 PM, Kurt Hindenburg wrote:
>> 
>>> I've noticed that more a few Portfile's have invalid homepage entries.  I hacked/copied/pasted portlivecheck.tcl to have livecheck output errors.  Is there already a way of doing this or a better way?
>> I'm not sure I understand completely. When you say invalid homepage, do you mean a homepage that returns a 404 not found error? or a web server that is offline? or a port having no homepage entry at all? And what is the behavior of livecheck currently in that situation, and what does your patch change it to be instead?
> Yes, the homepage urls are in the Portfile and there's some issue connecting to it.  As far as I can tell, the current livecheck checks the livecheck.url (which appears to be the master_site most of the time).  I didn't see anything that checks the homepage url. With my patch, livecheck also checks homepage (it is a hack/copy/paste).

The default livecheck.url varies. It could be freecode, or if master_sites refers to a sourceforge or googlecode or gnu or gnome URL, then livecheck.url is based on those, or with the github or bitbucket portgroups, livecheck.url is based on those.

I don't think we want to also check the homepage. If the homepage is the correct URL for an individual port's livecheck, it should set its livecheck.url to that.


>> The correct solution, if a port has an invalid or bad homepage entry, is to correct the homepage entry.
>> 
> I agree but I don't see any automated way of checking to find invalid homepage urls.

True, but there's no automated way of finding most other portfile problems either. We fix problems as we find them.

If you think it's a worthwhile endeavor, you could get the complete deduplicated list of URLs used as ports' homepages with this command:

port -q info --index --homepage all | grep :// | sort -u

Then use a URL checker on that list.


> I was more asking for comments about this issue rather than suggesting using the patch.
> 
> Example:
> jbigkit : checking homepage http://www.cl.cam.ac.uk/~mgk25/jbigkit/download/
> Error:   Failed to connect to 2001:630:212:200::80:14: No route to host
> Error: cannot check if jbigkit was updated (Failed to connect to 2001:630:212:200::80:14: No route to host)

I agree, that server is down.

http://downforeveryoneorjustme.com/www.cl.cam.ac.uk

We should contact the administrators of the server, if we can figure out who that is, and see what's up.

This livecheck.url is indeed based on master_sites, but since the homepage and the master_sites are on the same server in this case, also having livecheck check the homepage wouldn't help anything.


> jpeg : checking homepage http://www.ijg.org/files/
> Error:   The requested URL returned error: 403 Forbidden

jpeg port's livecheck used to work. In this case, they appear to have added rules to their server specifically to deny requests when the useragent contains the string "curl" (which MacPorts' useragent does, since MacPorts does use libcurl); that URL works fine in a web browser. We should ask the administrators of the ijg.org server why they have done this and ask if they would consider undoing it.