Is it time to start regression testing yet?

[Scott Haneda]

On Jun 8, 2009, at 2:18 AM, Ian Eiloart wrote:

> So, and automated test suite would (a) get errors diagnosed and  
> fixed quicker, (b) reduce the number of errors as a result, and (c)  
> give us a way of flagging them before a user starts a 20 minute  
> build process.


I have been reading this thread, looks like the current project I  
looked at with the web listing is really nice.

Curious, what if MacPorts were to implement a sort of CrashReporter  
type feature.  Granted, this is not a crash, but a system where the  
output of the build errors were sent somewhere, into a database, where  
they could be looked at.

I can see a few other values to this as well, and I believe there are  
a number of simple open tools out there that would facilitate making  
this easy to do.

The nice thing as I see it, is end user support.  If MacPorts has an  
error, the app could present them a message "Sorry we could not build  
port x, to date, there have been x build errors with this portfile.   
We have assigned this build error an id of x.  If you would like to  
follow or start a discussion of this issue, please follow this link: http://eample.com?id=x 
.  Users may have had similar issues, which you can see at http://example.com?id=x&porfile=y

These build errors could be auto posted to a mailing list, put into  
trac, whatever was wanted really.  The ability to help a user along,  
in order to get them to help out MacPorts, may be something worth  
looking into.

It is also a very good way to get an exact idea of what ports are  
being installed.  You can then see that port x has been installed 400  
times, 390 of those with no issues, 10 of those with some issue.
-- 
Scott * If you contact me off list replace talklists@ with scott@ *