Buildfarm try

[Simon Ruderich]

On Mon, Oct 15, 2007 at 01:26:30PM -0400, Juan Manuel Palacios wrote:
>
> On Oct 14, 2007, at 11:32 PM, Ryan Schmidt wrote:
>
> 	Very useful indeed! Tabulation of build-run results is another one of the 
> "services" MacPorts is sorely lacking. And I say "services" because I see 
> this functionality more as a server side thing that can be fed day in and 
> day  out by logs created by, you guessed it, 
> http://trac.macports.org/projects/macports/wiki/PortLoggingProposal ;-)

This would be indeed a very useful service. If we could create a (private)
distributed build farm which each builds ports and reports the results back to
the server it would be a very good addition.

> [snip]
>
> 	I would totally love it if any scripting that's put together to preprocess 
> build result logs is done openly (that is, in our svn ;-) and in 
> coordination with the login proposal above and the new website. Admittedly 
> there's still a lot to discuss to even figure out some basic guidelines for 
> the development of such script(s), so basically lets consider this message 
> of mine a call to start tying some of those loose ends. For starters, I 
> propose that, at least for starters:
>
> *) the script(s) is (are) written in php and checked into some subdir of 
> trunk/www (trunk/www/buildresults/ ?), so that the resulting web pages tie 
> nicely into our new project homepage.

I like the idea of a server part in php as it's easy to code and maintain. For
the client I would prefer ruby, because it's installed on each mac and is very
powerful in parsing the port output and also easy to read.
But that's a detail we can work out later.

But the question is if we should "wait" for an integration of this logging
feature to be integrated into macports or if we first try to create a
(distributed) buildfarm which parses the port output and later uses the
macports logging feature.

> 	What I envision moving forward is a full blown section of our project 
> website dedicated to the tabulation of build-run result logs once we get our 
> infrastructure (both hardware and software wise) in shape to host the 
> build-runs themselves. Links to this new section could be created anywhere, 
> like on the sidebar (to an introductory web page for our build-run results) 
> and off the "Available Ports" page (which could sport direct links to 
> individual ports build result tables), among others.

This would be a real nice feature and so everybody who wants to help macports
can just grab a port name and try to build and fix it on his own and then then
the patch to us. It would also make it clear how much of our ports are not
working.

> [snip]
>
> 	Much of the protocol for build-run results to be reliable still has to be 
> defined (things like build frequency and creation of a "clean" environment, 
> among many others), but getting this show on the road by taking some stabs 
> at how results page might look like (the proposed scripts above) is by no 
> means out of the question, in my opinion.

The question is how we want to design the logging/buildfarm features.

I would suggest we use a line based logging as the first try, as it's very
easy to parse/handle and also easy to generate. Maybe we can use xml later,
which is by the way good integrated into ruby.

The buildfarm is also another section. Should we create a real server based
build system where each build computer polls the macports server for a port to
build and then sends the results to the server, or should be make simpler by
just making it based on a computer which tries to build everything and then
sends it to the server.
I like the first approach (distributed) better but it's a bit more complicated
as we also have to create an own protocol to poll the server.

> 	So... Simon and Ryan, did I manage to nourish your interest into putting 
> some lines of php code together? ;-)

Of course ;-) It would like to help macports with this interesting problem.
But before we can get into coding we should collect as much ideas as possible
so we don't waste any energy.

Ryan, Juan and everybody who likes to help with this or has any ideas. What do
you think about our/my ideas/comments? Any ideas are welcome.

> 	Regards,...
>
> -jmpp

Thanks,
Simon
-- 
+ privacy is necessary
+ using http://gnupg.org
+ public key id: 0x6115F804EFB33229
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 186 bytes
Desc: not available
Url : http://lists.macosforge.org/pipermail/macports-dev/attachments/20071017/3a4cf029/attachment.bin