HOWTO: Get started, gain macports-foo, make bad first impression

[Jay Levitt]

Ryan Schmidt wrote:
> The other problem with parallel builds is that they are not identical 
> each time you run them. The build may succeed 3 times, then fail the 
> 4th. I personally haven't had the time to try to build each of my ports 
> several times with parallel build on to see if they repeatably build 
> correctly.

Good point.  So, to the extent that feedback has any automation attached 
to it, "bad" should probably be weighted more heavily than "good".  A 
successful build is "absence of evidence" of problems, not evidence of 
absence.  That said, with enough data points, if you get one "bad" build 
and eight "good" builds on what's ostensibly the same platform, there's 
probably something unique about the bad build configuration that you 
aren't measuring.


> Sounds nice to me. But how would we learn what port works on what OS / 
> architecture? I had proposed last year the idea of MacPorts 
> automatically sending information back to the mothership regarding what 
> ports people had installed on what OS and what architecture, which could 
> be used to determine which ports actually work, and could also show port 
> popularity. I thought this would make for an interesting front-page 
> www.macports.org display, showing the most popular ports. But several 
> people didn't want MacPorts reporting this information:
> 
> http://lists.macosforge.org/pipermail/macports-dev/2007-May/001604.html

I think that post was too focused on getting ultra-precise stats, which 
will always end up compromising anonymity.  See for example the 
Netflix/IMDB privacy leak:

http://www.schneier.com/blog/archives/2007/12/anonymity_and_t_2.html

How many active users do we think Macports has?  A thousand, ten 
thousand?  It doesn't matter if there are some GUIDs that we track that 
no longer get used.  We don't actually care if there are 53 users with 
jigdo installed, or 57; we care if 95% of them had no trouble building 
on 10.5.2 with a Core Duo.  If you try to be too precise, you end up 
with the Windows DLL registry: you can't uninstall a DLL, because some 
program, somewhere, registered it twice, and now the "lock count" is off 
by one.

So if we generate a GUID and it stops reporting to us.. that's fine. 
(If we need a more accurate "current user" count, we can always age off 
non-reporting GUIDs.)  If the user reinstalls, fine.  If some people use 
svn up, that's fine too.  In aggregate, the numbers will still be 
useful. It's a dial, not a switch.  Hell, you could even go as far as "I 
want to install ports that are at least 87% successful"; the named 
"stable" and "unstable" distributions could just be default dial 
settings.  (N.B. This is called over-abstraction.)

I envision something more like:

1. At install: "Would you like macports to send anonymous reports about 
build quality? Yes, No, More Info [N]"
2. Upon build: ...some data, samples of which we show at step 1
3. A nice dashboard on macports.org
4. When a build fails, offer the user the opportunity to opt in again 
(with a "never ask me again" option, of course).

I feel like I've seen a good build-farm dashboard somewhere, but I can't 
recall which project at the moment.

What sort of capabilities do we have on "the mothership"?  I see it runs 
GForge, which I know nothing about; can it run arbitrary servers on 
arbitary ports?  I'm starting to wonder if there aren't generic 
"talkback" solutions we could take advantage of.

Jay