Cleaning out the dustiest Trac tickets (was Re: CI system for PR builds)

[Perry E. Metzger]

On Wed, 11 Apr 2018 11:48:22 -0400 Craig Treleaven
<ctreleaven at macports.org> wrote:
> At the moment, only 18 of 403 open submission tickets are less than
> 12 months old.  A further 23 are between 12 months and 2 years
> old.

The PR queue was pretty long (about five pages of stuff!) when I
started working on it. It is now down to only three PRs. This took
about four months of very part time work, mostly time I'd have
otherwise devoted to mindless pursuits like sudoku.

The way I fixed this was pretty simple: to try to keep the number open
going down on average with time rather than up.

1) I tried to clean out low hanging fruit from the most recent PRs,
   because recent ones generally got rapid feedback from submitters,
and
2) I started working my way backwards from the oldest ones trying to
   get people to either finish or close them. For some, this meant
   that if the submitter was completely unresponsive we just closed
   them knowing they could be reopened at will. For many others, we
   managed to get people to clean things up and we got them committed.

You say we have about 41 submissions from the last couple of
years. Those sound like they'll be doable fairly quickly, and so those
are the ones we might want to convert over to Pull Requests or at
least to concentrate on in Trac.

I'd say go through them the same way I went through the PRs: clean
out the most recent ones that are easy part of the time, work our way
back from the two year line forward for the older ones.

As for the really dusty ones, things more than two years old, I would
recommend that people go through the oldest ones a bit at a time,
merge any obvious easy ones, and that if we were waiting for input for
more than six months that they be closed with "feedback request timed
out" or some such.

If there's some Trac magic we can do to make the
process of dealing with the dusty ones easier that might help.
Probably being able to mark who is looking at which of the dusty ones
(if anyone) will make it easier to find ones no one is looking at and
not to forget which ones one is trying to clean out.

As for what to do going forward: as we've already discussed, having
new submissions and patches go through GitHub's PR system helps a lot,
because the submitter is able to fix the problems with their own PR,
because we have a CI system that (when it isn't broken) tells us if
the submission actually works, because we have a review system that
lets you tag the appropriate set of people (often not just one) to
look at the thing and explicitly comment line by line on the submitted
diffs, etc.

So, I would recommend that going forward, we try to limit the number
of new submissions and patches that arrive via Trac, and try to slowly
clean out the Trac backlog. It might take six months or a year before
most of it has been dealt with, but it is possible to make a little
progress every day.

Biggest point: I've been doing open source projects a long time, and
the greatest mistake people make with the trouble ticket queue is
treating ticket systems as indefinite reminder mechanisms. As the
ticket queue fills up, it becomes harder and harder for people to see
all the open tickets that they might work on, even when things could
be dealt with in moments, so the rate at which tickets close slowly
goes down as the queue size goes up.

Eventually the ticket system resembles a mountain of unfulfilled
dreams and everyone is too scared to try to attack it.

This is totally fixable, though. The way to undo this is to slowly cut
the size of the ticket queue down, a little bit at a time, to
institute processes so that new tickets get handled fairly quickly
(which helps make the queue slowly shrink), and to make sure that the
things that get left open are really things that should be open (like
important projects that really can't be worked on now but really
_will_ be worked on someday and need documentation.) (Ideally you have
a separate queue for the projects vs. real user trouble, but never
mind that.)

> IOW, there is very little low-hanging fruit among the 400 open
> tickets.

That's fine. To assault this, we clean out what can be cleaned out,
close the things where one is in indefinite timeout with the original
submitter, slowly push the rest either to being pulled or closed. It
will take time, with 400+ tickets probably months, but it can be
done. Having better tools will help significantly going forward.

> Perhaps it is rude, but I think we should auto-close submission
> (and request) tickets that haven't had any activity for a period
> (say 4-6 months, maybe less).

I think that's a good thing if we've been waiting for a reply from the
submitter for that long. It's less fine if it was the fault of our
team. The former is not rude.

> How many submitters would actually wait for any length of time?  If
> you need or want a piece of sotware enough to create a submission
> ticket, why would you wait passively for months or years?  If a
> ticket isn't acted on in, at most, a few months the submitter must
> have found another way to scratch their itch.  Or the itch wasn't
> serious.

This is part of why I've been keeping the PR queue as clean as I can,
so that people feel like their fixes will be acted on quickly. I think
with time we can fix the Trac queue as well.

Perry
-- 
Perry E. Metzger		perry at piermont.com