The Hand of FuManChu

Divmod has a development methodology which they call UQDS. It's billed as lightweight, but I've been using it for 6 months now and find it burdensome. The basic flow of UQDS is: make a ticket, do all work in a branch, get a full review, merge to trunk. In theory, this brings the benefits of peer review and fewer conflicts between developers. In practice, however, I've found the following problems:

1. Review is great, but is often performed by whomever is "free" rather than by those whom the change most affects. The larger the group affected, the more review happens after the change is merged back to trunk anyway.
2. Conflicts are not reduced, they're delayed and distributed. One branch still gets merged to trunk before others, and those others often must all forward merge.
3. The amount of change in a branch grows too large, and often incorporates changes which have little or nothing to do with the original ticket. Comments and whitespace get touched up from one dev to another; small buglets get found and fixed; refactoring happens. Review is more difficult.
4. Commit history is unreadable. Changesets consist entirely of massive merges from branches and equally massive forward merges.
5. The review overhead isn't worth the supposed benefits; developers spend too much time in review, and often invent bikeshed problems in order to feel their review time is worthwhile. There is no distinction between the review process for tiny versus massive changesets.
6. Many problems still leak through the review process, but reverting trunk changesets is harder because they tend to be too large and mix concerns. That is, you often end up reverting 1/2 a changeset, which subversion does not make easy. UQDS tries to avoid reverting multiple changesets at once, but overcorrects by making fractional reverts more common. With Subversion, at least, I'd rather be saddled with the former.
7. UQDS says, "[developers] can take all the time they need to come up with a good message" when they commit to trunk. In reality, they don't--they just want to finish quickly and move on to the next ticket.

So, here's my answer:

The Xtremely Quick Development System (XQDS)

The goals of XQDS:

1. Code fast.
2. Improve the process documentation.

The strategy of XQDS:

1. Reduce changeset size. This allows everyone to code faster, since they don't need to exert as much mental effort to understand others' changes. It also makes the timeline more readable (and revertible!), since each changeset and its commit message is atomic.
2. Resolve conflicts immediately.
3. Apply review resources only where needed.
4. Make tests easier to run than not run. If your test suite takes so long that you're forced to run it on a remote machine, you've either done something wrong or this methodology is too small for you. Which is fine.
5. Record all decisions: on tickets where warranted, on trunk changesets regardless.

The flow of XQDS is:

1. A task is created in an issue tracker.
2. The task is discussed, on the ticket if possible. If the conversation is long or ephemeral, it may be conducted on IRC or elsewhere; however, at least the final decision(s) should be summarized on the ticket, not on a wiki page, mailing list, or other medium. This step may be ongoing as work is done.
3. Someone accepts the ticket. If work lapses, others are always free to accept it for themselves.
4. The worker does work in their local copy of trunk. Branches are created sparingly as needed, and the worker switches their local copy using svn switch.
5. Work gets merged to trunk as it passes the full test suite in the smallest functional chunks possible. Sometimes "smallest possible" can actually be quite large; however, refactorings, buglets, and doc improvements are committed in their own changesets. In my experience, there's nothing worse than trying to review a changeset that's 5% ticket fix and 95% whitespace changes.
6. Each commit includes a message that MUST describe the actual change; comments about the reasons or context are desirable but secondary. However, commits that directly influence a ticket always reference the ticket.
7. Everyone runs svn up on at least a daily basis, and definitely before committing. Conflicts are resolved as needed with each local copy.

Questions:

• Don't you lose the benefit of branching? What if Jethro checks in broken code and goes to lunch?

  1. Jethro shouldn't be checking in broken code. You fix this by increasing peer shame, not ignoring it or shifting the detection work onto another developer.
  2. This happens even using UQDS. Review by a single developer doesn't catch all possible broken code.
  3. You can still branch whenever you see fit. You're just not required to branch for each ticket.

• Don't you lose the benefit of review? Review helps avoid conflict and also teaches the reviewee.

  1. In my experience, review is better with XQDS than UQDS. First, you apply review resources where they are needed. Not every change needs review. This allows developers to invest more energy into reviews which merit it.
  2. You raise the bar for review: everyone is expected to watch the timeline, svn up often, and help resolve conflicts.
  3. Senior developers are allowed to touch up junior developers' code in situ. It's always faster and often much more effective to show improvements than explain them.
  4. People who care about or are particularly skilled about various aspects of code quality are themselves responsible for upkeep. At some point, you can stop trying to teach Jethro how to avoid contention over concurrent shared mutable resources because he's never going to get it. At some point, you stop trying to make the Twisted developer follow your PEP-8 whitespace conventions because she has no incentive to do so; you fix it yourself and get on with life.

• What if I need to share unfinished code with another developer? Or switch developers mid-feature? Or switch platforms mid-feature? Or switch features mid-developer?

  • Make a branch. XQDS doesn't prohibit this. But it doesn't mandate it like UQDS does. All that Combinator nonsense can be replaced with svn switch and a simple folder rename when you want to put aside some work for a while.

• Isn't trunk broken more often?

  • Not if you have a good local test suite. If your test suite can't detect broken code, you need to write more functional (end-to-end) tests.

• UQDS says it improves information flow to managers. Don't you lose that with XQDS?

  • Not at all. On the one hand, managers use the same tools (e.g. Trac timelines) that developers use to monitor progress. But with XQDS, the timeline is actually readable without all those cross merges. Managers can therefore also write Trac SQL queries, for example, which report developer activity based on the changeset activity.

• Doesn't XQDS require more conflict resolution?

  • Not more, just more immediate. As UQDS notes, this can result in a situation where you feel like you cannot commit your local work because it's now broken due to someone else's changes. But that's a false impression: just svn switch to a new branch and commit your (now broken) changes. You're going to have to resolve the conflict either way; but XQDS allows you to skip making a branch unless you need it.