Saturday, February 15, 2014

checkbox 0.17.6 released, new release process

Today I've released the next stable version of the classic Checkbox. Checkbox is a hardware testing tool developed by the Hardware Certification team here at Canonical.

After the initial bumpy morning I have released 0.17.6 and opened 0.17.7 for development. The release was built with launchpad, using a this packaging recipe and is available in both testing and stable PPAs. The daily development PPA is already tracking 0.17.7 builds. The release is tracked on the 2014-feb-14 milestone on launchpad.

This was a rather uneventful release but the release process was anything but. It is the first release built entirely on tags and merge requests. The release candidate 0.17.6c1 was branched from trunk , built from the release branch into the testing PPA. The recipe there is particularly interesting. Here is the relevant text:
# bzr-builder format 0.2 deb-version 0.17.6~c1~ppa
lp:checkbox/release tag:checkbox-v0.17.6c1
merge checkbox-packaging lp:~checkbox-dev/checkbox/checkbox-packaging-release tag:packaging-checkbox-v0.17.6c1
We are taking the release candidate tag from the lp:checkbox/release branch and a similar tag from the release packaging branch. This is mechanism allows us to release follow ups. Previously we could just not release if the release had serious issues. Now we can fix issues and release another candidate version.

The release was tested using our standard testing process (full certification run on reference hardware) and after reviewing results, was green-lit for final release.

Still on the release branch, a version bump was committed (to final release version), a new tag was added (using the improved releasectl script) and another version bump (to next development version) was committed. A similar operation was performed in the packaging branch. Both changes were pushed and merged to their respective trunks (for code and for packaging). Those merges kicked off one more build, this time just to have final version everywhere where it matters, using the new pair of tags using the stable release recipe the relevant portion of which you can see below:
# bzr-builder format 0.2 deb-version 0.17.6
lp:checkbox tag:checkbox-v0.17.6
merge checkbox-packaging lp:~checkbox-dev/checkbox/checkbox-packaging tag:packaging-checkbox-v0.17.6
The situation is very similar to what was quoted above for the release candidate. The essential difference is that now the final tags are being looked up in trunk. This ensures that we have actually merged the tags back to trunk and that the release can be reproduced later.

The release process is a little bit heavier than before, due to the extra builds and the extra tagging of the candidate version. We will be working to improve the automation around releases to alleviate that cost and make it a derivative of the fact that a specific tag was placed in trunk. Everything else is just a side effect of that.

This process has some very nice properties. It naturally prevents, at source control level, anyone from releasing duplicate version. It allows multiple releases to be in flight (in testing, preparing for release). It ensures that anyone can rebuild a release or branch off the relevant tag and add a bugfix and release again.

Since this was all pretty much experimental, we haven't written the instructions down in our release policy document but I plan to work on that early next week. We found only two actual issues during this experience. One is easy to fix, that tarmac is not merging or propagating tags. This seems easy enough to fix. The bigger problem is that launchpad is not showing tags in merge requests. In a process where you rely on tags this is a real issue as it requires trust that nobody is sneaking tags behind your back and that all the tags are placed on appropriate revisions.

So this is it, this is the new release process, what do you think? What would you change to make it better?