Oh, the Coder and the Tester can be Friends, can be Friends...

(with apologies to Oklahoma!)

Elisabeth Hendrickson just tweeted about a thought-provoking analysis by Michael Bolton of the much-vaunted Continuous Deployment at IMVU.

I'm a big fan of TDD, Automated Tests and Continuous Integration, but I'd hate to develop without help from good Testers. A couple of years ago I worked on a dream project; a very small, highly skilled team which included a couple of developers, a tester and a business analyst. We were pair-programming, and my pairs came from a pool that included some of the best agile developers in the UK. The Tester and Business Analyst were also outstanding.

Our defect rate was very low: five defects in UAT, none in production over a twelve-month period. There's no doubt that this was a team effort; the Tester and the BA both caught many defects that were caused by problems we developers hadn't though of, and therefore never tested for. Equally, the developers tested so thoroughly that the Tester and BA could focus on the hard stuff, where their skills contributed maximum value.

For reasons that need not concern us, the application initially went into production on a machine that was controlled by the developers. We could have done continuous deployment, but we didn't. We always asked our Tester to do a final check-through on our staging box; mostly he found nothing, but once or twice he found defects that would have been real embarrasments in production. Of course, that wasn't his only contribution; our acceptance tests were the product of very close collaboration by the whole team.

So it was no surprise to read to read about Michael Bolton's experience when he did some manual testing of the IMVU site. IMVU practise Continuous Deployment, and claim to rely entirely on Automated Tests. Read Bolton's 50 Deployments A Day and The Perpetual Beta and see the consequences.

Unit testing builders

Should one unit-test test Builder classes? If so, how?

A well-designed application is a composed network of simple components. Many developers use one or more builder classes to construct applications.

Builders can get quite complex.

In many cases we want a single component to be shared by several client components. It's easy to get this wrong. If we do so, we may build a network on which all dependencies appear to be satisfied, but we have actually created multiple copies of components when a single instance should be shared.

Builders often use lazy initialization, in which case they have state. This introduces additional opportunities for error.

Since it's easy to get builders wrong, we need to test them. End-to-end tests will usually tell us if we've build out application without all the necessary parts, but they don't pinpoint the problem, and they may not tell us if we've mistakenly introduced multiple objects instead of sharing a single one.

This suggests that we need to write unit tests for builders, but that's often difficult. The classes that are built will hide their implementation (as they should) so unit tests cannot easily verify that the right internal components are there inside them.

I'm coming to the conclusion that builder tests need to break encapsulation. That's normally a major code smell, but the nature of builders seems to leave no alternative.