Is measurement evil?

A while back Dave Snowden of the Cognitive Edge (formerly Cynefin) posted a well-aimed comment about the UK Government's folly in trying to fix their outcome-based measures.

I first met Dave Snowden when he presented a keynote at XPDay5. I was bowled over by his talk, as were many of the audience.

Not so Tom Gilb, who was sitting next to me in the talk. Perhaps I shouldn't have been surprised: most of Tom's professional life has been focused on helping organisations to improve thorough quantification and measurement.

Both Dave and Tom are brilliant thinkers and I have been pondering ever since how two such clever people could hold such diametrically opposed views.

I had a delightful talk about this with Clarke Ching at XPDay6. which helped to clarify my ideas. I've tried to summarize my understanding below.

1. If I am trying to manage others, it's useless for me merely to set them measures. My team will game them. I'll get what I measure. It will not be what I want. (I think that's at the core of Dave Snowden's position. It's a sufficiently common view that Duncan Pierce and Jason Gorman ran a session about it and XPDay5.)
   
2. If I am a customer trying to define my requirements, it is sensible to quantify the qualities that I require of my supplier's product or service. (I think that's at the core of Tom Gilb's position).
   

In order for 2. to work, I must be able to:

• quantify the qualities that I require
• agree scales of measure for those qualities
• agree realistic and achievable targets based on those scales of measure
• monitor progress towards those targets (weekly if possible)
• and negotiate changes to both the measures and the targets as appropriate.

This process relies on certain norms of behaviour by both parties. If either party seeks to cheat the other the process will break down. I think that this leads to the final precondition for Gilb's approach to work:

• Either party must be able to terminate the relationship at any time without direct financial penalty.

Dave Snowden can produce plenty of examples where these conditions were not met and outcome-based measures were disastrous.

Tom Gilb can produce plenty of examples where these conditions were met and the results were excellent.

Conclusions:

• both Dave Snowden and Tom Gilb have something valuable to tell us!
• when we use measures to manage a supplier relationship, we must make sure that either party must be able to terminate the relationship at any time without direct financial penalty.