Measuring and reporting are important

I've often heard said, "That which you cannot measure, you cannot improve." And while I do believe this is a general truth, I think it fails to tell the entire story. It is not just about what we can measure, but what we actually do measure that is significant; "That which is reported, will improve."

What is not measurable, make measurable. -- Galileo Galilei 1564-1642.

Measuring and reporting influence behavior

Metrics give us feedback. Metrics show a measure of progress toward our goals. Metrics remind us what is most important. Visible metrics, ala Big Visible Charts, help to focus the team on what is most important.

...if I have quick access to key metrics every day, my creativity stays within certain bounds – my ideas all center on how to achieve our goals. -- Paul B. Allen

Access to key metrics provides an individual or a team a quick means of assessing progress and focusing their minds on what is most important. These can be excellent tools for influencing a team toward a desired goal or level of performance.

But the outcome is not always what we intend.

The Hawthorne Effect

There is a now famous study that was done in the early 1900s. Hawthorne Works commissioned a study of the effects of ambient lighting on worker productivity. The results of the test were perplexing. Essentially, worker throughput seemed to increase despite the level of light. High, low, medium; with each change, throughput increased. Once the study was concluded, levels quickly fell back to "normal".

The are several criticisms of this particular study, but the phenomenon has been repeated under different circumstances since.

The conclusion; that which was measured, improved. Effectively, because people were aware their throughput was being measured, it increased. In similar studies, people's throughput increased when they were knowingly observed but received no guidance or feedback. So they assumed throughput was the objective

Rock On! Where is my plotter?

So this is pretty awesome, huh? Just let people know you are watching them and their productivity will soar! Throw up a few charts and you've got yourself a high-functioning team.

You wish

So here's the rub. An increase in throughput is not necessarily a good thing.

Why Not?

Do you believe your employees are slackers?
Do you think they lollygag about all morning, marking time until lunch and then perhaps featherbed it to 5pm before they skedaddle out the door?

If you do, then you have a significant problem. A problem that a few charts and some good old fashioned micro-management probably won't solve. But, hey, it is worth a try....

Our employees suck less than that

Good. So you have a reliable, forthright, responsible team of people. If this is the case, then you might do well to be a bit leery of easy increases in throughput. I'm not saying it is a bad thing, but that extra throughput had to originate somewhere. Perhaps it is less time spent in needless meetings. Perhaps it is better attention to priority items. If so...

Huzzah for metrics!

BUT...

You've been warned

That which you measure and report will improve.

If what you track and report is one-sided, your results will be one-sided.

When you are on an agile team and all of your metrics are about velocity, you send a clear message to the team.

Go Faster! quality ain't all that important and value don't matter

Goodhart's Law

Goodhart's Law is a principle first defined in 1975 in a paper by Charles Goodhart, chief economic advisor to the Bank of England. Goodhart's basic premise is that economic indicators made into targets for the purpose of conducting economic policy, thereby lose their ability to serve in the desired capacity.

More succinctly put - When a measure becomes a target, it ceases to be a good measure.

Goodhart's law is applicable to both economics and sociology. It is a proven fact (law) that when you take an indicator such as velocity and you make it a target, you thereby eliminate it as a valid indicator.

Good Metrics

Measure value, not throughput. How many of the high-value stories were delivered, regardless of points.

Focus on your customer's determination of value. How do your metrics tie directly to their single highest priority?

Study trends, not moments in time.

Use metrics as a diagnostic tool. When a problem is identified, devise one or two metrics that will help diagnose and monitor the health of the issue. When resolved, stop measuring.

Do not record or report metrics for reporting sake.

Do not set targets for your metrics.

If you must report velocity, report only team velocity.