Metric Misuse - The Hawthorne Effect

Now, metics are not bad. But, they are often used in bad ways.

It might help to be aware of some of the side effects of mismanagement of metrics. From inadvertently creating behaviors that actively work against our best interest, to altering the meaning of the metric, mismanagement can do more harm than good.

The Hawthorne Effect

Western Electric had commissioned an extensive study that ran from 1924 to 1932 at their Hawthorne Works in Cicero, IL. The intent of the study was to determine the impact of ambient lighting on worker productivity. Would employees be more productive under high levels of light or low levels of light? The workers in the factory were divided into two groups based on physical location within the plant. For one group, the lighting was increased dramatically while for the other group (the control) lighting levels remained the same. Researchers found that productivity improved among the group for whom lighting changed whereas the control group had no statistically significant change.

Employee working conditions were then changed in other ways. Working hours were adjusted, rest breaks were changed, floors were rearranged, workstations were kept cleaner, and several other adjustments were made, including returning the lighting back to normal levels and changing practices and policies back to original standards.

With every change, productivity made small improvements. By early 1932, and the end of the studies, the factory productivity was at an all-time high and employee attendance and retention were at record-setting levels. Some groups seemed to do better than others, but across the factory, all measures were improved.

When the studies ended, productivity, attendance, and retention soon returned to original levels.

The key takeaway from the Hawthorne studies is - that which gets measured will improve, at least temporarily. “The Hawthorne Effect” is described as the phenomenon in which subjects in behavioral studies change their performance in response to being observed.

This, at first, seems like a precious nugget of management gold.

  1. Measure productivity.

  2. Make it known.

  3. Ka-Pow! Increased productivity.

The perfect management formula.

But the reality was (and is), that while that which is being measured shows improvement, it does not mean the overall system has improved. Working longer hours can lead to employee fatigue and burn out, as well as lower quality. Lack of attention in areas not measured, such as quality or workplace safety, can lead to other negative outcomes.

If your team is slacking so significantly that merely measuring their velocity can result in a marked increase in velocity with no ill- effects, then you’ve a more serious issue at play than velocity.

What’s more, there is no guarantee that the thing being measured has actually improved. Velocity might have gone up because the team inflated story points. We should rephrase the key takeaway to that which gets measured will (appear to) improve.

This article is an excerpt from the book, “Escape Velocity”, available on LeanPub, Amazon, and elsewhere.

Remediating Technical Debt

I no longer believe we can return to the original, healthy and helpful definition of Technical Debt. Nor do I think we’re going to shift away from it to a better metaphor.

But maybe we can talk about remediation of Technical Debt in a way that helps us make better decisions.

The way I see it, Technical Debt Remediation is nothing more than Maintenance. Maintenance is to keep in an existing state (as of repair, efficiency, or validity) or to preserve from failure or decline.

Technical Debt Remediation (Maintenance) comes in different categories – Preventative, Deferred, and Repair.

The Experiment Canvas

In the past few years, we at OnBelay have had the honor of working with companies who are looking to improve their engineering culture.

One key tool we use today is the Experiment Canvas. My partner, Diane Zajac, and I co-developed the canvas. It is based heavily on our experience with A3s. It is still a work in progress, but I want to share with you where we are to date. Please feel free to use it and give us feedback.

Thoughts on Refactoring

Thoughts on Refactoring

I recently worked with a team on a fairly significant refactoring. I paired with different team members each day over a three day period as we moved code around, pushing responsibility into other classes, reducing complexity, and make the code more expressive. On the fourth day, I put together some notes on the things we saw and general guidelines for the team to keep in mind. Nothing herein is new to the community, but it might be new to you.

Delivering Value

A while back, Jason Yip, tweeted about delivery of value and started an interesting thread.

Much of the discussion was about the definition of “value”. Is it specifically about revenue generation or direct customer benefit? Is it more generally about any form of value such as revenue, progress, or learning?

Troubleshooting Velocity

Troubleshooting Velocity

Recently, on a private forum, a member posted a query about their team’s recent drop in Velocity. Concerned about how their boss would respond, this individual wanted to know how to troubleshoot velocity issues.

After spending over an hour crafting a response, I decided I would also add it to my public blogs in case there are others who have similar questions about velocity.

Velocity Anti-Patterns - Attempts to show increased velocity

Velocity Anti-Patterns - Attempts to show increased velocity

When leadership asks for an increase in velocity, there are a few common behaviors that occur. Each of them are an attempt to satisfy the potentially unrealistic ask.

It is intriguing to me how often a manager will make a change such as this to a system of work and then later proclaim that the team is gaming the system. This is simply not the case. In fact, the gaming of the system is the improper application of targets or goals for lagging indicators. The rest is just natural consequence.

You can find more on the topic of velocity and metrics for agile teams in my book, "Escape Velocity".

Velocity Anti-Patterns - Enticing More Velocity

Velocity Anti-Patterns - Enticing More Velocity

Leaders (and teams) attempt to achieve velocity increases in numerous ways. Most, as you can imagine, have unintended side effects on the teams and none significantly improve the actual flow/delivery of value to their customers. Here we explore a few ways teams might be encouraged to increase their velocity. Unfortunately, no matter how well intentioned, using such techniques still has a negative impact.

This content and more on agile metrics is available in the book, "Escape Velocity"

Velocity Anti-Patterns - Demand for higher velocity

Velocity Anti-Patterns - Demand for higher velocity

Demanding more velocity is far and away, the most common Velocity Anti-Pattern, and quite possibly the most harmful. It manifests itself in a number of differing fashions, but the basics are the same: Somebody determines that the team needs to get more done in less time. So they send out the message - “We are going to need more velocities.” This person is usually an authority figure and typically doesn’t do the actual work being asked of the team. And they clearly don’t know what velocity is. More velocities? Aw C’mon, really?

This content and more on agile metrics is available in the book, "Escape Velocity"