If you can’t measure it you can’t improve it – Peter Drucker
When a measure becomes a target, it ceases to be a good measure. … Goodhart’s Law
You don’t want to measure how well you are adhering to a framework/method, you want to measure how well you are doing at creating value. – Al Shalloway
Metrics are an important part of any transition or improvement initiative. The purpose of metrics includes:
- Indicate how well you are doing with your process
- Provide insights on what you can improve
- Provide data to indicate if you are improving
- Measure the value you are delivering
- Create visibility on things that are often present but not seen – we can’t manage what we can’t see
- Educates people on importance of looking at time
- Enables better management because they can better understand impact of environment on teams
All of these can be observed with the following types of metrics:
- Metrics that indicate how well the work is being done
- Metrics that indicate how much value is being delivered
- Metrics that are used to create visibility so that we can improve our methods
Metrics That Indicate How Well The Work Is Being Done
- Cycle Time
Cycle time of a work item is the calendar time it takes from start of a work item until its end. It is not the amount of time it takes to work on the work item. In other words, it one work item starts on Monday and is completed Wednesday then it’s a 3-day cycle time. It doesn’t mater if 2 developers worked on it full time or if one developer worked on it only an hour a day. Cycle time is an indicator both of how large work items are coming in and the process cycle efficiency of the work being done. Process cycle efficiency is calculated by looking at the percentage of time people are working on a work item over the cycle time of the work item. For example, if a developer works only on one work item from start to finish its process cycle efficiency will be 100%. If they worked half time on it the process cycle efficiency would be 50%. - Number of defects that make it out of the team
This is an indicator of quality of both the process and the code. - Number of defects that get corrected before work is delivered
This is more of an indication of the quality of the process. These are usually due to misunderstood requirements and missing edge conditions.
Metrics That Indicate How Much Value Is Being Delivered
It is difficult to measure actual value delivered unless you have a good way of measuring it. But one measure that is highly correlated to value delivered is the number of deliveries made. This is because the number of deliveries will be a measure of the size of the items to be deliver and the speed at which they are delivered. As the size of items gets smaller and the speed gets quicker feedback on value will increase. Both are indications of improved process as well.
Metrics That Are Used To Create Visibility So That We Can Improve Our Methods
Interruptions are insidious. Most companies have a significant number of them. And almost all have no real idea how many they have. Management typically believes there are relatively few while the development teams know otherwise. Just telling managers not to interrupt or developers they should not acquiesce to the interruptions doesn’t work. Managers have, at least in their minds, good reasons for asking a developer to do something. The need for this, however, is often a symptom of one or more of the following:
- People feel they can ask for work that isn’t as important as the work scheduled
- The developers are so busy that small things can never get done
- There is not an appreciation for the cost of the interruption.
Management often seems to think that interruptions are not a problem, but they usually are. The best first step is to start tracking them to get a sense of how frequent this problem actually is.
Metrics That Indicate How Much Value Is Being Delivered
# of deliveries is a useful metric. The beauty of it is that it cannot be gamed. If smaller deliveries are done to increase the number that is a good thing – value delivered faster.