How do you know your agile team is successful? How do you measure improvement and the payoff of moving away from traditional waterfall practices?
Managers have asked me to plot velocity and throughput each Sprint to see any fluctuations. I’ve also received requests for maturity model (or matrix) assessments. Both of these ideas make me cringe. Velocity and throughput are great examples of output metrics; respectively, they measure the effort/points and count of work items delivered, not the value or impact of the work. They are also gameable which means that teams can effort a work item higher to artificially inflate velocity or slice work thinner so that it artificially inflates throughput. Don’t forget – they may be impacted by events such as holidays and vacations, so fluctuations may not be due to team inadequacies but rather by less time to work. Hence, these measures are good for planning and forecasting and should not be used as a success metric.
On the other hand, maturity assessments are nice to get a baseline and look at how a team has improved (or not) over time; to me, it is more about the conversation than the score. Teams can use this opportunity to dig into their challenges and strengths in their current operating environment and develop action items for improvement, as if a Retrospective. I tell my teams to avoid getting caught up in whether they scored a 2 or 3 and focus on why that was the score and how they can improve their agile mindset and practices accordingly.
Working with my Scrum Teams, I do like to track cycle time because it gives insight into time to delivery and whether they have a level of predictability and consistency. I also like measuring work in progress (WIP) and work item age because these measures help teams identify the impact of context switching or trying to do too much at once without delivering items to our customers. Teams are able to come together in Daily Scrum to discuss not just that something is “in-progress” but that something is stalled and requires collaboration or that they noticed each person seems to be going in a different direction instead of upskilling and cross-skilling each other. Essentially, these measures are enabling conversations that were otherwise absent and helping the team to own improving their interactions and delivery to our customers.
Outcomes are a great measure; for example, in a call center, an output metric may be “number of calls taken” while an outcome metric may be “call reason resolved.” I would rather have employees aiming to resolve each caller’s reason for contacting us than aiming to get them off the phone quickly so they can grab another incoming call. Another example is in a hospital setting; an output metric may be “number of prescriptions written” while an outcome may be “percentage of accurate diagnosis and appropriate treatment." In general agile organizations, outcomes may include revenue, customer increase and retention, employee satisfaction, and innovation and process improvements. Now, we’re looking at the impact made.
What metrics are you using to gauge success? Are they gameable metrics or are they truly measuring progress and impact? It’s time to re-evaluate how we measure our teams and put the customer first.