As a manager, you must make your numbers — and manage to them.
Pick the right metrics and you can really drive performance. What gets measured, gets done. Yet picking good metrics is hard — and risky. As Seth Godin points out, managing to the wrong metric is harmful.
According to Lean experts from W. Edwards Deming to John Seddon, most of the numbers that most managers manage to, are the wrong numbers.
How do you pick the right management metrics? Follow these seven steps:
- Start with your Purpose
- Look at the Customer
- Look at Flow
- Never Blame Workers (for the System)
- Scan the Metrics Grid
- Pick a Good Ratio
- Just Get Started
Start with your Purpose
What’s the purpose of your Management Metrics — and what’s the purpose of the process you’re measuring? The vast majority of horror stories that come from mismanaging metrics, occur when managers have completely lost track of the bigger picture.
When the New York City police first started using CompStat, they used its daily metrics to allocate police resources. CompStat (along with other factors) is credited with reducting crime rates across the city by 60%. Citizens were delighted.
As the law of diminishing returns kicked in, leadership should have started looking for new metrics. Instead, they doubled down on the same countable activities – such as issuing tickets and making arrests – in areas where crime was already low, or where arresting and ticketing no longer showed any connection to improvements in perceived public safety.
Citizens became increasingly irritated by police ticketing and arresting, increasingly without good cause. The beat cop (front-line worker) was caught between management that insisted on chasing a number — setting up illegal quotas for numbers of arrests — and a citizenry (customers) for whom the old tactics no longer worked to produce valuable results. When one cop, Adrian Schoolcraft, exposed the NYPD’s illegal arrest quotas, management initially tried to have him declared insane.
Always put your purpose ahead of any particular metric, and be prepared to change metrics.
Look at the Customer
While your purpose is your own, Quality is defined by one and only one person: the customer. Your metrics should support you to increase the quality of the customer experience.
It’s extremely common for management and workers both to have strong opinions about what the customer wants and needs. Such opinions pale, like a candle held up to the noon day sun, in comparison to the opinions of paying customers.
When it’s time to select between two metrics, always pick the one that better predicts or describes Quality, as defined by the customer.
Look at Flow
The best systems are the ones where work flows smoothly. Are your systems working to provide smooth flow? Does each piece of the work flow smoothly from the moment the customer orders, until the moment the customer is satisfied? Does information flow smoothly? Are complaints, errors, and Things Gone Wrong (TGWs) all handled as precious commodities, flowing smoothly from the point of discovery to the point of systems improvement?
As you look at metrics, consider at least one that tells you whether your systems are flowing smoothly. Candidates for such a metric include the total time elapsed between customer order and customer satisfaction, and the number of TGWs collected and processed.
Creating a simple Value Stream Map is hugely useful at this stage, before you settle on specific metrics. The Value Stream Map looks from customer order to customer satisfaction and lays out the major 3 to 8 process steps. Look for areas where the system seems to snag, where work piles up, or where problems seem to routinely occur.
Never Blame Workers (for the System)
One kind of metric is particularly toxic: the one that blames the worker for the results of the system. These take many forms, and are always disguised as perfectly reasonable. These include measures such as Average Call Duration at a call center, or Mean Time to Resolution for a help desk.
You would be wise to mistrust Averages and Means generally. If you measure averages, you are hiding vast amounts of important data. If you decide that the average is relevant, you are implicitly saying that variability is not meaningful. This is usually false.
For example, at a help desk two workers were compared on their Mean Time to Resolution, with one ranking at the top, and the other coming in last place. The manager admitted that their cases were not assigned to them by the system, but rather were selected by each of them from a common pool.
The manager was preparing to have a disciplinary conversation with the last-place worker. But on what basis? This last-place worker wasn’t below some agreed standard of acceptable behavior — there was no standard. He was merely in last place on a specific metric.
If you insist on ranking people against each other (instead of against a standard), you’ll always have someone in last place. Why is that bad or wrong? It’s mathematically inevitable. You could rank the top five sprinters in the world and one of them would come in last place compared to the others — and still be a world class performer.
Worse, there’s no evidence that the last-place worker isn’t attracted to very tough problems. He may even be excellent at them. The first-place worker might be a slacker who cherry-picks only the most trivially easy cases. Maybe he should receive the disciplinary conversation. With existing data, we can’t tell who’s doing well overall, or who’s creating more value for customers, or who’s helping or hurting the team’s performance.
Never manage people (including yourself) to a standardless metric, especially not an average, that’s divorced from both context and customer value.
Scan the Metrics Grid
There are three sorts of metrics that relate to your Value Stream — your creation of value for a customer. They are:
- Activities and Outputs
These have been treated at length elsewhere. (Some treat Activities as separate from Outputs; I find it more convenient to combine them for purposes of metrics.) Here are some simple, useful definitions for use in devising metrics:
Inputs are things you allocate — money, staff-hours, conference rooms, assets. When you allocate an input to one thing, it becomes unavailable for others. Allocating the input is a planning and budgeting activity. The police department budgets dollars and staff hours; so do the auto factory and the restaurant.
Outputs are the direct results of the inputs as they get used up. When staff hours are expended, you get meetings, reports, research, widgets, phone calls made, clients visited, and so on. A police department allocates officer labor hours to patrol, with the output of “patrol hours” — if a given officer instead is allocated to a community meeting, she cannot also be on patrol. Outputs are good to measure — police departments need to know how many patrol hours actually occur, and where; auto factories need to know how many car doors got assembled and how many quarter panels painted; the restaurant needs to know how many dishes were ordered.
Outcomes are valuable results — things that customers value. These are crimes prevented, high quality cars built, food eaten with enjoyment, etc.
We can now start to build the Metrics Grid with these three columns, showing the time flow from left to right — Inputs, Outputs, Outcomes.
Next, consider the three flavors of metric — quality, quantity, and time. Make these rows on the grid:
|Flavor of Metric||Inputs||Outputs||Outcomes|
Brainstorm some metrics you can put in each cell. Don’t limit yourself to one, and don’t edit — just come up with as many ideas as you can. You’ll select some in the next step.
Here’s a brainstorm of potential measures for the Metrics Grid for the role of busboy in a restaurant:
|Flavor of Metric||Inputs||Outputs||Outcomes|
|Quantity||steps taken (pedometer)||number of tables cleaned||number of tables available to seat new customers|
|Quality||smiling||how clean each table is||how clean the whole restaurant appears|
|Time||hours on duty||time to clean a table, time to restock clean dishes||time delay before customer gets to sit at a clean table, time delay for cook needing a clean dish|
Pick a Good Ratio
Some of the best guidance I’ve found on picking good metrics comes from Coonradt in The Game of Work: How to Enjoy Work as Much as Play. He suggests creating a ratio of either Outputs or Outcomes (Results), per unit of Input (or Resources) — he calls it the Results to Resources Ratio, the RRR.
Examples of RRRs: Miles per Gallon; Views and Shares per Post; Correctly Closed Tickets per Staff Hour.
You’ll still need to pick specific metrics to put into your ratio.
The ratio itself brings lovely advantages:
- Insulation from outside variables (like inflation)
- Single number that captures efficiency
- Traces back to known inputs
- Workers can usually keep score themselves
People love to play games, and the best games use clear scoring that the players can keep themselves. A good RRR will guide the worker toward focus on the right outcomes, to focus on economizing the most precious inputs, and will leave lots of open playing space for workers to innovate new strategies to increase their scores.
For our busboy above, we might try three: “Smiles per Hour,” Number of Open Tables at Rush Times, and Number of Tables Cleaned per Person-Hour.
Of these, which involve flow? Number of Open Tables at Rush Times. If the busboy can’t keep tables open for new patrons to sit, he becomes a bottleneck to the entire restaurant.
Just Get Started
My friend the legendary business consultant Jim Grew gave me the best advice on setting up work measures: just get started.
Jim said, you’ll never get the metrics perfect up front, so don’t try. Instead, focus on the discipline of keeping some sort of score, and let yourself learn and adjust the measures for a while.
- What does my initial set of metrics not track?
- If I were to focus on these numbers, what would I be neglecting?
- If I were to focus on these numbers, what would I be overemphasizing?
Then once a week (for daily metrics), or once a month (for weekly or monthly metrics), give yourself permission to tweak the numbers. For your self-scoring, that’s enough. For a scoring system for people reporting to you, include some of them in the revision.
Your top performers will insist on good metrics. If the busboys complain that they’re being pushed to move too fast to clean tables thoroughly, and they feel they aren’t creating a quality dining experience, it hurts their pride of workmanship. Figure out a way to simply and easily measure quality. (For example, ask the hostess to judge cleanliness, and monitor customer feedback. Go out and look yourself as well. Create a five-point scorecard for cleanliness.)
Once you get to a stable set of good-enough metrics, stop tweaking. Nothing undermines the fun of a good game more than changing the rules in the middle. Let your people know when you’ve stopped adjusting the metrics. Give everyone a chance to innovate under a steady scoring system.
When you find people tracking their own scores, taking pleasure from it, and delivering superb customer value in the process, you know you’ve got good metrics.
One of the greatest payoffs of a system of good metrics is, it creates a sense of emotional safety. When each worker knows clearly the rules, the boundaries, and the scoring, they’re free to innovate. When they’re confused and unclear, they hunker down and try to avoid mistakes.
Create clarity through good measures, and set your people free.
For more on what’s a metric vs. a measure vs. a number vs. a KPI (Key Performance Indicator), read this article.