Companies are full of data and metrics. Every system has a dashboard, and every dashboard has a new set of data to look at. You feel the pressure to be on top of as many metrics as possible to understand your business.

But what if you shouldn’t feel this pressure, at least not for all your metrics?

This article advocates for having a large but constrained list of metrics which you focus on. While you need many metrics to understand the full complexity of your business, the act of deciding which metrics matter, and which do not, is critical to making sense of it.

Weekly Business Reviews (WBRs) Have Lots of Metrics - But They Are Constrained

When I talk to people about running WBRs at Xmrit and Commoncog, they are often surprised that we have so many metrics: 75 as of October 2024 with more on the way. Surprise turns to nervous chuckles when Cedric shares that Amazon’s WBR had 400+ metrics to get through within an hour.

The reason we have so many metrics, even for our straightforward newsletter businesses, is that you really do need lots of them to make sense of your organisation. People often get away with a shorter list of metrics in their business reviews by only looking at one function at a time e.g. a weekly Sales meeting or Operations review. But if you try to encompass the full complexity of your business, from beginning to end, you quickly realise you need far more than is commonly used in most organisations.

Getting through such long lists of metrics is not as hard as you think, as you only spend time on the metrics showing exceptional variation. You generally move on quickly for all metrics showing only routine variation, unless the routine variation was unexpected. This approach lets you quickly review all your critical numbers, building “data sense” for your organisation’s key metrics. Even during Amazon’s Christmas peak, they could go through 400+ metrics in less than 90 minutes.

But importantly, the list of metrics is limited—you can’t look at everything in just one hour. Paradoxically, this limitation, combined with reviewing the metrics as a group, accelerates your understanding of how your business functions far more than having unlimited data would.

Becoming data-driven requires less data than you expect.

Constraints vs Data Abundance

Claiming that constraints are a good thing for being data-driven is counterintuitive. In modern business, you can pull almost infinite amounts of data from an ever-growing number of systems. In theory, this allows you to be more guided by data, but in practice it makes you more lost.

For leaders, too many metrics swamps your ability to make sense of which ones are truly important. The decision of which metrics not to monitor is as important as the ones you do monitor, as you work to build a shared causal model of the business across your leadership team. Having too many metrics also creates incentives for other leaders to push you to have an answer on irrelevant data points. Without shared agreement on which list of metrics to monitor, leaders are pressured to know the details of everything, regardless of its importance.

For individual analysts having no constraints is also unhelpful, turning every investigation into a unique data research problem, not a standard process. Constraints help guide the initial path of an analyst’s investigation, reducing the amount of time spent on pure research. An example of this was last week’s Commoncog case study, where Cedric noticed that publishing had triggered the expected spike in visitors and unsubscribers, but not subscribers, leading to the discovery of the email signup box gap in the Case Library.

By limiting your metrics to a stable set you review every week, you gain three key benefits:

1. It forces you to state which metrics matter and which aren’t important.
2. It builds a causal model amongst the reviewing group of which metrics you believe influence others, and by how much.
3. It standardises where you should start your investigations when something exceptional happens

All these benefits are hardwon. The easiest option for many business leaders is to default to data abundance, and not take the tough decision on what metrics to focus on. This is especially hard as you try to find the controllable inputs that reliably predict your desired financial outputs e.g. profit and revenue. In module 2 of the Metrics Masterclass we walk through how to start this process in your company, and what success will look like when you embed this thinking in your organisation.

Why Self-Serve Dashboards Don’t Solve the Problem

Having come this far, you may be convinced of the benefits of the WBR’s limitation on the number of metrics. But you may be wondering, don’t self-serve dashboards solve this problem? They limit what data your employees can look at, acting as a constraint on metrics. However, despite having designed several self-serve dashboards myself, I have noticed that they regularly fail to benefit from the constraints seen in the WBR.

The first reason self-serve dashboards are less successful is they forget that making sense of metrics is a social enterprise, not just an analytical one. The WBR forces a group of leaders to look at all the relationships together and come to a joint understanding of how the business works. Self-serve dashboards atomise the sense-making process, allowing people to build up different narratives of the company in their heads.

The second reason is that self-serve dashboards often have too many metrics. Every dashboard starts off with a limited list of data that can be queried, but over time there is a constant push to add more and more datasets. This push comes from every function in the business, as each function wants to understand their world more deeply. In the WBR, continually adding data is impossible, as you simply run out of time to review the metrics. Since there is no time constraint for self-serve dashboards the number of metrics continually grows, defeating the original constraint benefit.

Easier Said Than Done

In many ways the message of this article is simple, decide on what the most important metrics are in your business and focus your team’s energy on them. But like all prioritisation it is harder to do in practice than in theory. Data abundance has a tendency to divert your attention away from what is critical, and if you don’t have the constraints of the WBR it is easy to give in and end up prioritising nothing.

You need to be ruthless in deciding which metrics matter, and collectively build a shared causal model of your business amongst your employees - from leadership down to analyst.