Output metrics and controllable input metrics are a simple idea, in theory.

Output metrics measure the outcomes you want to achieve. In business these outcomes are often financial targets like revenue or costs, but can be intermediate metrics like customer satisfaction or number of deals closed.

Controllable Input Metrics are the actions that act as inputs to your desired output metrics. When these input actions occur they result in improved output metrics. Importantly these input metrics are under your control, so you can consciously do more of them to improve your output metrics.

However when you look at websites for advice on what input metrics to set you find something strange. Despite saying that input metrics should be under your control you quickly find many of the suggested metrics are only partially controllable. One of the top results for input metrics on Google provides a definition and list of example input metrics for an e-commerce company.

input metrics are the things that you can directly control.

…

E-commerce checkout flow

Input metrics →

• Percentage of customers who add items to their cart
• The average number of items added to the cart per customer
• Conversion rate (percentage of customers who complete their purchase)

Wait a second… none of these examples are fully controllable!

For the first two cart suggestions you can influence customers to add items to their cart, but it is the customer who ultimately decides to add or not. You run into the same issue for the conversion rate metric, as it is the customer who ultimately decides whether to purchase or not.

There’s a common assumption that input metrics should be fully controllable, but this oversimplifies the complex reality of businesses. Interestingly, partially controllable input metrics often prove more effective at driving desired outcomes. The reason for this is 100% controllable input metrics tend to create problematic incentives that can actually harm business performance.

Using a case study from Amazon e-commerce, and a recent example from the Commoncog forum, we will see how choosing partially controllable input metrics yields better results than a 100% controllable metric.

In both cases the solution is to pick a metric which is closely related to the 100% controllable input metric but is only partially controllable - I call this “going up a level”. Sometimes going up a level starts to involve customers in the input metric, which further reduces the incentive problem. As you will see the challenge is making sure your partially controllable metric is still somewhat controllable while continuing to influence your desired output metric.

Case Study: Amazon E-Commerce’s Challenge in Finding the Right Input Metric

As Amazon expanded outside of books and into a wider range of items they wanted to help their customers select the best products. The leadership team believed that if they gave better product information to their users they would purchase more products, driving up the eventual revenue output metric.

Amazon initially started off by asking all their teams to increase the number of products with detail pages, which include information on the product, pictures, reviews, shipping information, and a purchase button. The goal of increasing detail pages, a 100% controllable input metric, was added to Amazon’s weekly business review (WBR). However, it did not have the effect that the leadership team expected, as seen in this quote from “Working Backwards”:

Once we identified this metric, it had an immediate effect on the actions of the retail teams. They became excessively focused on adding new detail pages—each team added tens, hundreds, even thousands of items to their categories that had not previously been available on Amazon. For some items, the teams had to establish relationships with new manufacturers and would often buy inventory that had to be housed in the fulfillment centers.

We soon saw that an increase in the number of detail pages, while seeming to improve selection, did not produce a rise in sales, the output metric. Analysis showed that the teams, while chasing an increase in the number of items, had sometimes purchased products that were not in high demand. This activity did cause a bump in a different output metric—the cost of holding inventory—and the low-demand items took up valuable space in fulfillment centers that should have been reserved for items that were in high demand

Amazon’s leadership team quickly realised their mistake and began to “go up a level” repeatedly with their input metric, moving away from the 100% controllable to partially controllable, to address the incentive issue.

1. Number of detail pages became →
2. Number of detail page views (credit only for detail pages seen by a customer) became →
3. % of detail page views where the product was in stock (credit only if can be stocked as well) became →
4. % of detail page views where the product was in stock and ready for 2 day shipping final metric ★

To mitigate the incentive issue of producing detail pages for products that were not wanted, the metric evolved to be the number of customer detail page views. In addition to being “one level up” it also includes the customer, which further reduces the incentive gap as it ties your business actions to the actual customer needs. You can only fool a customer for so long before they stop visiting your site!

As the metric evolved Amazon bought in stocking and shipping elements, going further levels up and involving more parts of Amazon and the customer experience. This is important because if you advertise the best product but can’t ship it to the customer is Amazon any better than getting it in the store?

The challenge that Amazon had was simultaneously creating controllable input metrics that were:

1. Mostly controllable by the teams responsible for the metric
2. Did not result in incentive problems, as seen with the # detail pages metric
3. Continued to be strongly linked to the targeted output metric of sales.

Meeting these three criteria is challenging, but Amazon managed it by creating input metrics that were only partially controllable - and in this instance by bringing in the customer’s decision to the metric.

Now let’s have a look at another example where partially controllable input metrics resulted in better incentives, this time using Commoncog’s forum Commonplace as an example.

Case Study: Commoncog’s Forum

Over October 2024 Commoncog decided to put more focus on driving engagement with its business discussion forum Commonplace. Our key output metrics were Daily Engaged Users and Pageviews. When Cedric wrote the PR/FAQ the key 100% controllable input he had was the number of topics posted to the forum. Within reason he has the ability to make as many posts as he wants to the forum, driving this controllable input metric up.

But immediately you notice there are some potential downsides to focusing on the number of topics as a metric. You also care about the quality, and actual user engagement with the topics that you make. If you ignore those factors you could end up hitting your input metric, but missing out on the output metrics of Daily Engaged Users and Pageviews.

The solution to this problem was to “go up a level” and create a new controllable input metric, one that reduced the control that Commoncog had and increased the influence of the customer (forum members). In the end Commoncog came up with the metric Active Hook Topics:

Active Hook Topics - The number of unique topics in a week where >= 2 different non-staff members have commented.

By reducing the control that staff members have over the input metric, and involving the decisions of forum members in the metric, we removed the incentive problem of spamming poor quality posts to reach a poorly designed input metric. As you can see below there is still a very strong relationship between the number of posts, active hook topics and Daily Engaged Users, something made possible by aligning incentives through carefully designing our input metric.

The Power of Partial Control

Both Amazon and Commoncog’s experiences reveal a counterintuitive truth about input metrics: partial control often leads to better outcomes than full control. In each case, the initial instinct was to choose a fully controllable metric—number of detail pages for Amazon, and number of topic posts for Commoncog. But both organisations discovered that moving “one level up” to partially controllable metrics produced superior results.

The main benefit of moving up a level is it reduces the chance of you over focusing on a very narrow input metric. Very narrow focuses almost naturally lead to incentive problems, as you ignore the second order impacts of your actions. In both cases going up a level meant including some element of customer decisions in the metric, which further reduces the possibility of bad incentives by ensuring your actions are tied to customer value.

As you design your own input metrics, consider whether you might be too close to the action. Could moving “one level up” and incorporating customer behaviour create better alignment with your ultimate goals? The key is to ensure your metrics are still mostly controllable, avoid bad incentives, while still retaining their link to your desired output metrics.