In the end, there can be only one
When it comes to the optimization of stock levels, or prices, or assortments … merchants need to look at many business performance indicators to be able to make the correct operational decisions. However, numerical optimization, much like statistical forecasting, is deeply counter-intuitive. In particular, there is a deep and subtle catch when using indicators to optimize an aspect of your business: in the end, there can be only one. Maintaining multiple indicators to drive the final decision that results from an optimization process is a recipe for picking a posteriori the metric that makes the management looks good, while damaging the business in the process. Let’s review how the whole thing unfolds.
There are many indicators that are typically found in commerce. For example, we have the total stock value (the lower, the better), the average inventory service level (the higher, the better), the total sales volume (the higher, the better), the average gross margin (the higher, the better), etc. When looking at just one indicator in isolation, everything is simple: there is an obvious “improvement” direction (e.g. the higher, the better). However, as soon as we consider multiple indicators at the same time, things get more complicated – a lot more complicated.
Indeed, all these indicators are conflicting: lowering the stock value negatively impacts the service levels, increasing the gross margin (nearly always) has a negative impact on the sales volume… Thus, the whole idea of improving one indicator at a time is bunk: this one improvement nearly always comes at the expense of a deterioration. Then, for larger companies, the problem is amplified by the corporate structure itself: the supply chain division is held accountable for any increase in stock, but it’s the contact center division that is rewarded for the improvements in customer satisfaction.
However, the problem does not stop at simply managing conflicting indicators, time is also of the essence, since market conditions are changing all the time and there is a lot of noise involved. As a result, whatever the management might be doing, there are (nearly) always some indicators that will improve from one quarter to the next. Thus, in order to avoid looking bad, it is extremely tempting to cherry pick the indicators deemed as most relevant. With the risk of sounding very technical, it’s a case of ex-post-facto rationalization: we (un)consciously tend to build some good narrative after stuff happens to explain why everything went according to plan.
Therefore, whenever a business optimization initiative is at stake, there can only be “one” indicator that consolidates all the relevant business drivers. For example, as far as inventory optimization is concerned, the pinball loss function is a first step forward towards building an indicator that properly reflects the asymmetry existing between over-forecasting and under-forecasting the future demand. While the pinball loss is far from telling everything about your inventory situation, it can already give sensible results as far as the trade-off between “stock value” vs “service levels” is concerned. Having this “master” indicator is the only way of optimizing just about anything, because as we have seen, when you get the luxury of hand-picking conflicting indicators, everything becomes blurred.
Nevertheless, it is important to clarify that while a “master” indicator is essential, there is no need to discard all the other indicators. Commerce typically tends to be complex, and in order to apprehend this complexity, it typically takes many indicators to gain all the necessary insights. However, these indicators should be used precisely for that: gaining insights, not driving operational decisions.
Coming up with one efficient master indicator is difficult. This indicator should properly balance all the different business drivers intertwined in the problem being addressed. In practice, it is frequently a composite indicator built from a combination of conflicting indicators with strategic “weighting” variables. These variables represent the best strategic understanding that management can produce about their business. Indeed, there is no “quantitative” answer to highly ambiguous questions like: do we want more growth or more margin?
A common pitfall that we frequently observe when designing master indicators is “naïve rationalism”. This refers to indicators that, while being perfectly formalized, do not capture one or more of the essential drivers of a business. As a result, improving such indicators is like accelerating while driving in the wrong direction. Naïve rationalism is dangerous because it gives a false sense of confidence to the people involved. As the saying goes, it’s better to be roughly right than precisely wrong.