Oddities in classic forecasts - Inventory Optimization Software

Oddities in classic forecasts












Home » Resources » Here

Update May 2016: This page focuses on classic forecasts, which typically applies when Lokad is used as a demand forecasting tool, and not as an inventory optimization tool. Probabilistic forecasts are substantially different and more powerful as well, and should be used whenever the intent is to perform inventory optimization.

Statistics is a counter-intuitive science. Thus, the forecasting engine of Lokad, as a statistical webapp, tends to deliver puzzling results. In this article, we try to shed some light on the classic forecasts, that is, daily, weekly or monthly forecasts.

Getting started

At this point, we assume that you have managed to get started and you have already generated your first Microsoft Excel report with Lokad. If you are wondering about the interpretation to be given to certain columns, you can refer to the documentation. We recommend to double check that the input data is correct, typically double-checking the historical values reported in Lokad with the values you get from your existing company systems.

Backtesting and model selection

Before discussing the actual forecasts, let’s review briefly how those forecasts are produced by Lokad. Our forecasting engine includes a large library of forecasting models, ranging from naïve to very complex models. For each time-series, Lokad performs a backtesting, that is, we go back in the past, we truncate the data, we generate the forecasts using only this truncated data, and we measure the accuracy of the result. Then, this process is repeated for each day, week or month - depending on the period applicable for the given historical data.

At the end, for each time-series and for each model, the forecasting engine had collected a set of accuracy measurements that let it pick the most accurate forecasting model. This selection mechanism is strictly performance-driven. The logic does not say, for example, that a seasonal model must be applied for Product X; seasonal models just happen to be present within the forecasting engine, and when those models outperform the other models, they are selected.

Flat forecasts and erratic history

Most of time the historical data tend to be relatively erratic, especially when looking at very disaggregated levels such as SKUs or products. The blue line in the graph below illustrates a sample time-series. The red line represents a tentative forecast of this time-series.

Image


This forecast visually looks good because it reproduces the sort of fluctuations that have been observed in the past. However, from a statistical viewpoint, this forecast is widely inaccurate. Indeed, the time-series is actually made up of random values drawn between 0 and 1. There is no pattern to be learned, it’s just noise.

Image


Here above, we have revised the forecast: it’s now a flat line at 0.5. Visually, this forecast seems at odds with the history: it presents a regularity that has never been observed in the data. Yet, from statistical viewpoint, this forecast is much more accurate than the previous one.

This idea is actually a general (but no so intuitive) aspects of statistics: the more erratic the demand history, the smoother the forecasts. Any forecasting system that would behave differently would actually be much less accurate.

Simple but efficient forecasting models

While the forecasting engine contains very complex models, very simple models still tend to be applied sometimes on certain datasets (when they happen to outperform the others during the selection process).

Among many other examples, we have models such as:
  • Flat last period (weekly or monthly forecasts): The forecasts uniformly repeat the value observed for the last week or month.
  • One year seasonal (weekly or monthly forecasts): The forecasts repeat the values observed exactly one year before (respectively 12 months or 52 weeks).
  • One week cyclical (daily forecasts): The forecasts repeat the values observed exactly the one week before, matching the day-of-the-week pattern.
  • Flat yearly average (weekly or monthly forecasts): The forecasts uniformly repeat the historical demand as averaged over the last year.

There is no problem per se in using any of those models to produce a forecast. The challenge is to know exactly when a more complex model should be favored over simpler ones, and that's why Lokad backtesting is so important.

Nevertheless, from a customer point of view, flat forecasts - especially 12 months ahead forecasts - are frequently considered as disappointing, for instance when the seasonal pattern expected by business experts is not found in the values returned by Lokad. However, our forecasting engine has many seasonal models available and those models are benchmarked every time. Thus, when a non-seasonal model is selected, it means that this model did quantitatively outperform all the seasonal ones. Trying to force a seasonal pattern in such situation would only degrade the accuracy - again, we must admit, that this might not be very intuitive.

The accuracy column

The accuracy column is optional and not activated by default. It represents the anticipated forecast error rate of the classic forecasts. You can think of this feature as of a self-diagnosis of the system. Without getting into technical details of actual definition of this accuracy indicator, let's say it's a percentage: 100% being a totally accurate forecast and 0% being a totally inaccurate forecast.

100% accuracy is not a realistic expectation for a forecasting process (statistical or otherwise). In particular, when looking at low sales volume, a small error of +1 or -1 can already reduce the accuracy expressed as a percentage to a value lower than 50%. Contrary to intuition, the overall level of accuracy is not a consequence of the forecasting method, but of the level of aggregation of the data itself.

For example, if we are forecasting daily nationwide electricity consumption from one day to the next, a forecast with a 99.5% accuracy might be considered as rather poor, while forecasting the promotional sales of a fresh food product, a 30% accuracy might be considered as a significant achievement. Yet, this does not mean that a better forecasting solution cannot improve the situation.

When it comes to forecasting, there is no such thing as good or bad forecast in absolute terms; the only question that matters is how good are those forecasts compared to the status quo or alternatives?.