00:00:00 Introduction to the interview
00:01:46 Forecast accuracy’s impact on profit
00:03:25 Defining accuracy in forecasting
00:07:36 Assessing with a quantitative instrument
00:09:35 Time series as one measurement
00:11:16 Demand expressed at basket level
00:13:22 Analogy of time series forecast
00:15:17 Limitations of time series in perishable food context
00:18:51 Time series not reflecting business core
00:21:41 Mention of forecast value added
00:24:47 Discussing forecast accuracy as a KPI
00:27:55 Fashion industry’s focus on generating wants
00:30:35 Transition to discussing aerospace industry
00:33:05 Lokad’s probabilistic approach
00:36:22 Example of selling backpacks and future decisions
00:39:34 Supply precedes demand, example of iPhone
00:42:37 Difference between Lokad and other companies
00:46:13 Lokad’s approach to problem solving
00:49:11 Disagreement with common supply chain perspectives
00:51:49 Transition to audience questions
00:54:06 Planning doesn’t reflect business understanding
00:57:35 Textbook definition of accuracy is irrelevant
01:00:01 Question on forecast accuracy and execution
01:03:38 Question on interdepartmental communication and silos
01:06:15 Example of B2B distributor of electrical equipment
01:09:34 Basket analysis more prevalent in B2B
01:12:30 Measuring forecast accuracy
01:15:02 Questioning the title of the video
01:17:10 Analogy of removing a cancer
01:20:32 Managers’ secret spreadsheets
01:23:03 Joannes’ response to question
01:25:00 Question about time series forecast in forecasting structure
01:27:05 Question about universities teaching traditional forecasting methods
01:30:54 Snop for corporate alignment
01:33:23 Experiment during lockdowns with planning department absent
01:36:12 Discussing company transformation
01:39:10 Grand Illusion in supply chain
01:42:13 Accuracy exemplifying exactly wrong mindset
01:42:49 End of interview
Summary
Joannes Vermorel, CEO of Lokad, criticizes the traditional understanding of forecast accuracy in supply chain management, arguing that it doesn’t reflect the core DNA of a business. He suggests that time series forecasts, which are commonly used, are overly simplistic and do not accurately represent the future for supply chain purposes. Vermorel proposes a different approach, focusing on being quantitatively faithful to the essence of a company. He criticizes the focus on incremental improvements and suggests that companies should look for simpler, better solutions. Vermorel emphasizes the importance of understanding the essence of the problem and producing quantifiable statements that make sense for the business.
Extended Summary
In a conversation between Joannes Vermorel, CEO of Lokad, and Conor Doherthy, the topic of forecast accuracy and its role in demand planning was explored. Vermorel, a French software entrepreneur, challenged the traditional understanding of forecast accuracy in supply chain management, a concept that has been deeply rooted in the industry since the 1920s. He argued that while forecast accuracy is directly linked to profit in stock market speculation, this model does not apply to supply chain management as there is no direct translation between forecast accuracy and profitability.
Vermorel proposed two ways to define accuracy: the mainstream way and the Lokad way. The mainstream definition, he explained, is a time series forecast, a periodic forecast with equal intervals. However, Vermorel criticized this approach for making significant assumptions, such as symmetry between the past and the future, locality of measurements, and agnosticism to the computational or software environment. He argued that time series are an overly simplistic model that does not faithfully represent the future for supply chain purposes.
Using the example of a supermarket, Vermorel illustrated that time series forecasts ignore important relationships between products. He argued that time series forecasts are blind to important dimensions and do not reflect the structure of the future. He suggested that time series forecasts may be sufficient for small businesses, but not for large companies operating complex supply chains.
Vermorel also criticized the supply chain textbook’s focus on time series for accuracy, arguing that it doesn’t reflect the core DNA of a business. He emphasized that supermarkets are structured to sell baskets of items, not individual products. He questioned the logic of using predictive tools that look at products in isolation when supermarkets are engineered for customers to buy many items at once.
Vermorel also discussed the complexity of demand, using perishability as an example. He explained that if half of a store’s stock expires the next day, it doesn’t truly have 50 units in stock. He also mentioned that customers may choose products with the longest shelf life, which can affect the urgency of selling certain products.
Vermorel argued that time series cannot accurately reflect important patterns like baskets and perishable items in a supermarket. He believes that the accuracy of time series only reflects its own paradigm, which is why Lokad diverges from this approach.
Vermorel also criticized mathematical solutions that are technically correct but impractical in the real world. He acknowledged that critics might argue that tools based on time series work in practice, despite his criticisms. Vermorel noted that vendors have been claiming their tools work for the past 45 years, promising to automate everything supply chain-related. He argued that despite these claims, everything is still done through spreadsheets.
Vermorel believes the core issue is that the time series perspective is incorrect and doesn’t fit the structure of the problem. He criticized the one-dimensional view of the future offered by time series. When asked what should be pursued instead of forecast accuracy as a KPI, Vermorel suggested that the goal should be to produce quantitative statements about the future that make sense for the company.
Vermorel concluded that time series is almost invariably wrong for most businesses. He compared trying to fit a mathematical model with the wrong structure to trying to fit a round shape in a square hole. He suggested that there are many other ways to approach the problem, depending on the business.
Vermorel gave examples of different business models, such as supermarkets and fashion companies, and how time series doesn’t make sense for them. He argued that to think about the future, you need to consider the “halos of wants,” which don’t match the time series vision.
Vermorel also discussed the aerospace industry, where the consumption of parts is driven by the lifecycle of aircraft. He concluded that using time series is a crude approximation for any business vertical. He compared using time series to approximating a cow as a sphere, arguing that it’s a poor approximation for real-world situations.
Vermorel also discussed the issues he sees with the traditional approach to supply chain management, which assumes that the past is an exact mirror of the future. He argued that this is not the case, especially in supply chain where future decisions have not yet been made and are influenced by various factors, including competitors’ decisions.
Vermorel used the example of selling backpacks to illustrate his point. He explained that the number of variants a company introduces can significantly impact future demand. He argued that the traditional approach of deciding on the assortment first and then forecasting is nonsensical, as the demand is not set in stone and is influenced by the company’s decisions.
Vermorel further explained that companies engineer their demand by introducing products to the market, which then generates demand for those products. Doherthy brought up the practice of forecast value added, where insights from different departments are used to make revisions to the forecast. Vermorel criticized this practice, arguing that it is often just a way to back up gut feelings with numbers and does not contribute to the actual decision-making process.
Vermorel explained that Lokad uses numerical recipes that are more versatile and not limited to time series models. He discussed the importance of making a statement that is faithful to the future of the business and aligns with what the company is trying to achieve. Vermorel emphasized the importance of understanding the essence of the business and engineering a model on top of it.
Vermorel criticized the perspective expressed in most supply chain books that dismisses the specificities of different verticals. Doherthy asked Vermorel how he responds to large, successful companies that disagree with his views. Vermorel argued that companies do not have opinions, only the people working for them do. He believes that many executives in large companies would agree with his views, as they often feel frustrated with the traditional approach to planning.
Vermorel argued that the traditional supply chain textbook definition of forecast accuracy is flawed because it’s based on a time series forecast paradigm, which he believes is incorrect. He suggested that Lokad’s approach, which focuses on being quantitatively faithful to the essence of a company, is more valuable.
Vermorel agreed with a viewer’s point about embracing uncertainty through probabilistic forecasts, but he also emphasized the need to move beyond one-dimensional thinking and to consider future decisions that have not yet been made.
Vermorel explained that a forecast is just an ingredient and doesn’t have value in itself. He agrees with the idea that the value of a forecast can only be assessed through its execution in the supply chain. He also warned against sharing too many KPIs across teams, arguing that it doesn’t necessarily create value for the company.
Vermorel explained that sharing data should not involve manual processing by humans. Instead, everyone should have programmatically access to all the data in the company to optimize their own decisions. He warned against creating bureaucracy by forcing other departments to read reports.
Vermorel argued that the basket concept is essential for B2B businesses, using the example of a B2B distributor of electrical equipment. He explained that the bulk of their business is driven by construction sites, which require large orders of equipment to be delivered at specific times. This, he says, is a form of basket analysis.
Vermorel argued that the alternative to time series forecasting doesn’t have to be a complex AI. He suggested that there are many other mathematical models that are not more complicated than time series, they’re just different.
Vermorel explained that Lokad uses a financial perspective to reconcile the many conflicting goals of a large company’s supply chain. He suggested that expressing all goals and constraints in dollars provides a unified language to manage these conflicts. He emphasized that this is not about thinking in terms of dollars, but about practicality and scalability in complex companies.
Vermorel asserts that accuracy and time series are the same thing in the mainstream supply chain paradigm. He suggests that Lokad wants to separate them, and while there is a way to make accuracy matter, it is radically different from what is presented in supply chain textbooks.
Vermorel criticizes FVA as over-engineering a process based on a flawed concept of time series accuracy. He argues that it moves the company in the wrong direction, adding unnecessary bureaucracy without making the supply chain more competitive.
Vermorel describes how large companies often rely on unofficial spreadsheets rather than official SNOP forecasts. He suggests that these spreadsheets, which are more aligned with the essence of the business, are what actually drive the business.
Vermorel argues that an improvement compared to the status quo is not necessarily an overall improvement. He criticizes the focus on incremental improvements and suggests that companies should look for simpler, better solutions.
Vermorel agrees that time series can be a component of the structure, but warns against relying solely on time series. He suggests that companies need to expand their vocabulary and horizon.
Vermorel compares classical time series and machine learning to black and white television and LCD screens respectively, stating that while machine learning has its advantages, it’s still not a quantum leap from classical methods.
He criticizes universities for not teaching the right forecasting attitude, emphasizing the importance of understanding the essence of the problem and producing quantifiable statements that make sense for the business.
Vermorel shares that Lokad ranked fifth in the Walmart competition using a simplistic parametric model, demonstrating that complex models aren’t always necessary for success.
He argues that there is a continuum from classical models to advanced machine learning models, and the distinction between them isn’t as clear-cut as some might think.
Vermorel reiterates his criticism of universities for not teaching the right forecasting attitude, emphasizing the importance of the right mindset when dealing with future supply chain issues.
He explains that the goal of the S&OP process is to create company-wide alignment, but in practice, it often devolves into endless meetings.
Vermorel argues that information flows through IT systems and that alignment doesn’t require constant communication between people.
He suggests that S&OP meetings should focus on numerical recipes and clarifying the strategic intent of the company.
Vermorel argues that many large companies could function just fine without their time series forecasts.
He shares an example of companies operating at 80% capacity during the 2020 and 2021 lockdowns, despite their planning departments being inactive.
Vermorel suggests that if a company can operate without a division for 14 months, that division may not be mission critical.
He shares an example of a company that underwent a massive transformation during the lockdowns, shifting from 5% e-commerce to two-thirds e-commerce.
Vermorel challenges the importance of certain functions within a company, given that some companies were able to undergo massive transformations and still operate effectively.
He argues that accuracy isn’t the only important factor in forecasting, citing the example of companies that operated normally despite their planning departments being inactive for over a year.
Vermorel criticizes the mainstream time series accuracy paradigm for not asking important questions about the instrumentality of forecasts.
He emphasizes the importance of connecting the dots from decision to mathematical model and assessing the real-world financial impacts of those decisions.
Vermorel criticizes the common practice of assessing the accuracy of a forecast in isolation, arguing that it’s not reflective of real-world conditions.
He concludes that the problem with accuracy is that it’s often framed incorrectly, and that an approximately correct gut feeling is better than a sophisticated but mismatched business model.
The interview ends with Conor Doherthy thanking Vermorel for his time and promising to save his remaining questions for another day.
Full Transcript
Conor Doherthy: Welcome back to Lokad TV live. Joining me in studio today is Lokad founder, Joannes Vermorel. Today, we’re discussing a very interesting topic: forecast accuracy, its role in demand planning, and whether or not it even matters. Feel free to submit your questions at any time during this chat, and we will get to them in the second half of the conversation. If you disagree with anything you hear, we’ll answer those questions first. So let’s get started. Joannes, we are, I think it’s safe to say, somewhat of a contrarian company. This might be the most contrarian take we have. So before we get into our position and why we think demand forecasting accuracy is unimportant, why is forecast accuracy viewed by so many companies as the Holy Grail of demand planning?
Joannes Vermorel: The ‘why’ is, I believe, relatively straightforward. It is what is written in supply chain textbooks. It has been written for the last maybe 50-70 years, maybe even before it was called supply chain, it was called operational research. I suspect that we could even go further than that, even back to the 1920s, and we would find the sort of premise of that with the emergence of the professional economic forecasters. If you take this idea of forecasting accuracy back to its root, which are the economic forecasters in the USA at the beginning of the 20th Century, then accuracy has a one-to-one translation to your profit if you play the stock exchange game. So literally, if you’re forecasting the price of commodities, will the price of pig iron raise or not, then if you have an accurate forecast, you can potentially beat the market and do fantastic returns. Now, this is true for speculation. The problem is, do you have a forecasting model that can beat the market? The short answer is no, at least not an easily accessible model. So you cannot really beat the market anymore. There are some caveats to that, some arbitrage companies are making money on doing that, but that’s just one point. Supply chain-wise, my point is that there is no direct translation. But my criticism is not exactly aligned to that. The problems are deeper and more fundamental because it’s not about just getting a number and if you have the number right, it will automatically translate into you making money, just like it is when you’re playing the stock exchange.
Conor Doherthy: So, are you saying that there’s no correlation between increased forecast accuracy and bottom-line profitability?
Joannes Vermorel: The problem here is that there is a deception going on with the terms themselves. So I will maybe start by clarifying how we define accuracy. There are at least two ways: one mainstream way of defining accuracy and one that would be the Lokad way of defining accuracy. Let me start with the Lokad way, which is not mainstream, on how we approach accuracy. The whole idea of accuracy is that I am making a quantitative statement about the future. The accuracy is a qualification in terms of quality, how good it is, how faithful it is, about this statement. So you have a statement about the future, the future should look like this, and this is not a qualitative statement but a quantitative statement. And then on top of this quantitative statement about the future, you want to qualify to say how good it is, how faithful it is, how does it really depict the future, and you want this assessment to be quantitative in itself. And that’s what the accuracy should be. If we define accuracy the way I do, I would say fine, I agree. This is very relevant, this definition leads to something that is meaningful and significant and potentially profitable for your company. Now, this is absolutely not the definition you will find in supply chain textbooks, not even close. The mainstream definition of accuracy is a time series forecast. So, when people say accuracy, they are implicitly speaking of time series forecast and not any kind of time series forecast, it’s going to be a point, equispaced time series forecast. What does equispaced mean? It’s a periodic forecast per day, per week, per month, per quarter, potentially per year, potentially per hour, equal intervals. So that’s a periodic forecast. So it’s not any kind of forecast, you can imagine many other alternative forecasts, it’s a time series, one-dimensional, and periodic, all periods are the same. On top of that, we are talking of a point forecast where each period gets one value, that’s the time series forecast. My definition is very different. The one I gave was much broader and nonspecific about what sort of quantitative statement I’m making about the future. I’m saying that my definition is completely agnostic, it’s just saying we are trying to qualify a statement about the future and I’m just saying that this statement should be quantitative. So I’m not challenging something like “I believe this will be a good year”, no, that’s not a quantitative statement about the future, just something that is qualitative. So I’m just saying that accuracy applies to quantitative statements about the future and that we want to assess it again with a quantitative instrument. The mainstream approach is much more direct and it makes very significant assumptions. The assumptions are time series, one-dimensional, periodic or equispaced, and point forecast. So that’s pretty much the core assumptions. There are some more fundamental assumptions that people might not even see, like symmetry between the past and the future, locality of measurements, and agnosticism to the computational or software environment.
Conor Doherty: Thank you. And again, to push back a little bit, when you’ve described the difference between Lokad and the mainstream approach, but for some people, it might not be clear. What is the problem with the time series perspective? You say it’s one-dimensional, okay, in what respect and why is that an issue?
Joannes Vermorel: When you decide that you want to describe the future with quantitative statements, we are so used to, and in the textbooks, it’s literally supply chain textbooks but other business textbooks also, don’t even acknowledge that there might be another way to look at the future than time series. People, I mean, and that I think the biggest mistake is that it gives the impression as if the only way to look at the future quantitatively was time series. And I say, certainly not. And more than that, time series are an incredibly simplistic model. It’s like one measurement with one tick over time at every period. It’s the simplest of the simplest of the mathematical model that we have. Is it a faithful representation of the future? Does it reflect in a sensible way something that you know about the future? And my proposition is that for supply chain purposes, no, and not even close. And if we start with some examples, let’s have a look at, for example, the demand that a supermarket will observe. The time series perspective says you can take any product that is being sold in the supermarket and have a time series, one per product sold by the supermarket. Is it the correct way to think about the future demand? No, why? Because people do not walk into a supermarket to buy one product in isolation. What they want is a basket, or at least the vast majority, occasionally you might have some people that walk into the supermarket to buy one product, but the vast majority of the sales are driven by people who go to the supermarket like once a week and they buy an entire basket of products. So what matters in terms of demand is expressed at the basket level. This is what people will observe, what they will feel, and if they think that there is, and if we have to think in terms of quality of service, it will be perceived in terms of baskets. So it’s, do I have all the things that I needed for my shopping list? So, and this perception which is at the basket level has nothing to do with those time series in isolation and those time series in isolation, they just completely ignore all the relationships that there are between the products and all the substitutes and cannibalization that may happen. So it’s sometimes blind to those effects. So we have a problem of blindness, is that this one dimension that is at the core of the time series, it ignores the higher dimensions that can be extremely important. They just completely ignore all the relationships that exist between the products and all the substitutes and cannibalization that may happen. So, it’s sometimes blind to those effects. So we have a problem of blindness.
This one dimension, that is at the core of the time series, ignores the higher dimensions that can be extremely important. And my proposition is that it’s not an accident. Take the example of a supermarket, or any company, and actually think about what the future means, what we are actually looking at. You will realize that at the core, we are not looking at time series. We are looking at things that have structure, but not necessarily, and usually not, time series structure. There might be some incredibly simplistic businesses, mom and pop shops, where the time series is sufficient, but those businesses are the exception, they’re not the norm, especially in the world we live today with large companies operating large supply chains with a lot of ambient complexity. Each time series would treat any given item or any given SKU independent of every other given SKU in a catalog. Thus, if there are interrelations, if there are bundles or substitutions, it will be agnostic or blind to that. And that ultimately makes the accuracy for any individual item not wrong, but misleading. If I want to do an analogy, imagine you have a television and it’s black and white. That would be your time series forecast. You’re lacking something. You can add pixels, that would be adding accuracy. But you still have only black and white. And if you think, “Oh, but if I add a lot of pixels…” Yes, but you’re still black and white. It doesn’t matter how you can make the television bigger, you can increase the refresh rate, you still do not have colors, you only have black and white.
So, I’m taking this as an example where it doesn’t matter what you do, if you’re missing dimensions, you cannot salvage the case. And there are so many dimensions that are missing. Let’s revisit this supermarket example with perishable food. Perishable food, let’s say you have products on the shelves, but every unit that you have on the shelf comes with its own shelf life. And what many shoppers do when they visit the store is that they look at the expiration date and their opinion about the product differs depending on whether there is only one day left of life for this product, or whether there are three weeks left. It’s still very fresh. But if we look at the data from a time series perspective, this is absent. You cannot, through a time series representation of your sales or demand for, let’s say, a pack of yogurts, represent the freshness. It’s absent. That would be, if I go back to my television equivalent, to say, “Well, I only have black and white, but do you know what? I can just buy three TVs and I would say the first one is going to display the blue, the second one is going to display the green, and the third one is going to display the red. And technically, I have all the colors. I just need to somehow visually recombine that.” I would say, “Yeah, but that’s a very, very convoluted solution to the problem. It’s not a good solution.” From a practical user experience, it’s complete crap.
And that would be the same if we were to do for the supermarket to say we are going to just deal with perishability by just adding more time series. Yes, in a very technical sense, you could potentially do that, but it’s just going to be a very impractical solution. It’s not going to be a good solution. And you see, here again, perishability, the problem is that the demand is not a one-dimensional thing. You have another dimension which is freshness, and it matters. It impacts the demand and it also impacts your stock. If you think that you have 50 units in stock but half of them expire tomorrow, then you do not have truly 50 units in stock. And that’s only if the client doesn’t have an adverse behavior of picking the units that have the longest shelf life.
The clients who pick products on a shelf in a supermarket might even pick the products that have the longest shelf life and thus they might actually adversely select the products that are the least urgent to sell. So, back to the initial case, the time series cannot, and we have just mentioned one example, supermarket, and we already have two examples of super important patterns like baskets and perishable items. They are very important, they are very core, and they do not fit the time series paradigm. And then the accuracy, the textbook accuracy, only reflects the time series paradigm. It only matches implicitly and that’s why I say Lokad diverges. The supply chain textbook, when it comes to accuracy, it’s only about time series. And my point is that yes, you have an instrument that measures something that is inconsequential and it doesn’t match the core DNA of the business, what makes the business tick, those baskets, perishable things. It sounds like you’re talking about constraints. There are a lot of constraints, a lot of other not even constraints, structure. The basic structure of the problem, meaning in a supermarket, it is not about selling units, products one at a time. It is about selling baskets. This is what makes the supermarket tick. This is the essence of the supermarket. It’s the supermarket has literally been engineered from the floor to ceiling to sell baskets.
That’s why you have those points of sales where you can unload your whole stuff and have the whole things move forward. That’s why you have a cart. I mean, everything has been engineered in the supermarket so that people can buy a lot of things at once. If you just want to buy an extra cup of coffee, it doesn’t make any sense to go to a supermarket. So my point is that because everything has been engineered, including the parking lot in front of the supermarket, to buy an entire basket, does it make sense that your predictive tool looks at the case one product at a time, everything in isolation? And my answer is no, it doesn’t make sense. So, there’s no way to adapt the time series perspective to reflect the unknowns or the intangibles that you’re describing. A mathematician would say if you pile enough time series, you can. Because you see, we could always say we can add more time series. And that’s exactly like saying we have a television that is black and white, you can have multiple television sets, and then you will have one for every color, and then technically you have the colors. So, you see, we need to be careful here. If you say that for time series you’re allowed to just introduce more and more time series, then yes, technically you can deal with any number of dimensions because you increase the dimensionality of your instrument by just adding time series. But it is not a practical solution. Just like if you want to have colors on your television, having multiple television sets is not a good solution. In mathematics, there are plenty of solutions that are widely impractical. Mathematicians are very good at inventing crazy solutions that are technically correct, but they are only mathematically correct.
In the real world, it is insane. It’s not the way you would approach the problem. It’s not going to give you a good solution. It’s going to give you a very theoretical solution. Okay, but critics might then say there are plenty of tools that are predicated upon the time series approach that actually, in practice, work. Take for example, forecast value added. Now, what you’ve just described, I presume, doesn’t fit onto that. But people who advocate that would say, actually, it works contrary to everything you’ve just said. So, yeah, people have been claiming that their tools just work since essentially the late 70s. So, since the last 45 years or so, vendors have been saying we have advanced automated software that can literally automate everything that is supply chain related. Vendors have been saying we have advanced automated software that can literally automate everything that is supply chain related. And when people say we have enterprise software to do management, nowadays when people say I have a CRM, customer relationship management, it’s just about the clerical records, data entries. But if you go back in the 70s, when they were saying management, they were thinking decisions as well, all the intelligence. So, my proposition is, in theory, we have, since the last four decades, software that are supposed to robotize entirely all those decisions: inventory, replenishment, production, scheduling, inventory allocation, price optimization. All of that is, according to vendors, fully automated, 100% automated, over since four decades. And most vendors, if you look at the way they communicated in the 80s, they were saying this will be done by the machine entirely. It used to be a clerk who did that, but that’s not the case anymore. Over the last decade, I’ve met over 200 supply chain directors and invariably, there’s software in place. There’s been a series of software solutions implemented, but everything is still done through spreadsheets.
We have several generations of enterprise software focused on time series forecasting that supposedly automated everything. They’ve been doing this for decades, but the reality is that it’s still done in Excel. What went wrong? I believe the core issue is that the time series perspective is incorrect. It doesn’t fit the structure of the problem. There are other issues, but the biggest one is that it doesn’t fit. This one-dimensional take on the future is too simplistic and everything falls apart from there. If we’re not supposed to pursue forecast accuracy as a KPI, then what should we pursue instead? First, we need to rethink what we’re trying to solve. We’re trying to produce quantitative statements about the future that make sense for the company. A statement about the future is largely domain-specific, which is the opposite of what is stated in supply chain textbooks. Supply chain textbooks claim that time series is all you need. My conclusion, after observing hundreds of companies, is that this is almost invariably wrong. If there are businesses that can be adequately modeled through time series, they are the exception, not the norm. The structure is not aligned with a time series. If you try to project a mathematical model and it doesn’t have the right structure, you’re not going to properly model the realities you’re trying to model. It’s like trying to fit a round shape in a square hole. If you’ve only ever seen a round shape, you might think that’s all there is. But there are plenty of other ways to do it, and these other ways depend on the business. If you’re a supermarket, your DNA is the baskets. If you’re in fashion, it’s going to be completely different. If you’re a fashion company, you want to generate wants, and time series don’t really make sense for that. Let’s say you have a new pattern that becomes trendy. You can generate many products that play with that, but you can have more or less products.
The bulk of your clients are in the middle. If you go for very extreme colors, you might not have enough demand to support having so many variants. If you want to think about the future, you need to think about those halos of wants, and it’s not something that matches your time series vision. If you’re selling merchandising for action figures, then it’s something even stranger. The entire business is structured around those heroes. Batman is a lot more powerful in terms of merchandising than Green Lantern, and that has been a constant for the last couple of decades. If we go into aerospace, that would be yet another thing. The consumption of parts is driven by the fact that you have a fleet of aircraft. Every aircraft has a lifecycle that lasts something like three to four decades. The consumption of parts will follow a certain curve in this lifetime. The proper structure if you want to support a large MRO that supports fleets of aircraft is to think about what are the fleets of aircraft that I’m supporting and how are they ramping up and ramping down. The reality is that whenever you pick a vertical, if you apply a time series, it’s a very crude approximation. It’s not even close to a faithful representation of the structure of the problem. If we consider the structure and go back to an exercise my physics teacher used to do, we would say, “Okay, this is a cow and we’re going to approximate the cow as a sphere.” It’s good for a toy exercise, but the cow is not a sphere in reality and not even close to a sphere. So, this is a very wacky approximation. It’s good for an exercise, but it’s not good for anything real. If you have to deal with real cows, I would not advise approximating your cows as spheres. It will not end well. This is not a valid approximation.
Conor Doherty: Yet again, when you say that a time series is a very simplistic approximation of the future, we at Lokad routinely describe our approach, which is probabilistic, as better to be approximately right than exactly wrong. Is this just a difference of terms?
Joannes Vermorel: As I said, first we have the structure. And by the way, that’s also one thing where Lokad diverges. We are using the probabilistic approach as a rallying flag, but the reality is that my issue is first probably about the structure. The second problem that I see is another one. It’s the classical textbook approach on supply chain that just asks for accuracy and assumes that the past is the exact symmetric of the future. This is not the case. This is true in a sense if you’re looking at, let’s say, the movement of planets. So, things where you’re just an observer, humanity observes it and cannot change anything. So, if you want to forecast the movement of planets, let’s say planet Mars, then yes, assuming that the past is just a symmetric of the future is just fine because we do not have any tangible measurable impact on the movement of the planet Mars. But for supply chain, it is not good because all your future is conditioned to decisions that have not been made yet. Your future is conditional to your future decision and not only your future decision but also future decisions that will be made by other people like your competitors. So, there is this radical asymmetry between the past and the future and the classical time series perspective, which is characterized with accuracy, is just entirely dismissive of that. It’s not even mentioned. It does not even exist and it is not even assessed in this sort of accuracy metrics. If you want to make a faithful statement about the future, whatever this statement is, it has to embed in itself the fact that the future is still up to decisions. You want to make a statement that is still useful despite the fact that decisions have not yet been made.
Conor Doherty: A lot of people sort of think they stand astride either side of demand and that they just observe it as you said like observers. But you’re saying that okay we don’t control the future but we can co-author it with the choices that are made. What are those though for people who are not aware?
Joannes Vermorel: Let’s say you’re selling backpacks. How much are you going to sell? It depends first on how many variants are you going to introduce. If you only have like one black backpack and you put it everywhere on your e-commerce and all your stores, then maybe you’re going to sell a lot. But then if you have more variants, you have other backpacks that are kind of similar, a little bit bigger, and then you introduce half a dozen of colors. Every single time you introduce one more variant, are you going to double your sales? No, obviously there will be cannibalization. The demand that you have for the future is not written, it’s not carved in stone. It very much depends on how many variants do you introduce. That’s a choice that you have that has yet to be made. And if you split the problem, say no, I just want to decide the backpack assortment first and then forecast second, I would say this is nonsensical. Because obviously, if you decide first your assortment and then you forecast, if you do realize that some products have not enough demand, you’re going to remove them. Literally, we are engineering the demand and that’s what companies do. That’s also the law of Jean-Baptiste Say, the economist. Supply precedes the demand. You have to push the stuff to the market to create the demand. Before Apple introduced the iPhone, the demand for iPhone in the market was exactly zero. You have to push the product first to the market and then you will generate demand for the product.
Conor Doherty: But operating within the paradigm that you’re criticizing, there are practices like forecast value added where you have demand and I go to marketing and I go to sales and I elicit their insights. We’re going to introduce x amount of variants and then that like there’s an awareness that our decisions will author the future and revisions are made downward or upward.
Joannes Vermorel: But I would say again, after observing companies for over a decade, almost a decade and a half, those are just bureaucracies. When you look at how things truly happen, you have some people in the company somewhere say, “Oh, we have an opportunity, we’re going to do it.” And then they think that if they just do it like that, it sounds unscientific. So, they want to back up their gut feeling with numbers and some people will throw numbers on top of it and then it would say, “Okay, we have numbers, it’s now scientific, we do it.” But no, it was a very valid gut feeling about the market, it was a very valid high-level reasoning about something and then they had the back of the envelope calculation to kind of size the initiative right. And then all the rest was just bureaucracy to kind of stamp the initiative but it did not contribute to the thing. It was not the initial spark, it was not the impulse, it was not even the true scientific mastery of anything that made it possible. It was just paperwork that happened afterward after the battle. You just described what Lokad does with its own clients. We communicate, they give us insights into their future plans, and we incorporate that into the numerical recipe. The functional difference is that fundamentally, we have numerical recipes that are much more versatile. We are not stuck with time series and we rarely use time series models in practice. If you want to display a curve on the screen, it has to be a time series. This is because screens are two-dimensional and we have one dimension that is time.
Under the hood, the model is not one-dimensional. Most of our predictive models do not operate like time series forecasting models. We do have accuracy metrics that align with the vision that outline the faithfulness of a quantity statement about the future. But it has very little to do with mean absolute percentage error metrics. We ask ourselves a question: are we making a statement that is truly significant, that is faithful, that is aligned with what we are truly trying to do? For example, in the aviation industry, do we have something that truly embraces this idea that we’re serving a fleet and that the fleet has some parameters that we can control? An aircraft has a lifetime of maybe three to five decades. This is very well constrained, so we can literally bake those things into our models. When we operate with clients, we have models where we just do simple things. We take the time to understand what they’re trying to solve and what statements would make sense to be faithful to the future of their business. It’s very different. If we have an accuracy metric, we start from the essence of the business, try to capture the structure, and then engineer something on top of it. It’s not even about capturing the peculiarities of a vertical, but its DNA. For example, in aviation, you have to take into account that what you’re saving as spare parts are airplanes. In clothing, there are certain fads and trends that come and go. In aviation, you have fleets that come and go. For example, the Boeing 747 is being phased out, but the Airbus 350 is being phased in. If you want to do fashion and you say you’re going to ignore novelty, my answer to that is it’s not going to end well. I strongly disagree with the perspective that is expressed in most supply chain books that these things are details. They are not. You cannot approach a vertical while being entirely dismissive of what makes this vertical specific. You cannot do merchandising for sports teams while ignoring the fact that you have tournaments and that every year the structure of your problem is that you have one team that wins. For example, let’s go back to this company selling accessories for baseball teams. How do you fit the fact that there is always one winning team and one only into a time series? You are engineering your accuracy. You’re engineering something on top of a model, this time service model, that is not making sense. You will get numbers, but…
Conor Doherty: Well, I am mindful that I do want to start wrapping up and look at some audience questions. We are a company that is bottom line financially driven from a purely financial perspective. There’s one simple criticism that may have already been asked, I don’t know, but I’ll pose it to you now. There are multi-billion dollar companies that completely disagree with pretty much everything you’ve just said. Multi-billion dollar companies that have run for a century or more. How do you respond to them who say, “Look at our bank balance, Joannes, we disagree with you”?
Joannes Vermorel: On several levels, first, companies do not agree or disagree on anything. Companies are just large collections of humans, they don’t have an opinion in and of themselves. Only people working for those companies do. So, companies do a lot of things, a lot of things, especially in large companies, is just accidental. It wasn’t really engineered that way, it just happened that way. So those are the accidents. When we say that I disagree with time series, my experience is that when I discuss with executives in large companies, they are very frequently in agreement with those sort of fundamentals that I just mentioned. When I discuss with the CEO of a large fashion company, he is usually exceedingly baffled by why the planning teams want absolutely to fit everything into time series, which is a complete mismatch to his own vision. So, am I really in disagreement? I do not think so. My experience when I deal with executives that have spent decades in a vertical, usually they have a lot of frustration with the way planning is done because it just does not reflect their core perception and understanding of their own business. At the end of the day, I trust my gut feeling more than the numbers that come from the planning team. The fact that those executives are saying that and that the company is successful proves that they’re kind of doing right. They have a planning team because they can’t scale their gut feeling. So, you need more numbers, you need this planning team and you need those tools, but they are not actually super good. I diverge significantly from what is written in the textbooks, but I’m not sure that I’m diverging that much from the gut feeling of most executives that I had the chance to talk to.
Conor Doherty: Can you give a summary of your position on why forecast accuracy is not important and then we’ll transition?
Joannes Vermorel: It’s unimportant because if I take the supply chain textbook definition, it is all wrong. It’s built on top of a faulty paradigm which is a time series forecast paradigm which is all wrong. So that’s why I say it is a complete paradigmatic mismatch. It mismatches the problem it attempts to solve and thus it’s just a fancy mathematical or statistical solution to the wrong problem. So it doesn’t matter in this sense. However, if we say the Lokad way, which is do we have something that is quantitatively faithful to the essence of the company, then it does matter greatly.
Conor Doherty: Thank you all for your questions. I’m not sure we will do our best to answer them in order they were submitted behind the scenes. So, I am reading what was presented to me and some of these are statements that you will, I guess, respond to. So, from a chap named Dustin, “Forecast accuracy is important, however, the current method of quantifying it by measuring the accuracy of a point forecast is limiting. The ultimate goal should be to measure the accuracy of a distribution of probabilities. Do you agree?”
Joannes Vermorel: Again, Lokad, we move through the probabilistic forecast in the right direction. The probabilistic forecast lets you embrace uncertainty. But still, it is not enough. That’s why I say yes, embracing uncertainty is certainly necessary, Lokad, we’re all for it. But again, go back to, if you’re still one-dimensional, it’s still not good. And if you still treat the past as a symmetric of the future, you’re still dismissing entirely this potential of decisions that have not been made yet.
Conor Doherty: Are you suggesting that forecast accuracy is more about the accuracy of execution, which encompasses insights from both internal and external changes? From a forecast perspective, should the focus be on quantity and value? Paulo believes that KPIs have the most significance when shared across different functions, especially commercial, marketing, and finance. In your view, are upside and downside scenarios useful? There are a lot of little questions there, I’ll let you choose.
Joannes Vermorel: That’s something very interesting. First, a forecast is an ingredient, an artifact in itself. It doesn’t do anything for a company. If you produce a quantitative statement about the future, the software is just an artifact. It has no value in itself. I think Paulo is very right that whatever your assessment cannot be something that is intrinsic to the forecast. It is only through the execution of the supply chain that you can assess whether this instrument, this numerical artifact, was suitable or not. You produce your numerical artifact, your forecast, and then you can only judge whether it was a good or bad forecast by its consequences, its far-reaching consequences. That’s where you need to walk back from the far-reaching consequences up to the numerical recipe used to produce the forecast to assess whether it was good or bad. That’s a very consequential approach that I have to the forecast.
As for the KPIs in sharing to the different departments, I would say be careful. Companies do not make money by having people reading numbers. Having numbers shared across teams is well and good, but does it create value for the company? Not really. And when people say KPI, it’s supposed to be key, like key performance indicators, supposedly a few. But my observation is that companies have dozens, hundreds, sometimes thousands of KPIs. So it’s not KPIs, it’s performance indicators, like a truckload of performance indicators. My point is, yes, to some extent, but beware. Companies are already paying way too many people to spend time watching metrics while doing very little afterward.
Conor Doherty: I might just quickly follow up on that because correct me if I’m wrong, you’re saying that sharing too much interdepartmental communication can be bad. But isn’t the opposite of that silos, which I know you’re not a particular fan of?
Joannes Vermorel: What are people going to do with those numbers? My take is that if you want to share data, it should not travel through the eyes and the brains of humans. We are talking about a typical client for us that has more than a terabyte of transaction data. That’s a lot. So realistically, if we say that you know through your eyes, how many digits can you read per second? Something like five digits per second. It would take an entire lifetime to channel this data through human brains. So obviously, the data, when we say we want to share data, we don’t mean that it has to go through people. Breaking the silos is not about making sure that Bob from the other department has to consume all the data that you produce and generate and reports and whatnot. It’s just about making sure that everybody has programmatically access to all the data there is in the company so that they can optimize their own decisions. And if they have to coordinate, it is about aligning the numerical recipes themselves that take the various decisions. It doesn’t mean that people themselves have to use their own time and bandwidth, human time and human bandwidth, to manually process this data. Breaking the silos is not about generating work for the other department by creating a report that you expect the people from the other department to read. Here, you’re just creating bureaucracy. You just create a bureaucratic task that you force on another department. And that’s my intuition is that most of the time when you do that, it’s not going to end into something profitable for the company. It might, but it’s not a given and most of the time it won’t.
Conor Doherty: Thank you for that. I’m mindful we have limited time, so this question is from Sashin or Sain. How applicable is the basket concept or the basket perspective for B2B businesses as opposed to just consumer goods?
Joannes Vermorel: It’s essential. Let’s take an example. One of our clients at Lokad is a B2B distributor of electrical equipment. It’s a very large company. When you sell electrical equipment, your clients are big businesses and the bulk of your business is driven by construction sites. So yes, there is the occasional company that is going to order a light bulb or a light switch just to do a small repair, but the bulk of the business is driven by construction sites. There is a new tower and 6 months from now, you need 4,000 light switches all the same model at the same time and you need 200 km of cable, literally. And so we have, and that’s not an edge case, this is a very classical thing when you look at civil construction. There is a building that is constructed, there will be businesses that pass big orders to say all they need to equip in terms of electrical equipment the building. And so let’s say 6 months ahead of time, they will not expect this electrical distributor to have everything on stock. Nobody has this amount of stock readily available, so the company doing the installation of the building knows that. So months ahead, they will pass a big order and they know that it’s not going to be available, so they pass it months in advance. But they say we give you a lot of time, but at this date next year, end of March, we want to have everything ready because then we will proceed with the deployment in the building and we need every single thing. So we have a big order, a thousand references, for every reference there are hundreds of units and we need every single thing down to the last unit perfectly available at this date. And we don’t trick you, we give you plenty of months to make this happen. And this is what, so you see in this case, the interesting thing is that we deviate again from the time series. We have demand but if you think about the demand as time series, you’re missing the point. The point is you have two dates, you have the order date and you have the intended availability date for the merchandise. So this is a basket as well, this is B2B and it comes with an extra complexity compared to the supermarket that whatever you need is announced ahead of time. Now is it perfectly known? It is not because what happens is that there might be small deviations in the construction schedules and then the client may come back to you say we need those things one week earlier or one week later. So you see there is still some variability and then as construction site progress, they might make marginal adjustments to their basket. But you still have the bulk of the information available a lot of time. So you see again, even if we’re looking at B2B, we have those sort of phenomena. Even more so, I would say B2B is all about repeat business about well-identified partners. So this sort of basket analysis is even more prevalent in B2B than it is in B2C retail.
Conor Doherty: From Stefan, or Stefane, the French, I believe Stefan has a comment. He says, “One can potentially feed a vast amount of data, structured or not, to an advanced AI to get a forecast. However, there is a catch to this, isn’t there?” That’s a question. Maybe you know?
Joannes Vermorel: Yes, I mean, people think that the alternative to a time series is some sort of Skynet AI. My answer is, why do you think that? If all you’ve ever seen in your life is round shapes, you’ve never seen a square shape, you might think that the alternative to a round shape is an incredibly complicated shape. That’s not what I’m saying. I’m not saying that the alternative to a round shape is something that is impossibly complicated. It might be just a square shape. I’m not saying that the alternative to time series is a Skynet level AI or whatever. Most of the models that Lokad uses are very simple, they are just not time series. There is this sort of cult that it has to be time series. I say, why not? The mathematics are vast, there are tons of alternative things that you can do that are not more complicated than time series. They are different. Time series being the simplest of the simplest, yes, they are a little bit more complicated, a little bit, because there is almost nothing that is simpler than time series. Time series is literally one quantity with a time dimension, so it’s hard to do because we will have to deal with the time dimension. It’s hard to be simpler than time series because time series is already super simplistic. But it doesn’t mean that the alternative to time series is Skynet level AI. Those models are still parametric, very simple, and it’s just about embracing the structure of the problem that you’re trying to solve. When I describe the structure of the problems, like a series of baseball games with one winner every year, the other teams lose, we are not talking about impossibly complicated structures. Those things are not that complicated, they can be described in minutes, and the models that Lokad typically uses can also be described in minutes. Time series can be described in seconds, so we are more into the sort of stuff that needs minutes to be described.
Conor Doherty: Well, in terms of the difference between the shapes, and I will briefly follow up on this, when people talk about measuring how good or faithful a forecast was, they look at accuracy. We don’t look at that. The other shape that we use is financial impact. Is that the alternative shape?
Joannes Vermorel: That’s part of our bag of tricks. The financial perspective is not like we are adamant about it. It is just that in my experience, when we are dealing with a large company with a vast supply chain to manage, we have a problem of reconciliation of dozens of conflicting goals. You have so many goals. You’re a large company, you want not to waste, you want high quality of service, you want maximum utilization of your warehouse and your assets, you have constraints such as maximum storage space, you have shelf life. So, you have constraints and goals all over the place. We need a language to unify all of that. It’s a very practical thing. These things kind of conflict against each other. Quality of service conflicts against waste. If you say I want to have a super high service level, then if we go for perishable, you have a very high service level that means that sometimes you will have inventory that expires that you have to throw away, so you create waste. There is tension. You cannot say I have zero waste and very high quality of service. If you have high quality of service, you will have some waste, and if you entirely eliminate the waste, it means that you very frequently end up with stockouts. This is inevitable. This is just the design of the problem itself. You have those conflicting goals. So now, let’s magnify that. We have a large company, we need to unify all those things, and my proposition, that has been the bag of tricks that Lokad uses, is that if we put all of that expressed in dollars, we have the lingua franca. We have the way to just unify. It’s just a bag of tricks. It’s not that I want to think in terms of dollars, it’s just my experience. It’s the only thing that scales when considering complex companies. That’s just a matter of practicality at scale.
Conor Doherty: Thank you. So, should this video be titled “Does Time Series Matter?” You believe quantifying uncertainty and forecast accuracy is vital, but disagree with the current methods, correct?
Joannes Vermorel: Again, accuracy and time series are the same thing. If you look at supply chain textbooks, I’ve never seen any supply chain textbook in which the accuracy is not immediately associated with a time series. Most supply chain textbooks would not even bother giving the mathematical definition of a time series. They would directly jump to the accuracy definition, which in itself defines the time series. So, you see, those things are co-substantial in the mainstream supply chain paradigm. They are one and the same thing. And Lokad is saying that we want to tear them apart. Indeed, there is a way to make accuracy matter, but it is something that is so radically different from what is actually presented in those supply chain textbooks that I’m very hesitant. I’m on the fence of calling it accuracy. Accuracy is a good term, it is valid, and that’s what we do morally. But what we’re doing is such a radical departure from what is found in the supply chain textbook that it just creates confusion when we use the same term.
Conor Doherty: Thank you. I believe we did touch on this in the last two questions. This is from Constantine. Some advocate for FVA, your favorite, as a means to determine if efforts to improve accuracy are worthwhile. You recently released a review criticizing FVA. What do you suggest as an alternative?
Joannes Vermorel: So, here I will give an answer. It’s not from me, it’s actually from TOA. When a surgeon removes a cancer from your body, what do you replace the cancer with? So, with FVA, my take is that the accuracy done with the mainstream paradigm is a bogus idea. It does not stand scrutiny when you want to look at the essence of the business. Is this mathematical instrument, a line, making sense at a high level with my business? And my proposition is that when you look with a modicum of attention, it does not. So now, FVA is just over-engineering a process on top of a bogus paradigm, a bogus tool. So, you’re just making it worse. FVA is just moving the company further away in the wrong direction. So, you had a bogus concept, those time series accuracy, and now you want to engineer a process on top of it to create some kind of mini bureaucracy in the company. So, my take is that it’s not the first, nor will it be the last, useless bureaucracy that gets introduced in the company. Large companies have dozens of many useless bureaucracies floating around. So, in the end, it’s not endgame for the company by just having one more useless bureaucracy floating around. But will it make the company’s supply chain more competitive? No, not even close. It will do the exact opposite. Although it’s not going to break the company either. It’s the sort of thing that just adds some cost and the company will move on.
Conor Doherty: Okay, I will just push back a little bit on that because we both like Thomaso’s analogy. When a surgeon removes a cancer, what does he put in its place? If you were to apply that to this context, it’s almost like saying, well, we’ve removed this, sit on your hands, don’t put anything in its place. What will fill that vacuum?
Joannes Vermorel: Let me describe the reality of what accuracy actually means in a large company. There is this SNOP process with a bureau CES that produces forecasts, and then people assess those forecasts. Are they used? No, they are not. All the large companies I’ve been in touch with for the last decade, over 200 large companies, when I inspect and audit them, I realize that the entire company runs on shadow IT spreadsheets. All those numbers that float from the SNOP process, they’re not used. The sales people, manufacturing, supply chain people, people with logistics, transport capacity, they don’t use those numbers either. It’s like a Potemkin village. There is this illusion of rationality where people produce these grand things with SNOP and revisit it once per quarter. But then every single manager has their own secret spreadsheet on the side that they use, and this is what drives the business. The interesting thing is that every single manager thinks they’re the only one to have this hidden spreadsheet. I’ve had multiple times where a VP of supply chain told me they have a secret spreadsheet because the numbers they get are garbage. But for their subordinates, they demand that they stick to the official SNOP process. As part of the audit, I interview the subordinates and they tell me they have a secret spreadsheet. They don’t trust the numbers so they do it differently. And they all think they’re the only one to have this secret spreadsheet. I’ve seen this situation over and over. You have bogus numbers in the SNOP plan, yet the final decisions come out right. How is it possible? The answer is invariably there is a spreadsheet somewhere that is crafted in a way that is a lot more aligned with the essence of the business. People are just hiding the spreadsheet because it’s not the official policy, but nevertheless, it is what makes the company tick, not the grand Potemkin village of those grand numbers.
Conor Doherty: Thank you. We do still have a few questions to get through, so I’m going to have to beseech brevity going forward. This is from Sean. He writes, “Forecast accuracy is one element in the supply chain. It may not be the key constraint in a particular business. Capitalizing on an improved forecast usually requires other changes in the supply chain. Do you agree?”
Joannes Vermorel: Does capitalizing on an improved fax machine matter? You see, that’s the thing I’m trying to convey. When people tell me we have better accuracy in the classical sense, it’s just the same thing as telling me you have a better fax machine. It’s not because it’s an improvement compared to the status quo that it’s an improvement overall. That’s the bane of incrementalism in supply chain. People only see improvement through the lens of, “Yes, it is incrementally better compared to what we have.” If you can only think about better fax machines, you’re not in a good position. When people say, “Oh, you’re talking about AI, Skynet,” I say, for example, email is fundamentally simpler than a fax machine. A fax machine is more sophisticated, more demanding in terms of technology, and yet it is a crappier solution compared to the alternative. That’s my point. When people say, “Oh, we have this improvement,” I say, “Yeah, you just have a better fax machine. Congratulations. But you’re missing the point. You’re missing the opportunity to just do something that is simpler, better, more aligned, faster, leaner on all fronts.”
Conor Doherty: Thank you. Moving on, from Philippe, “When discussing structure in forecasting, can a time series forecast be a component of that structure to some extent when applicable?”
Joannes Vermorel: It can be. Time series is such a fundamental structure. It’s very difficult when you craft something to not have time series that just emerge accidentally even as a component of your predictive ingredient. My message is not that time series should not be used. This is not the essence. I’m just saying that if all you have is time series, it’s very simplistic. You need to expand your vocabulary, your horizon. There are other things, and in those other things, yes, you can have time series. It does happen occasionally.
Conor Doherty: Next, from Manuel, “Universities continue to teach traditional forecasting methods and emphasize their accuracy. With the recent introduction of machine learning models that consider many additional factors, has this changed the viewpoint presented today?”
Joannes Vermorel: The difference between having a black and white television, which would be a classical screen, and a light that projects things, that would be the big flat screen, that would be the old school time series. Machine learning just gives you the black and white LCD screen. It’s still black and white, one has better qualities, it’s leaner, it has its place. My problem with universities is not about the better numerical model. My problem is not that universities don’t teach the right forecasting algorithm, it’s that they don’t try to teach the right forecasting attitude. Are you looking at the essence of the problem? Are you trying to produce a statement quantified that makes sense for the business? Is it what you’re doing? Does it make sense? Do you take into account the fact that the future is not the symmetric of the past? Again, attitude. And then we have the technicalities. For me, classical time series and machine learning, there is a world spectrum. If we look at the Walmart competition that we earned fifth place in, the trick was we used a super simplistic parametric model with five parameters. So does it count as classic? We ranked out of a thousand competing teams, we ranked number five and we even ranked number one at the SKU level, above everyone, with a super simplistic model. The interesting thing is that it’s a super simplistic, like five parameters model. So in a sense, it is an old school model, but the way we learned those parameters was through more elaborate differentiable programming. So, is it machine learning? Is it old school? For me, there is a world continuum from super classical autoregressive models to super fancy deep learning models. There is no quantum leap, all of that is there. My problem is not that universities do not teach correctly those forecasting algorithms, it’s that they do not teach correctly the forecasting attitude, the mindset that you need to have when dealing with the future for supply chain purposes. That’s the problem. The goal of the S&OP process is to create company-wide alignment. That would be the goal, so that people on the production front produce what is going to be sold by the sales team and that sales people produce what you can fulfill. It’s literally about corporate alignment. But in practice, S&OP practices are an endless series of meetings. That’s what it is.
My take is that information flows through the IT systems, the applicative landscape. We have competing paradigms. We are not even on the same page. I say the information flows and if there is coordination, it’s not going to be about the information. The information flows through the applicative landscape. You don’t need to have people that talk to each other if you have to create alignment. It’s going to be about the numerical recipes and clarifying the strategic intent of the company, which is absolutely not what is done in S&OP meetings. A lot of large companies have decent results, but those time series forecasts are just part of the bureaucracies that do not contribute at all to anything. You could remove it and it would work just fine. During the lockdowns of 2020 and 2021, some companies in some countries had some parts of their white-collar workforces that were put under technical unemployment for 14 months. The company was still operating at 80% capacity. It was reduced but it was non-zero. Due to those lockdowns, all the white-collar workforces, especially from the planning, were literally told to stay at home and not ever touch the corporate computers. We had a grand experiment where the entire planning department is gone for 14 months and everything is fine. So, if a company can operate without a division for 14 months with all of the people of this division absent, what does it say about the division? Probably that it’s not exactly super mission critical. We even had a case where a large company, a manufacturer, essentially became an e-commerce during the lockdown period. The e-commerce segment was 5% of their business before the lockdowns. At the end of 2021, e-commerce was two-thirds of their sales. So, the company underwent a massive transformation going from 5% e-commerce to being de facto an e-commerce company. If your company can undergo massive, rapid transformations and execute them well, what does that say about those functions? I’m challenging that notion. I’m not saying that accuracy doesn’t matter, especially in the specific sense that Lokad has. But if we look at how it’s usually practiced, I’ve observed over and over that we’ve had lockdowns, we’ve even had the grand experiment of shutting down the division in charge of those accuracy metrics for over a year, 14 months to be exact. And what was the impact on the business? Nothing, business as usual. Some of those businesses even thrived after that. That was an eye-opener for me. This is an experiment that shouldn’t have happened, but it did.
Conor Doherty: Thank you. And the last question, also from Nicholas, possibly a different one, I don’t know. I often find departments trying to override statistical data with gut feelings. How do you define the impact of forecast accuracy on improving inventory and customer experience in real time?
Joannes Vermorel: That’s the thing, this is a question that is never asked as part of the mainstream time series accuracies paradigm. It’s not a question that is being asked in supply chain textbooks. This is absent. But this is not the only concern. There is a whole area that we did not touch, which is the instrumentality of the forecast. How well are they to be actually used in the company? And those things are absent. So yes, this is very important. And connecting all the dots from decision to the mathematical model that produces those statements is very important. But that means that you need numerical recipes that go end to end, from the predictive generation of those quantitative statements about the future to the decision that you take, and that have real-world consequences with financial impacts on your company. And that’s how you will assess whether your predictive model is faithful or not. I’m using faithful because I don’t want to use the word accurate. And part of the bag of tricks is this financial perspective because it facilitates doing that. But the way it is usually practiced, it stops halfway. There is this grand illusion that is entertained by supply chain textbooks and most supply chain software as well, that you can cut the problem at the forecast stage and say in isolation we are going to assess how good we are or how bad we are in isolation with the rest. And this is complete nonsense. There is no such thing as assessing in isolation the adequacy or the accuracy of a forecast. This is just about benchmarking mathematical models. It’s good, but it’s not real life. It’s just like if you want to have the winner of the shooting range. You may have an Olympic champion of shooting, but if it comes to an actual military exercise, people do not shoot with actual guns in an actual war the way people shoot in a controlled environment. This is completely different. The concluding point is that part of the accuracy problem is that the problem itself is framed incorrectly. It’s not that accuracy is false in the mathematical sense. That’s not what I’m saying. I’m not saying that, for example, forecast value added is incorrect in a statistical sense. This is not what I’m saying. I’m saying that the paradigmatic environment that surrounds those concepts is inadequate. If you have to choose between gut feeling that truly embraces a business versus super sophisticated but completely mismatched business, approximately correct trumps exactly wrong every single day. That’s the thing. And accuracy exemplifies the traditional way this exactly wrong mindset works.
Conor Doherty: Joannes, I don’t have any other questions. I mean, I do, but I’ll save them for another day. Thank you very much for your time. And for any of you who have stayed with us this long, thank you very much for your time. We’ll see you next time.
