Full transcript
Conor Doherty: This is Supply Chain Breakdown, and for the next 30 or so minutes we’ll be breaking down the hype surrounding digital twins, particularly in supply chain. My name is Conor. I’m the Communications Director at Lokad, and joining me in studio to my left, as always, the ineffable Joannes Vermorel, Lokad’s founder and CEO.
Now before we start, do comment down below—first of all, where are you from, and secondly, what do you think about digital twins? Remember, we’re here to talk about your concerns. Also, get your questions in whenever you have them. If you hear something Joannes says and you want to push him on it, feel free to do so. Put it below and I’ll ask him in about 20 or so minutes.
And with that, no more wasted time. Joannes, take away the hype of digital twins because I know it’s been around for a while, but set the table: what is a digital twin in basic, simple terms? And also, what is the problem that people at least think digital twins are designed to solve?
Joannes Vermorel: So, considering what is digital twins, I would say if you take the consulting version—which is not the one I would follow—they would tell you: it’s an accurate representation of your supply chain, fully digital, that lets you project pretty much anything into the future. As if you had a complete simulation, perfectly accurate, of your entire supply chain. At least that’s the ideal perspective for your supply chain digital twins.
And the idea of digital twins also exists in other domains, and for other domains you can have sometimes a very accurate simulation of a physical system. Now, in practice, what is it? In practice, it is just a repackaged Monte Carlo simulator. Those simulators have been around for like four decades. The one key characteristic is they are very, very easy to implement, and it is relatively fun, as a toy project, to implement a Monte Carlo simulator of a system as long as you don’t really care about anything like accuracy of simulations.
Conor Doherty: I’m immediately going to jump in because you raised a couple of interesting points there. So you gave at least two applications there. One was how digital twins might be marketed in a supply chain context—that’s one class. But you said there’s a pre-existing or more enduring application; you said for physical products. Could you tease apart that separation a bit more?
Joannes Vermorel: Yes. I mean, the idea of having simulators—there are three key ideas: micro-analytical, discrete, and stochastic. That’s three ideas. When people think Monte Carlo simulators, that’s the three aspects that you have.
Micro-analytical means that you want to decompose your system into something like atoms or very, very small elements that have simple behaviors governed by simple laws that you can completely quantify. That’s the micro-analytical take. Then discrete: for your computer you’re going to assume that everything behaves with nice increments—in supply chain it does make sense—including for the time dimension. So you’re going to say, “Okay, I’m going to create a simulator that operates with one step per day,” or something like that.
And the last thing is the stochastic aspect: some behaviors, you’re just going to add some randomness. That’s why I’m talking about a Monte Carlo simulator. You attribute some behaviors that have some random or stochastic behavior to them, and then you can run repeatedly your simulator many times to supposedly project a future state of your supply chain, or at least you can have the simulator moving forward in time. And because it’s supposed to be a digital twin of your supply chain, it represents the future of your supply chain.
Conor Doherty: Thank you, and I think this actually raises again a key point for where the conversation can be separated. When people talk about using a digital twin in, let’s say, manufacturing or repair, you can say, “I have a digital twin. It is a deterministic representation of a full plane.” So an A320 has approximately 320,000 parts. You know how many parts there are; you can model that; it’s fixed. That is a digital replica or digital representation of a physical, known product or property.
How well can you apply that same concept to a geographically distributed network like a supply chain that encapsulates not only physical parts like products and trucks and stuff, but also behaviors, tendencies, politics—all of which impact those things?
Joannes Vermorel: The main problem is the laws that govern your atoms, your elemental simulation units. If you are in the physical world you can literally apply electromagnetism and whatnot to have a physically accurate evolution of your system, because you assume that if you decompose things enough, what remains will obey very, very well-known laws. If you want to describe fluid going through various tubes and whatnot, you have the microfluidic equations and you can do a finite-element simulation.
But the key insight here is that it assumes you have laws that govern the elements of your system that are pretty much perfectly known, and then the only loose end is the resolution of your simulation, which might not be sufficient to be super accurate. The problem with a digital twin in supply chain is that the challenge is not really the resolution. The challenge is that it’s not clear at all that you have any relevant prior knowledge about the behaviors of those simple units, because those simple units—let’s say you want to simulate a supplier—it’s absolutely not simple.
There is no— you can make simplistic assumptions about the behavior of this supplier as much as you want, but it’s not because you’ve just conjured an assumption about this entity that it’s going to be true. And the same can be said for everything: customers, warehouses, routes that connect your whole supply chain, etc. So there is really a fundamental problem: yes, you can create a simulator by decomposing your system and just conjuring behaviors for all the atoms, but are those behaviors of any kind of correctness? That’s the big, big challenge, and it’s typically something that is never discussed with those supply chain digital twins—the correctness of the modelization.
Conor Doherty: Correctness and other metrics we’ll come back to, but I want to dig down a little bit deeper. In fact, I’m going to read you a very crystal-clear definition. Whether or not you agree with it, it is a clear definition, and I want to get your reaction to it. I’m not going to tell you who it’s from yet—you can guess later—but it’s punching up. So, quote: “Digital twins can be used to model the interaction between physical and digital processes all along the supply chain from product ideation and manufacturing to warehousing and distribution, from in-store or online purchases to shipping and returns. Thus digital twins paint a clear picture of an optimal end-to-end supply chain process.” Now, what is your take on that? What parts of that do you disagree with?
Joannes Vermorel: This definition says nothing except the intent. It just says, “Okay, the intent is to have something that does all the things this definition described,” which is an end-to-end accurate representation of blah, blah, blah. That’s the intent. It says nothing about how it does it. And my message is: the how matters.
You can invent or wish for a lot of incredible tools. I would like—what is a perfect AI assistant? That’s something that has a much greater intelligence than a human and that can do everything that a human can do, but just better. Okay, I’ve just defined what would be a perfect AI assistant. Does that mean there is one readily available? The problem is if you define something by its intent, it doesn’t say anything about the feasibility or the fact that this thing is even real.
So yes, we have a nice intentional definition. And me, when I say “Monte Carlo simulator,” I did the exact opposite. I started with, “Okay, what are companies doing under the hood when they use the buzzword ‘digital twin’?” And my answer is, as far as I can observe: Monte Carlo simulator. Why? Because it’s extremely simple to implement. Literally a first-year computer science student can do that in just a few days—probably even the smart ones can do that in one afternoon. And yes, as long as you don’t care about whether your simulation has any relevance for the real world, it’s super straightforward to implement.
Conor Doherty: Coming back, you’re talking about the intent. The purpose, at least as presented by many people, would be—give you something to respond to—that many vendors and consultants and champions of a digital twin would argue that a digital twin improves decision-making by effectively providing—even if it’s through Monte Carlo—a sandbox in which you can play with what-if scenarios, like “What if the supplier is late? What if there’s a Suez Canal blockage?” or whatever.
Joannes Vermorel: An artificial general intelligence improves the profitability of any company—by definition. If you have something that is as smart as a human but has no salary, yes, it will improve. Okay. But let’s get back to the question of: is this thing available, and what are the techniques? Those parties present it as fait accompli: it’s already there, we have this, it has all those good properties. But, hold on—what do you have under the hood?
And my message is: as far as I know, literally all the companies that I’ve seen pushing forward digital twins had nothing but naive Monte Carlo simulators, which in my book is toy-level sophistication. It is fancy. It’s the sort of nice exercise that I would give to first-year computer science students. Yes, it is fun to implement that, and it is easy. But the problem is, again, it is just entirely made up.
Yes, you can decompose your supply chain into a list of SKUs, list of clients, list of suppliers, and assign behaviors to all those entities, and then let the simulation run—absolutely. But is it actually correct? That’s a very big question. An analogy would be: just take SimCity 2000, the video game—an old one. They have a map editor. You can literally edit a map that would look just like Paris, plot all the roads exactly like they are in Paris—almost, again you have the problem of discretization, so it’s not the exact same roads, but close enough. Then you can say, “This area is housing, this one is industry, this one is commerce.” Yes, you could do that, and then let the game simulation move forward.
Does that give you an accurate simulation of the future of the Paris urban environment? My answer is absolutely not—especially when Godzilla is going to kick in; that’s a disaster that happens with the game. It’s not because it’s very easy to create. Again, I would restate: it is very easy to create simulators, and it’s fun. The question is really: why do you even think it’s going to have any kind of accuracy?
In other areas such as physical sciences—let’s say material sciences—your simulator is good because they obey fundamentally the laws of physics, which are very well-known and verified. So fundamentally your behaviors are not made up: you literally take physics textbooks and apply electromagnetism or fluid dynamics or whatnot, and these are the behaviors that govern your elements, your atoms. But in a supply chain digital twin you have to invent and conjure those—or discover somehow—those behaviors, and that’s a very, very tricky part. To my knowledge, digital twins for supply chain—vendors, the people who are pushing that—don’t really say anything about that, and that’s the whole crux of the tangent.
Conor Doherty: You’ve mentioned accuracy and correctness, and again, just to push back—by the way, that previous definition was from McKinsey, the one you disagreed with. And from one to another, because this one’s now BCG—oh, sorry, you want to say something?
Joannes Vermorel: Again, the problem is they describe the intent. When there is a piece of technology, I really prefer to focus on the how. Defining a piece of technology through the intent of what you want to achieve is nice, but then you can’t reason about whether it is a good piece of technology or not. I don’t disagree with the intent—the intent is fine, yes. The question is: do you have a definition that lets you identify if this piece of technology is better or worse than another piece of technology for your supply chain?
Conor Doherty: On that note, according to—again, we’re punching up with the citations—according to Boston Consulting Group, companies with digital twins improved their forecast accuracy by 20 to 30%. I mean, isn’t that something that most companies would kill for? Are they wrong to pursue this?
Joannes Vermorel: Digital twins have never surfaced, as far as I know, in any forecasting competitions. There are plenty of forecasting competitions; there are dozens of techniques used to get better forecasts in those competitions. I’ve never heard of anything related to digital twins doing anything whatsoever with regard to forecasting.
So you see, the problem is that, for me, because they don’t define what techniques they are actually referring to, it could be anything. My take—and that’s an empirical take—is that companies who are marketing digital twins are essentially building Monte Carlo simulators, and these do not bring superior accuracy to the table. Again, I don’t know how those numbers have been collected, but the reality is that it’s not even in the realm of the stuff that can actually improve forecasting accuracy.
Conor Doherty: But granted there’s another point to consider, surely, which would be that—is greater accuracy something that by necessity leads to better decision-making, the thing people are using the digital twin to achieve? So there’s an instrumental use and then there’s the thing you’re actually trying to accomplish—the goal, the purpose.
Joannes Vermorel: Yes. If we say that what we want is better decisions with higher return on investment, then conceptually the digital twin could say: if I have a policy A, I run it through my digital twin, I assess the return on investment; then I have policy B, I do the same thing, and I will pick the policy governing my decision that just gives me a superior rate of return. Conceptually, fine—why not?
But again, all of this hinges on the fact that your digital twin is giving you a correct economic assessment. So this accuracy problem comes back at you in terms of dollars: you have a rate of return for policy A, a rate of return for policy B. But again: does your digital twin have any kind of correctness? Why would you trust it? That’s a very complicated question.
We can go back to SimCity 2000 and Paris. I can let the game run forward with different taxation levels for the city—it exists in the game. But will this thing tell me something very accurate about the city of Paris? I think it would be obviously completely insane to think that a video game can be used to accurately model the evolution of an actual metropolis. That’s the same sort of problem I have with those digital twins for supply chain. Unless you add a lot of things, what you just have is wishful thinking.
Yes, it would be super nice if we had something that does all of that very accurately. If you tell me this Monte Carlo simulation is something groundbreaking that is going to make this possible—I really don’t think so. At best it is a tiny ingredient: super simple. That would be a little bit like: “Computers are going to be involved.” Yes, probably. It’s a good idea to involve computers, just like having a Monte Carlo simulator is a nice technique. It’s a tiny one; I won’t disagree. It’s just very, very shallow. It’s a little bit like, “It would be better to use metal for a car.” Certainly metal would be involved, but it’s not going to get you a car.
Conor Doherty: I don’t want to tar too long on this because there’s audience questions and I also want to talk about some of the stuff that was mentioned on LinkedIn, but before we push on, one last, I guess, technical-ish question. One of the reasons why people like digital twins—and they’re so popular—is they are vaunted, they are marketed, as a way to tackle volatility and confront uncertainty. Now I know for us at Lokad, and for many other practitioners, probabilistic forecasting will be the way to do that. So, high level, what are the limitations? How does probabilistic forecasting, in your estimation, fill in the gaps that the digital twin leaves?
Joannes Vermorel: First, a Monte Carlo simulator at the scale of your supply chain is effectively nothing but a probabilistic forecasting model. That’s literally what it is. When we say probabilistic forecasting, we are not very specific on which models underneath are actually used for that. In the lecture I give a series of examples of how you can build those models. But when you just say “probabilistic forecasting,” you’re not saying anything about the model itself.
You can build your probabilistic forecasting model with a Monte Carlo simulator. You just let the simulation run, you gather your probability distributions that are empirical, and if you run your simulations thousands of times, you’re going to collect your nice multi-dimensional histograms that will give you your probability distributions, and here you have your probabilistic forecast. So there is a duality between a Monte Carlo simulator and a probabilistic forecast. From the probabilities, you can generate deviates—so that gives you those stochastic behaviors. But if you have the stochastic behaviors, you can run them and get the probability distributions. Fundamentally it’s the same.
However, there is one key thing: as soon as you start thinking of your Monte Carlo simulator as a probabilistic forecasting engine, then you can assess its accuracy—which is also very important and that I find very lacking in the vendors pushing digital twins. They never even mention the fact that what they have is a probabilistic forecasting engine, and thus it needs to be assessed in terms of accuracy with metrics relevant for probabilistic forecasting.
Conor Doherty: Okay. I know we could talk about that forever, but again, the topic as advertised was specifically digital twins. A couple of days ago you ran a poll on LinkedIn where you asked your audience, “What are the biggest blockers for digital twins?” because a lot of people who are watching, and who will see this later, are companies who are considering adopting this technology or are already adopting this technology.
Now, in that poll, 52% of the voters said that the biggest blocker was messy or incomplete data. First of all—and this is something I remember you said a few years ago—considering the size of companies that would entertain a digital twin, you’re talking about very large companies; presumably they have expensive, reliable ERPs. So the question becomes: how exactly is messy or incomplete data a blocker to something that you said is quite simple to implement?
Joannes Vermorel: Here, if you define by missing data the behaviors—what characterizes the behavior of all those entities—I would say yes. But there was never, I think, an expectation to find in an ERP the parameters that could characterize the behavior of any of your clients, for example. So I don’t think that’s what people mean—no doubt.
My take is that data is always the scapegoat. When the technology is super shallow and it doesn’t deliver what was promised, invariably the bad data is the scapegoat. That really does not match the experience that we have at Lokad. My experience is that companies above one billion dollars or euros of annual revenue have excellent data. They know pretty much exactly what they’re selling, they know exactly what they’re buying, and they know exactly what they have in stock. Yes, there are clerical errors here and there, but we’re talking about something like 0.1% error for clerical errors.
Overall, as far as data accuracy is concerned, it is perfect. The whole flow of goods—from acquisition, transport, transformation, and distribution—all of that is known with near certainty. The quality of this data is very high. Yes, some other data can be a little more fuzzy, especially if you go into promotional plans or these sorts of things, but the core transactional history is excellent. Technically, that’s the only thing that you truly need to build a simulator. You would have these systems that evolve over time, generating transactions, and your digital twin should be able to be the mirror of this thing into the future and predict the future state of your systems.
But they’re not. That’s the problem. According to them, they are not, and then data is blamed. Whenever I hear those poems about data problems, what I hear is an inadequate technology that was essentially generating garbage, and people say, “Oh, the output is garbage,” but it must be because the input is garbage—the GIGO problem, garbage in, garbage out. But what if the input is just fine? My take is that the input is just fine, pretty much—and has been fine for the last two decades—for the vast majority of large companies.
That doesn’t mean there are no data challenges. One of the core challenges is that when you look at an ERP, there are 10,000 tables, and each table has like 50 fields. That’s a massive challenge. But that doesn’t mean the data in there is garbage. It just reflects the fact that your business systems have not been engineered to make your life as a data scientist easy, in order to engineer your Monte Carlo simulator. That’s a completely different problem.
Conor Doherty: Speaking of problems, another problem cited by 21% of the voters was fragile or buggy modeling. You previously said that installing this is not especially complicated—it’s the kind of thing you could throw to a first-year computer science student. So again, I come back to the question: if you’ve got multi-billion-dollar companies, and an exercise that’s a very straightforward process, why are a fifth of respondents saying that modeling is the problem?
Joannes Vermorel: Monte Carlo simulators are conceptually super simple, and in terms of implementation, super straightforward. However, you will face very basic performance problems very soon. Let me explain. How many SKUs are we talking about? A million—that’s give or take for a company above a billion. Let’s say a million SKUs.
Then, how many CPU cycles do we need to simulate a SKU just for the behavior that we’re talking about? Let’s say 10 CPU cycles minimum, and we are super efficient if we say that. Then how many days? Let’s say we have a simulator that runs one day at a time—100 days. So we are talking about 1 million times 1,000—a billion CPU operations—to simulate 100 days, as a ballpark.
The problem is that it’s stochastic—it’s a micro-analytical, discrete, stochastic simulator—so you need to repeat your runs. How many times do you need to run your simulation so that you can have somewhat reliable statistics? Because of the resolution problem, you need to repeat at least a thousand. So now we are at a thousand billion operations. Not a problem with modern computers, unless you do something really fancy with the computer and start doing parallelization and whatnot. We’re talking about 20 minutes’ worth of compute, again plenty of solutions to parallelize all of that, but people who are just doing a quick and easy simulator are not going to do all of that. So we just assume something straightforward—fine. Twenty minutes doesn’t look too bad—except.
Except that now you want to check options. For example, “Should I put this one unit here or this one unit here?” Every single option that you explore, you are going to have to repay this tax, because you’re going to run your simulation for your baseline and then you try something and run it again. If you have just a super high-level concern—such as “What if the cost of working capital changes? I just want to know the output”—that’s fine: you run it twice and you will have the difference.
But if you want to start asking questions like “How many units should I be putting in every single store location?” you will have to run your simulator thousands and thousands of times—one for every single micro question you want to answer. Suddenly people realize, “Oh, 20 minutes is so slow. I need to run this hundreds of thousands of times, possibly millions—one for every SKU or something.” Then it becomes a big problem. The solution becomes: this micro-analytical take where I was simulating everything at the SKU level—ah, it becomes such a pain. “So we’re going to have a simulation at a much more aggregated level.”
But now we have another big problem, which is: everything was hinging on the fact that the behaviors for your atoms of simulation are correct at the SKU level. It was already a stretch to say that it is easy, that you have obvious behaviors that would apply. If you start aggregating at the DC level, what makes you think that you can ever come up with the correct laws that govern the flow in and out of a distribution center? This is a very complex piece of your network. There is no reason to think simplistic laws will govern this thing. Same thing if we talk about a client in B2B—a client that is ordering tons of very diverse products on diverse timetables, etc.
So you had this simulator problem. The simulator is easy at the micro level, but you very quickly end up with performance problems. You can re-aggregate, but if you re-aggregate then the problem of having accurate behaviors for your simulator is absolutely magnified.
Conor Doherty: I think a key part of that, just in case it’s the first time people are hearing us talk about that: when you talk about decisions, the decisions under the Lokad rubric are not limited to just, “Well, I have one unit; I send it there or not.” I mean, there are combinations for all of these decisions. I could send one there or none. I can send one there and one there or none, or two there and one there, or liquidate, or I can raise the price of one unit here in this place. So the local perspective is incredibly granular, just in case people were not clear on how granular you’re talking about here. It’s like under a microscope—that’s how granular.
And by the way, the reason that we are so granular is that if we go back to modeling those behaviors, if we want to have any confidence in the economic outcome, we need to be at the very bottom. It’s the only area where we can actually reconnect how much it cost to produce this, how much the client is paying for this one unit, etc.
Joannes Vermorel: Exactly. That’s the reason we’re at the very bottom. The problem with digital twin initiatives is that they very quickly—with their Monte Carlo simulators—realize they have performance problems, and so they directly switch to a higher level of aggregation that’s easier to compute. But then the problem of having those correct behaviors becomes absolutely enormous, and then yes, you have a simulator that is potentially widely inaccurate. It’s not even clear why you should believe the simulator in the first place, because those behaviors that govern those entities are entirely made up.
Conor Doherty: Again, we’ll come back in another talk about the value of decisions—we spoke on that recently with Adam Dejans Jr and Warren Powell. Anyone who wants more on that, check the most recent video here. Pushing on: another key blocker to adoption mentioned by your audience was change management. That’s something that typically comes up for basically any kind of technology. But when we’re talking about something—you used a mirror image—you described it as essentially, if it works well, you have a mirror image of what you already have. So the question is: why would a mirror image of a pre-existing state require extensive change management?
Joannes Vermorel: If you had—and I really challenge that the companies who are pushing these technologies do—but if you had something that is a high-dimensional predictor of the future state of your supply chain, which is inspirationally what those digital twins aspire to be, now the interesting thing is that if it’s done end-to-end you can—there are no silos anymore. You can literally simulate the effect of every single policy change to be done in every single silo, and then you have the rate of return for the company as a whole. It is very attractive. I don’t deny that. Lokad is very much on the same path.
What again I think—and thus it requires tons of change management—because if you have a technology that lets you bypass all the silos of the company, then it’s going to create challenges all over the place. Suddenly you will realize that the policy adopted by, let’s say, this division is actually detrimental to the company as a whole. Maybe it improves the KPIs of this one division, but it’s at the expense of the performance of the whole. So yes, the amount of change management and resistance can be expected naturally to be very high. That part, I think, is reasonable.
Conor Doherty: Joannes, we’ve been talking for about 35-ish minutes, I believe, and there actually are a number of questions to get to. I’m going to transition to the audience questions in a moment. Please get in your questions before we finish. But as a closing thought to this section—given everything that we’ve discussed—what is your parting advice to supply chain managers and executives who are either in the process of trying to adopt this technology or are considering adopting this technology?
Joannes Vermorel: You really need to have a serious look at what goes under the hood. My take is—just like “demand sensing,” there are other buzzwords that float in supply chain circles—where, I think in the case of demand sensing, there is nothing; in the case of digital twins, most of the time there is a little bit of something: it’s just a Monte Carlo simulator. But you really need to challenge what is under the hood.
Yes, people can build all sorts of nice user interfaces on that. They can have fancy graphics. If they put a map, you can have an animated map and whatnot. That’s selling a dream. You really need to check what is in this thing. Apply mechanical sympathy: you should demand of your vendor, “Please tell me why this thing is going to work.” And don’t confuse the intent. People say, “This optimizes that.” No, hold on. You’re optimizing the profit—optimizing that’s the outcome. I’m not asking you about the outcome. You’re selling me something very positive for the company, but tell me how it’s being done numerically.
Investigate. If at the end you see that it’s just hard-coded rules applied in sequence for this Monte Carlo simulator, then the emperor has no clothes. It is just very, very shallow.
Conor Doherty: To close on your own thoughts: I said at the start that this is a conversation you had before my time here. Back in August 2022 on this topic, you said, and I quote, “My perception on digital twins is that it’s just one of those buzzwords. It looks like nothing more than a glorified simulator.” So, in closing, in the last three years, do you stand by those words?
Joannes Vermorel: As it is an intent—“digital twin”—I have not seen any vendor who pushed something that was really of technological substance. I do not challenge the intent. If tomorrow someone comes and says, “I have a digital twin, but instead of doing just a super-naive, first-year computer science student Monte Carlo simulator, I have this amazing new technique, and I give you the blueprint of this amazing new technique; it’s very fancy, and it does things in ways that are way better than this naive Monte Carlo thing,” I would say, “Yes, absolutely, maybe it’s actually relevant.”
It’s just like if someone says “AGI is super good,” I would say, “Yes, AGI is super good. Do you have one?” So again, I do not challenge the intent associated with that. I really challenge the set of techniques that goes underneath. And for now, three years later, I’ve still not seen any vendor come up with techniques that I would consider remarkable.
Is there any vendor—and I would be very interested if the audience can push something—where they would say, “Joannes, have a look at those techniques that are truly remarkable; they are pushing the state of the art when it comes to stochastic simulations”? I would say I’m all ears. I have not seen that.
Conor Doherty: That’s an open challenge. If anyone has anything they want to share with Joannes, connect with him and do so. Anyway, Joannes, I will switch over to the audience questions. There’s a few to get through. I will read—some of them are quite long—so pay attention.
From Philip: let’s take the Suez Canal incident as an example. Suppose my model estimates a one-month lead time for shipping goods from Australia to France, accounting for normal traffic conditions. What if a ship blocks the canal unexpectedly like what happened before? The model wouldn’t be able to foresee that disruption, and I would suffer serious delays and hardship as a result. Question: how do we deal with such unpredictable events in supply chain modeling?
Joannes Vermorel: That’s an excellent question. Here, in fact, the stock—Monte Carlo simulator—does provide an insight that is not too bad, and that’s the same insight that Lokad is using. It is a programmatic approach. Monte Carlo simulator is fundamentally a programmatic paradigm: you implement, in a relatively general programming language, your rules that characterize the behaviors.
At Lokad, the way it’s done is that a big portion of the behaviors are learned on historical data. But because it’s a program, you can interfere with this program and introduce an interference, intentional and programmatic. Why is it needed? Because you need to make really sure that you are selectively bumping up the lead times—the projected lead times—for the suppliers who are going to suffer from the problem in this canal.
For example, if you have suppliers in Asia but they are shipping things by air, you don’t want to bump their lead time. So you have to be very careful, and you will have to maybe check in the historical data who were the suppliers that had the lead time that correspond to a boat shipment—or you have this information—and then you can programmatically add this extra nudge. I believe that having a programmatic approach here is a win. On the digital twin front, I believe they are approaching the problem from the correct perspective, which is through programmatic lenses, just like Lokad does. On this front, it’s good. The situation is actually cleaner compared to alternative non-programmatic approaches that just expect to have menus and buttons to cover every possible situation. If you have access to a programming language at the core of your model, you can always implement bespoke rules that correspond to unexpected events.
Conor Doherty: Thank you. I’ll push on— from Manuel: “In theory this technology, the digital twins, could have a major impact on decision support in supply chain. What do you think about the evolution of this technology and realization of its potential?” I think we covered it a bit.
Joannes Vermorel: As an inspirational direction, the idea of having an end-to-end modelization of the supply chain is a great idea. Lokad is also on this track. What are the key ingredients? There are a lot of key ingredients for that: differentiable programming is one of those; algebra of random variables is one of those; relational algebra also is one of those. You have tons of ingredients.
The idea of having Monte Carlo simulators—by the way, for example, Lokad has also Monte Carlo components—and there are some very interesting things to do. If you have random behaviors—intentionally random—part of this Monte Carlo process, it creates a very subtle problem when it comes to debugability. You run your program: it crashes—that’s bad. You run it again: it doesn’t—and that’s worse, because you have a bug that appears intermittently, and when you want to have a closer look at the bug, it’s gone. At Lokad, we call those problems “Heisenbugs.” It’s a bug that just appears intermittently; when you look closer, it disappears.
That’s a massive defect of the classic simplistic take on your Monte Carlo simulator. It’s something that finance encountered in the early ’90s. What you want is pseudo-randomness and, in fact, something that is completely deterministic, even if you have a Monte Carlo simulation going on. It requires relatively clever tricks. Also, you want this determinism to hold even if you’re doing distributed computing, so that the whole thing is run on many CPUs and possibly many machines—because, as I pointed out, if you’re single-CPU, single-threaded, you will very quickly run into performance issues. No problem: you can scale out across many CPUs and many machines. But that requires tools where, if you have those bugs or crashes or whatnot, the whole system remains completely deterministic.
My take is that there are a lot of things to ultimately engineer those digital twins that vendors can bring to the table. I hope Lokad is bringing quite a lot of those. What I’m saying—the key of my criticism—is that the people who advertise themselves with this etiquette, this buzzword, do not bring much substance to the table. It’s very, very shallow, and when you look at what is behind—ask for a demo—you just see a very simplistic, naive Monte Carlo simulation, which is just not going to deliver these sorts of things.
Conor Doherty: Perfect transition to the third question, thank you. From Shashank—forgive me if I’m mispronouncing this: “Joannes, what’s your view on Monte Carlo simulator versus agentic stochastic models across supply chain networks?”
Joannes Vermorel: You have several angles. Monte Carlo is a very useful tool; it’s a numerical trick. It is very useful—just like linear algebra, it’s a super basic, fundamental trick. In itself, it’s very useful. Now, in a Monte Carlo simulator all the intelligence lies in the behaviors that you’re simulating, because essentially a Monte Carlo simulator is: I have a million items; I take item one; I have my behavior; I apply my behavior; I have a deviate—a non-deterministic behavior—so I get the deviate at the output of atom one. Then I repeat for atom two, three, etc., and I repeat over every time period.
The fine print is those behaviors, and that’s really, really tricky. The plus side of the Monte Carlo numerical trick is that it fits quite nicely into the very quantitative, high-dimensional world of supply chain. If you’re talking about agentic AI—that would be LLMs, let’s say, to be specific—LLMs, on the other hand, deal with sequences of symbols. An LLM—a large language model—is a machine to predict future sequences of symbols, called tokens.
In terms of match with the data that you have, it’s really not that clear. An LLM is not exactly a super obvious match to your supply chain system. My take is that, in the future state of supply chain technologies, will Monte Carlo simulators be there 10 years from now? Yes, because it’s such a basic building block. It’s again like saying, “Will you have square roots in your systems?” Yes. Agentic AI like LLMs—I think they have their place, but their place might not be in the quantitative assessment of the supply chain. Their place is more at the periphery, where you have to interact with customers, suppliers, or potentially even third parties, and then you have text-based conversations. That’s a way to bring some data or complete some interactions, but it’s not exactly the core of the optimization itself.
Conor Doherty: Thank you. I’ll push on to Tao: “In your view, is the real issue with digital twins the fact that the tools currently used to simulate the supply chain are flawed, or is it that the right tool for supply chain optimization might not require a digital twin at all?”
Joannes Vermorel: It’s exactly the first reason. Those tools are flawed—and in a very specific way. This is a super shallow, easy-to-implement sort of tool. People have to think that in the world of enterprise software, there is a temptation: as soon as you’re closing with the board, with a CEO, the price tag is going to be very high—even if what you’re selling is essentially nothing. It may sound strange, but in enterprise software you will be selling something at $100,000 plus per year, pretty much no matter what you’re actually selling.
So there is this ongoing flux of people who come to the market and say, “Look, I have this thing, and now if I can create enough excitement for it, some companies will pay for it.” At their scale it’s a drop in the bucket. If you’re a billion-plus company and you pay $100,000 per year for some gadget, it’s not a big deal; it’s not going to endanger your profitability. But as a consequence, CEOs and boards—C-levels in general—they don’t necessarily have that much time to go into the weeds of what they are exactly buying, especially if the price point is not through the roof. For enterprise level, $100,000 is not through the roof. For this price point, the CEO of a billion-dollar company is not going to spend days to validate these things; that’s going to be a decision that must be taken very fast.
The consequence is that, unfortunately, you have a lot of actors who come up with very easy technology—super simplistic—but they have mastered the art of packaging. You have your Monte Carlo simulator—frankly, it’s nothing, it’s a nothing-burger—but it has a nice fancy packaging. Then you put, just like the definition we got, a definition that is, “Who wouldn’t want to have something that is going to optimize this and that, and improve this and that, for manufacturing and everything?” Yes. “What is the price of this widget that makes my whole company better?” “Such-and-such—$100,000.” “Okay, it’s super cheap. Let’s try it.”
But my message is: investigate what goes under the hood. Is it really substance? Packaging is not enough. What I see is a pattern: “digital twin” came with a packaging. Once you’ve seen another vendor with a package, replicating the packaging is straightforward, because the packaging is literally what you see from the outside. What is inside—if it’s just a Monte Carlo simulator—you can hire a software engineer and this thing will be implemented within days.
All those ingredients together create this temptation to go to market with the right package, prospect C-levels across the board, and if you’re lucky enough, you will land a series of deals—and then, bam, you have a business. I’m just saying that this is typically the sort of anti-patterns that I see in the realm of enterprise software.
Conor Doherty: Thank you. And Joannes, this is the last question—I feel I can guess the answer, but it’s from Murthy: “How can digital twins evolve from reactive monitoring tools to proactive decision-making agents in real-life enterprise ecosystems?”
Joannes Vermorel: First, in this definition and the technical layout that I have just presented, what is making the digital twin reactive rather than proactive? Conceptually, nothing. Why wouldn’t you use this simulator—super granular, end-to-end—to proactively challenge every single policy that you are about to carry for your supply chain? Obviously, this thing is meant to be proactive.
So, if the way it is used in practice is not proactive, then we have to ask why. On paper, a Monte Carlo simulator: you have something that is decently accurate and representative of your supply chain and its future state; you can—because it’s programmatic—inject any kind of policy in it. Fantastic. That means you can challenge everything; obviously it’s meant to be proactive.
But it isn’t. Why is that? Analyze from a software angle: what are they doing, how is it implemented? That will give you the answer. The answer is: we are back to one million SKUs, 10 CPU cycles, 100 days, etc. You end up with: yes, this thing can give you a simulation of what will happen, but unless you make substantial engineering efforts—which those vendors do not do—you will get one answer per, let’s say, 20 minutes, or even per one hour if it’s not super well implemented.
Suddenly you realize you have a problem, because how can you pilot anything forward when you have this sort of performance issue? I’m not even talking about the modeling accuracy problem—just basic performance in terms of compute resources that you need to inject. That’s a problem that is very real. That’s why people end up with a very reactive take, because they realize the whole thing is super slow. You are essentially trying to ask millions of questions to your system, but if you need to wait 20 minutes for every single answer, it’s incredible—and you have to ask those one million questions every day.
You end up with very basic problems. If you re-aggregate the data, then suddenly you cannot ask questions that really matter, because you are at an aggregation level that does not reflect the decisions that you have to take anymore, such as: “Do I need to produce this? Do I need to move the inventory there? Should I raise the price of these products?” Suddenly, if you’re at an aggregated level—for example, if you want to reason about the pricing of a product category—it doesn’t really make sense to say, “I’m going to increase the price or decrease the price for all the products in this category.” You probably want to differentiate based on the characteristics of every single product. But again, that goes against the idea of aggregating everything.
Conor Doherty: As a quick plug, I know you cover extensively the concept of decision optionality inherent to the flow of physical goods—your definition of supply chain. That’s covered, I think, in Lecture 1.2, “Supply Chain in a Nutshell.”
That’s literally the first—okay. Well, watch the first two if nothing else, because then you get to “Quantitative Supply Chain in a Nutshell.” That’s pretty good—better than pretty good. Anyway, Joannes, we’ve been going for an hour. We’re out of questions, and now we’re out of time. As always, thank you very much for joining me and for your answers.
And to everyone else, thank you for attending. Thank you for your questions. If you don’t already, follow Lokad on LinkedIn, please. And if you aren’t connected with us, send us a connection request. We’re always happy to discuss supply chain issues. And on that note, to everyone, I say: get back to work.
