Probabilistic Forecasts & Sequential Decision-Making (with Warren Powell) - Ep 163

00:00:00 Introduction of guests
00:01:27 Warren’s role at Optimal Dynamics and book
00:03:06 Challenges and forecasting in truckload trucking
00:04:31 Warren’s paper and understanding of uncertainty
00:06:41 Joannes’ journey and success in software
00:08:12 Embracing uncertainty and academic perspective on supply chain
00:09:39 Sequential decisions and difficulty of industry collaboration
00:11:13 Probabilistic vs deterministic forecasting
00:13:27 Metrics for probabilistic forecasts and understanding difficulties
00:15:28 Importance of forecast coverage and sharp forecasts
00:17:09 Lokad’s journey with probabilistic forecasts and challenges
00:19:15 Difficulty of uncertainty problems and lack of unified community
00:21:05 Decisions and uncertainty in math programming
00:23:03 Diverse lab experiences and ADP book applications
00:25:07 Need for toolbox in supply chain and transition to new approach
00:27:35 Deterministic optimization and cost function approximation
00:29:57 Google Maps as a look ahead example
00:31:51 Stochastic look ahead and value function approximations
00:33:21 Joannes’ perspective on decision-making and problem statement
00:35:29 Importance of correct state, transition function, and cost function
00:37:28 Considering problem dimensionality and large-scale problems
00:39:15 Fragility of solutions and network of supermarkets comparison
00:41:16 Approach to problem-solving and writing a supply chain analytics book
00:43:44 Simplicity in model design and different types of uncertainty
00:45:13 Importance of swift iteration and real-world constraints
00:47:51 Importance of computational tractability and development of Graphics tool
00:50:27 Role of instrumentation in optimization and industry challenges
00:53:11 Working with carriers and critique of academic approach
00:55:23 Joannes’ journey from forecasting to decision-making
00:58:28 Difficulty of data access and dealing with future forecasting
01:00:32 Importance of considering major disruptions and accepting pessimistic models
01:02:45 Variability in supply chain and fashion industry
01:04:43 Renewing products annually and forecasting new products
01:06:27 Importance of modeling uncertainty and critique of corporate rules
01:08:16 Warren’s approach to stochastic optimization and managing inventory
01:09:56 Planning for contingencies and decision-making under uncertainty
01:11:39 Importance of real-time dispatching in trucking and right load selection
01:13:27 Challenges in decision making under uncertainty and issues with educated individuals
01:15:45 Excel’s limitations in dealing with uncertainty and CEOs’ understanding
01:19:30 Limitations of supply chain books and importance of user-friendly tools
01:21:46 Lokad’s educational initiatives and creating relevant data sets
01:25:01 Three essential questions for problem-solving and developing decision categories
01:27:48 Non-quantitative MBA’s challenge and companies burying decisions under workflows
01:30:26 Price of simplicity and sequential learning as a decision-making tool
01:32:33 Teaching the concept of aiming higher and challenges of rigid policies
01:34:34 Difficulty of comprehending exploration concept and importance of active learning
01:37:41 Differences between trucking and supply chain and size of truckload business
01:40:04 Book’s title, purpose, teaching style, and five elements of modeling
01:42:39 Praise for Optimal Dynamics, Lokad, and sharing academic ideas
01:43:22 Closing remarks and thanks

About the guest

Warren B Powell is Professor Emeritus at Princeton University, where he taught for 39 years, and is currently the Chief Innovation Officer at Optimal Dynamics. He was the founder and director of CASTLE Lab, which focused on stochastic optimization with applications to freight transportation, energy systems, health, e-commerce, finance and the laboratory sciences, supported by over $50 million in funding from government and industry. He has pioneered a new universal framework that can be used to model any sequential decision problem, including the identification of four classes of policies that spans every possible method for making decisions. This is documented in his latest book with John Wiley: Reinforcement Learning and Stochastic Optimization: A unified framework for sequential decisions. He published over 250 papers, five books, and produced over 60 graduate students and post-docs. He is the 2021 recipient of the Robert Herman Lifetime Achievement Award from the Society for Transportation Science and Logistics, the 2022 Saul Gass Expository Writing Award. He is a fellow of Informs, and the recipient of numerous other awards.

Summary

In a recent LokadTV interview, Conor Doherty, Joannes Vermorel, and guest Warren Powell discussed probabilistic forecasts and decision making in supply chains. Warren Powell, a retired Princeton professor and Chief Innovation Officer at Optimal Dynamics, shared his career journey and insights on planning under uncertainty. Joannes Vermorel, Lokad’s CEO, discussed his transition from deterministic methods to probabilistic forecasting, criticizing academia’s lack of real-world application. Both agreed on the superiority of probabilistic forecasting, despite its complexity and businesses’ struggle to apply it. The conversation highlighted the need for a broader perspective and a unified approach to dealing with uncertainty in decision-making.

Extended Summary

In a recent interview hosted by Conor Doherty, Head of Communication at Lokad, Warren Powell, a retired professor from Princeton University and Chief Innovation Officer at Optimal Dynamics, and Joannes Vermorel, CEO and founder of Lokad, engaged in a thought-provoking discussion on probabilistic forecasts and sequential decision making in supply chain under the presence of uncertainty.

Warren Powell, a seasoned veteran in the field of decision making across complex fields, began by sharing his career journey. His work started with the deregulation of freight transportation in the United States, which led him to focus on planning under uncertainty. He also discussed his role at Optimal Dynamics, a startup he works with, where he guides his former PhD students and contemplates new directions for the company.

The conversation then shifted to Powell’s book, “Reinforcement Learning and Stochastic Optimization,” which delves into the realm of distributional or probabilistic forecasting. Powell shared an anecdote about a company that wanted to understand the value of offering a shipper a discount if they could predict future loads. This sparked his interest in the topic and led him to explore the challenges of forecasting in truckload trucking due to its stochastic nature.

Joannes Vermorel, on the other hand, shared his journey from deterministic methods to probabilistic forecasting. He discussed his realization that deterministic methods were not working and the need to embrace uncertainty in supply chain problems. He also criticized the academic community for its lack of real-world application and its focus on proving theorems and running numerical work.

The discussion then turned to the difference between deterministic and probabilistic forecasting. Powell explained that while deterministic forecasting provides a single, actionable number, it fails to account for real-world variability. He argued that distributional forecasting, which provides a range of possible outcomes, is superior, although businesses often struggle to understand and apply this concept.

Vermorel agreed with Powell, adding that probabilistic forecasting requires more complex metrics and a deeper understanding of probability distributions. He likened deterministic forecasting to looking at a tiny, detailed section of a desk through a microscope, while probabilistic forecasting provides a broader, more complete view.

The conversation concluded with Vermorel sharing his experience implementing probabilistic forecasting at Lokad. He noted that it took several years to figure out how to optimize decisions based on these forecasts. He also discussed the lack of a unified community or paradigm for dealing with uncertainty in decision-making. Powell agreed, describing the field of decisions and uncertainty as a “jungle” due to the variety of different communities, languages, and notational systems. He shared his diverse experiences in various fields, from freight transportation to energy systems, and how these experiences led him to realize the limitations of certain approaches and the need for a broader perspective.

Full Transcript

Conor Doherty: Welcome back. Identifying and evaluating viable supply chain decisions is difficult, particularly if you still use traditional metrics. Warren Powell, today’s guest, has spent 40 years analyzing decision making across various complex fields. As well as that, he’s written five books, about 250 papers, and is a retired Professor from Princeton. So, Warren, first of all, welcome back to Lokad. Secondly, for anyone who might have missed your first appearance, could you please reintroduce yourself and give everyone a flavor for what you’ve been up to?

Warren Powell: Well, thanks for inviting me back. I’ve had an interesting career. My career started when freight transportation in the United States was deregulated, and so I got thrown into this industry called truckload trucking. One of the first issues they talked about was uncertainty and how do you plan under uncertainty, and that pretty much ended up defining my career. I went around a number of various applications.

I have ended my career helping my startup, Optimal Dynamics, in truckload trucking, which is what started my career. We’re using a variety of techniques, but fortunately, I’ve been able to work on enough different applications to realize there’s more than one tool in this toolbox of uncertainty. So, I’m looking forward to this discussion. It’s nice to talk to other people who share my passion for modeling uncertainty.

Conor Doherty: Thank you. And you mentioned Optimal Dynamics. You’re the Chief Innovations Officer, the CIO. I’d never heard that term before. Could you explain what it is you do there?

Warren Powell: They like to call me Yoda. I’m not involved in any of the management. Nobody works for me. I have five of my former PhD students working there, and I pretty much work with them the way I did when I was a professor in the lab. I wait for them to put up their hands and say, “Hey, we need some help.” Otherwise, I spend my time thinking about things and also thinking about new directions for the company if the opportunity arises.

But every now and then, I do get pulled back in to help with a problem, and I’ve come up with a few new innovations that have helped out. But I’m really here to help the company when they need help, and otherwise, I like to stay out of their way. I’ve learned that as an academic, one of the biggest challenges, especially when you’re working with bright people, is to know when to help and when to stay out of their way. And so, fortunately, that’s given me a lot of time for book writing and things like that.

Conor Doherty: Actually, mentioning book writing, one of your books, “Reinforcement Learning and Stochastic Optimization,” is one of the things that we were most interested in talking to you about. Your approach to decision-making and I know you have an interest in the distributional or probabilistic forecasting approach that Lokad does. So, to get the interview started properly, what is it about distributional forecasting that fascinates you so much and led to this conversation today?

Warren Powell: Well, the biggest challenge when I turned to modeling my truckload problem, so in trucking, truckload trucking is very sparse. You might have a load going between a pair of cities, or you might not. When you send a driver, say from Chicago to Atlanta, when you arrive at Atlanta, there’s loads going in very different directions. You may have a load going to Texas, or you may not. So, you have something that’s 0 or 1. What do you forecast? Do you do zero or one, or do you forecast a 0.2, which is the more realistic expectation?

I had a company here in the United States, Schneider National, that back in the 1970s saw that deregulation was coming in, and they worked with a faculty member at the University of Cincinnati on building early optimization models, but they were all deterministic. And somebody from Schneider visited me at Princeton and looked at me and said, and this was somebody with a masters in operations research, “Warren, truckload trucking is stochastic.

We don’t know what loads will be available even tomorrow. We’d love to know what would be the value of giving a shipper a discount if he would tell us the loads in the future.” And I remember sitting at that dinner conversation thinking, “Oh my gosh, what a great question.” It’s not that I don’t know the answer, it’s that I don’t know how to think about it.

Later in the 1980s, I wrote a paper that I call my museum paper. In fact, it’s on the internet as the museum paper. I have five different ways of modeling these truckload trucking problems, all dealing with uncertainty in a different way, and I was fully aware none of them would work. And so there I was, late 1980s, going, “I don’t know what else to do. Nothing coming out of the academic community is working.”

So that started this multi-decade process of where I would sort of get a handle and I’d have these aha moments. And so I had a big one early 2000. Schneider actually came to me and says, “Hey, Warren, we really need help. Could you build us this model?” That model ended up being the foundational software for Optimal Dynamics. But even since that model was built, which could handle uncertainty, that was when my work in approximate dynamic programming came out.

I would say that every few years, I would have another one of these big aha moments. In fact, even since I graduated, I’ve had a few more aha moments. I mean, this field is just stunningly rich, and I keep having this, “Oh my gosh, I never thought of it that way,” type of moments.

Conor Doherty: Joannes, does that track with how you got to probabilistic forecasting? A lot of aha moments?

Joannes Vermorel: Yes, I mean, kind of. It was for me a slightly different journey because when I started Lokad in 2008, I actually went straight with the mainstream supply chain theories. So, it wasn’t so much as somebody walking to me and even pronouncing the word stochastic. I’m pretty sure that most of the people I had met until very late in my life, if I had pronounced the word stochastic, they would not have been sure if I was talking about something like a variant of elastic or something.

But anyway, they were smart, but they were not statisticians or probabilists or whatever. And so my path was more like during the first few years of Lokad, I actually applied those deterministic methods with quite success as an enterprise software vendor, which means that you manage to actually sell your stuff. It doesn’t mean that it actually works, you know, that’s two different metrics. You can succeed as an enterprise vendor and still have nothing that actually works.

There have been competitors who made their entire career of doing just that. But so, it took me actually a few years to realize that it just wasn’t working, and it would never be working. That the mainstream perspective, the supply chain perspective, which is built on this fully deterministic perspective that there is no uncertainty, that success wasn’t around the corner. It wasn’t about getting yet this extra 1% of forecast accuracy that suddenly would make it work.

No, it took me quite a few years, more like four years, to kind of give up on the idea that despite making progress on forecasting, despite improving the process, improving everything, that no, success wasn’t around the corner. And so we had this aha moment, but it was more out of desperation rather than as a consequence of a great conversation with somebody who was kind of enlightening. So, anyway, we came to that a bit. It took time. But I would say fast forward a decade now, it is now painfully obvious. I would say completely up my first few years at Lokad were to try to address supply chain problems without embracing uncertainty. That was just such a dead end, and well, it took me a few years to get there.

Warren Powell: What are the challenges I found, if I can, so coming from the academic side, so Joannes, when I speak to you, I almost feel like I’m speaking to a fellow academic, but you’re coming from the industry side. My lab was unusual from day one. I had to go out and pound the street and talk to companies and get money. The National Science Foundation, which funds a lot of academics, they had an explicit policy in my field. They said, “We do not fund research. We bless it. Go get money from industry, and then we’ll sprinkle NSF angel dust.”

But we have way too many academics, and it still persists today, where they’re not working with industry, so they work with made-up models, and they prove their theorems, they run their numerical work, and it’s all entirely within the academic community. And this is particularly true of stochastic optimization. It’s not that true with machine learning. Machine learners go out, they get real data sets, they fit models.

It’s not even true with deterministic optimization. There’s no shortage of real-world deterministic optimization. But what I like to now call sequential decisions, that, by the way, gets me away from that word stochastic, there’s just something about that field where there’s this ocean of papers of models that have been made up by academics where they don’t really understand what the real problems are because they’re hard to work with industry, and you have to get companies, and I’ve had companies. They had to work on what they later called the bleeding edge, where they had to be the company where I learned what did and didn’t work.

So it’s really a problem with how academics work. One thing, I had a successful publishing career, but boy, toward the end, I was like, “You know, it’s a bit of a game.” You know, to get published, you have to follow a certain style that the journals want, and the stochastic optimization community is not one community. It’s over a dozen. They all have their own languages and styles and little tools and techniques, and they’re all very proud of it, and they prove their theorems, and they even run numbers, but almost none of it is working in practice.

Conor Doherty: Well, thank you. To underline the point there, the difference between a purely academic approach and a more practical one, we were talking about the deterministic approach to forecasting versus distributional or probabilistic. I’m just going to use the term probabilistic for ease. Warren, to you first, to sketch out for people who might be hearing this for the first time again, this dichotomy. What is the difference from your perspective in terms of the deterministic approach to forecasting versus the probabilistic and why is the probabilistic, let’s say, superior in your opinion?

Warren Powell: Okay, so anytime I meet anybody from business who uses the word forecast, I immediately say okay, he means point forecast. Everybody loves the point forecast. They want to know, “I’m going to sell 500 widgets or two cars or there’ll be six truckloads of freight.” They love that number because it’s actionable. It says, “Oh, there’s going to be six trucks so I have to have six drivers.”

The challenge is, and by the way, this happens every day in truckload trucking. You’ll have a hot shipper, but he knows he’s one of your top shippers and he will call in and, to quote the words of one dispatcher, says, “Look, this guy can call on needing anywhere from 10 to 20 trucks.” Well, that’s pretty frustrating, but that’s the real world in dispatching. But in the forecasting models, all the mathematics is designed to come up with a single number.

People also like a single number. It’s actionable, it’s easy to understand. If you say, “Look, it’ll be somewhere between 10 and 20,” you know, how many drivers am I supposed to have to meet a demand that’s somewhere between 10 and 20? Well, I will tell you what the truckers do. They’ll say, “Well, that’s a really important trucker. Maybe I won’t have 20 drivers, but maybe I’ll have 17. But if he comes in and he only needs 12, then I’ll take those five drivers and send them off somewhere else.” And they’ll have what’s known in optimization as a recourse. It’s like, “Well, if this happens, then this is what I’ll do.”

But everybody loves that point forecast. I first started doing distributional forecasting in the 1990s and I was working with Yellow Freight. I said, “Look, I’d love to do confidence intervals,” and they came back and said, “Our guys just don’t know how to deal with that.” Our biggest problem, not too long ago, we were working with a major shipper and they got really excited about distributional forecasting and then they turned around and said, “Yeah, well let’s take it and see how accurate it is.” I see Joannes smiling. It’s like, “Okay, so how do you deal with the, ‘Oh, that’s great, distributional forecasting, that sounds cool. How accurate is it?’ How do you answer that question, Joannes?”

Joannes Vermorel: Yeah, I mean, with something like cross entropy or any other metric that works for probabilistic forecast, CRPS is another one. But indeed, that’s the case. When you enter the realm of those probability distributions, you still have metrics, but they are not like those easy, intuitive metrics that you can literally take children from junior high school and they would get it. The norm one, norm two, junior high school, you kind of get it. What is the distance?

When you go into probability distributions, to be fair, it’s not that difficult. It’s not especially if you go for, let’s say, maximum likelihood or something. It’s not something where you need to have a PhD in statistics to get it, but it will take more than 2 minutes. And then intuition, you would probably need to go through the motion of the formalism and that will take something like half an hour, maybe two hours if you’re very ignorant.

Warren Powell: Yes, and at that point, the business people, their eyes are glazing over and they’re going, “Oh yeah, I got that. So how accurate is it?”

Joannes Vermorel: That’s something that is very strange. This is about having a richer forecast. When we have these decisions that we want to optimize, it’s about improving the kind of your depth of vision. What do you see? I mean, you’re making a projection about the future, a statement about the future. But how not accurate, but how complete, how much coverage do you have into your forecast?

So it’s something that is very unusual because people would say with point forecast, they would have something that is like incredibly sharp. It’s a bit like if you take a microscope and you just zoom a thousand times on a point of your desk. So you can have your microscope and then you look one square millimeter on your desk and you have perfect vision, but the rest of your desk, you don’t see anything. And people say, “Oh, you know what? I think I need a bigger microscope so that I can look at this one square millimeter even more sharply.” And probabilistic forecast is, “No, you should probably have a look at the rest of the desk rather than concentrate on this one point that you already see quite sharply compared to everything else.”

Warren Powell: Now here’s something that any business person, especially in say retail sales, will absolutely understand is demand coverage. And they’ll say, “Look, we want to satisfy 97% of demand.” Now that’s not an unusual request. Now how do you satisfy 97% without the concept of a distributional forecast? So this is where you can come back and go, “Yes, but you want to cover 97% of demand. I can’t do that until I have a distributional forecast. Do I need 20 extra units or 200 extra units?” So this is perhaps the lead in to say, “Look, you guys want to cover a high percentage of your demand. I mean, nobody wants to cover the average demand. You’ll be short half the time. So somehow we’ve got to learn how to bring that very familiar business requirement into the, ‘Well, if you want this, then we have to do distributional forecasting or probabilistic forecasting.’”

Joannes Vermorel: And the interesting thing is that once at Lokad we started doing that in 2012, and as a segue to your book, it took us actually a few years after we started to do probabilistic forecast until we really managed to figure out how to do any kind of sophisticated optimization on top of it. Because you see, I would say, probabilistic forecast was difficult to come to terms with the fact that we needed to do that. So that was the first part of my journey at Lokad.

It turned out that in 2012, probabilistic forecast had become quite popular for completely different reasons in deep learning. They were very popular in deep learning because the cross-entropy metrics like that give you very steep gradients that help for the optimization. So the Deep Learning Community was using those probabilistic forecasts, although they were absolutely not interested in the probabilities. They were only interested in the point forecast, but the super steep gradients that you could get with essentially cross entropy were very nice numerical properties to make those models work.

So that was like, okay, a little bit of a deviation. We started to use those probabilistic forecasts for their own sake as opposed to just being nifty numerical tricks for gradients. But then once you have that, you realize that, okay, I have decisions that I want to optimize. I want to take the best option and obviously there will be that’s a repeat business so there is like this sequence of decision.

And then you end up with, “What do I need as a software instrument to just solve that?” And that’s where, as a segue to your book, it’s a very difficult problem because the main challenge that I faced was even a semi-vacuum in terms of paradigms. There is not, as you said, you had like half a dozen of communities where you could publish, but there is, to my sense, even today, there is not yet a really unified community that just takes those problems with uncertainty where you want to do optimization and just run with it. It’s just not there.

So it was like a hit or miss. I have had done a little bit of reinforcement learning, had done classical optimization. My challenge was really those lack of paradigms. And that’s quite interesting in this very heavy, you know, 1100 pages, is that you actually go and propose your own paradigms to just think the domain through and just slice and dice the domain. And yeah, I mean, it’s still, you know, this book is still kind of one of a kind. There is not that many.

I mean, if you want to have a book about, let’s say, classifiers, there is, you know, for machine learning, there will be 500 books that give you all the classics from classifiers, from linear classification to support vector machines and gradient booster trees and whatnot. There is like 500 books that frame the problem of classification and whatnot. Here, it’s still very much, I would say, it’s still something that, well, sorry for my very long answer, where the community is still just not looking at the problem yet.

Warren Powell: Yeah, the issue with decisions and uncertainty is an astonishingly rich field. If you go to deterministic math programming, yes, there’s a lot of deterministic math programs, but they all follow the fundamental paradigm that was set up with George Dantzig. You have an objective function, you have a constraint, you have a decision variable, you have an algorithm. Okay, and so because everybody fits within that framework, machine learning, statistics, machine learning, again, very much, you got some function, you’re trying to fit it to data.

Now, there’s many different problem sets, but because they’re all basically in that same approach, here’s one of a family of functions. And so most of the popular statistics books will expose you to all the different functions. And so you go and take a course in statistics or machine learning, pretty much everybody walks out with roughly the same tool set. And that also allows them to use these public domain software.

Once you mix decisions and uncertainty, back in 2014, I gave this talk, a tutorial at INFORMS, and I called it “The Clearing the Jungle of Stochastic Optimization.” And I had to write a tutorial article. I always remember one of the referee reports. The referee says, “Oh, it’s not so bad. Maybe you should call it ‘The Garden of Stochastic Optimization.’” And I cracked up and I said, “You never tried to publish a paper in these fields. It’s a jungle because you have all these different communities, well over a dozen, and they speak different languages. I counted eight fundamentally different notational systems. And then of course, you get the spin-offs.

So reinforcement learning adopted the notation of Markov decision processes, but stochastic control has their notation and stochastic programming has their notation, the decision trees. And it’s just a mess. But each one has a fairly substantial community. So they have their group of people that all speak the same language. And when you write papers, they expect certain things.

I ran a lab that was big enough and diverse enough. So while I started and ended in freight transportation, in the middle, I ran a whole energy systems lab. I did a whole pile of work on optimal learning and material science. That was a neat experience for a while. E-commerce, finance, and I have, at Princeton, you have to do undergrad senior thesis. So I supervised about 200 theses. And I’ll tell you, when you supervise enough students and do a wide enough of problems, back in the time when I had just written my ADP book and I thought, “Wow, ADP is great. Look, I can optimize trucking companies. And this isn’t fake. This is for real. There was a time when I would say, “But this is not just some joke application. This is a real industrial application. It must be able to do everything.” Boy, was I wrong.

Okay, it was the second edition of my ADP book that I did write a chapter, chapter six, that said, “You know what, there seem to be these four classes of policy.” Now I didn’t have all four. I had three of them. The fourth one, I got wrong. And six months after I sent it to the publisher, I had this, “Oh my God, I figured out the fourth class of policy.” And then from there up until, well, the big book in 2022 came out, I kept evolving, and I wrote another tutorial article, 2016, and then the big thing was European Journal of Operational Research invited me to write a review article.

That, uh, Roman Slowinski, one of the lead editors, uh, invited me to do this, and that was, that paper ended up being the outline for this big book. As soon as I had done that paper, I’m like, “Okay, that’s the new book”, and I was wanted to do a third edition of my ADP book, and what I say is, “No, I just can’t, ADP, specifically meaning approximating value function, it’s a very powerful tool for a very small number of problems, and if you have a hammer, and you’ve got your favorite hammer, and we all have our favorite hammers, everybody in academia has our favorite hammers, you can find problems that fit your hammer.

But if you come from an application area, take a rich application area like supply chain management, well, you’re going to need a toolbox. You can’t go into supply chain management with some hammer, and I don’t care what hammer you have, you’re going to have to go in with a full toolbox, because doing distributional forecasting is nice, but at the end of the day, you do have to make a decision, which means you’re making a decision under uncertainty.

Conor Doherty: Well, if I can just follow up there because perfect segue. When you talk about the toolbox, and you talk about the emphasis on, on probabilistic forecasting, providing something that is actionable, and people wanting something actionable, well then, could you please unpack the toolbox and explain, in the supply chain management context, how your universal framework for sequential decision-making actually leads to better decisions?

Warren Powell: Well, one of my biggest transitions, I have this PowerPoint slide that I love. I got about 15 books on it, all of them dealing with some sort of sequential decision problem. Every one of those books, and one of them is my ADP book, is a hammer looking for a nail. We all have our pet technique for making decisions, so the books were written around one or two fundamental hammers.

If you come from applications, you’ll start to realize that all of these hammers are good. None of them work on all problems. When you actually come from an application side, uh, you know, you can’t pick your problem. So, academics working on methods, we pick what problem to test our methods on. When you come from an application, you can’t. You’re told, “Here’s the problem. I have to solve. What are you going to do?”, and my great career accomplishment was to realize that all of the methods fall into these four classes, and then by the 2019 paper, I realized that those four classes fell into two major categories.

The simpler category makes a decision based on some function that is not planning into the future, but it has tunable parameters that you have to tune so that they work well into the future. Simplest example in supply chains is inventory ordering. When the inventory goes so below some level, you order up to another level. I’m not looking to the future. I’m not planning. It’s just a rule, but those order up to levels have to be tuned so that they work well over time.

The second of the simplest ones is typically, it’s almost always a deterministic optimization model, simplified, but with tunable parameters. Now, this is something that I gave the name “cost function approximation”. You won’t find it anywhere other than my big book, but widely used in industry. People from industry would say, “Yeah, we do that all the time. We just thought it was an industry hack”.

I realize that if you take a linear program that’s an approximation of some messy stochastic problem and then you put in some tunable parameters to capture things like buffer stocks or corrections for yield or a slack, you know, airlines will do this, will just say, “Okay, when I’m flying from Atlanta to New York, there might be a weather delay. I’ll throw in an extra 20 minutes”.

Doing these deterministic problems with tunable parameters is exceptionally powerful. The academics love to put it down as it, “Oh, just some deterministic nonsense”. I decided it’s really not unlike parametric forecasting. So, when you’re doing forecasting, you want to know, demand is a function of price. Well, a higher price will be lower demand. Let’s make up some downward sloping function, maybe just a line, maybe an S-curve, and then let’s fit the best function I can. We can do the same thing with parameterized deterministic models.

Now, the academics love the other class of policies. These are the policies that make a decision now by planning into the future. One is, if I make a decision now, now I take the action. Let’s say I have a certain amount of inventory. I order more inventory. That puts me in the state in the future, and I get the value of being in that state. That’s what they call dynamic programming or Bellman’s equation. Academics love Bellman’s equation, or if you come from engineering, they call it Hamilton-Jacobi equation. And if you go into any course in any good university that’s teaching how to make decisions over time under uncertainty, the first thing they’re going to show you is Bellman’s equation.

I wrote a 500-page book all built around approximating Bellman’s equation. I was very proud of it. It’s a powerful technique that works on this many problems. Okay, I mean, honestly, go out into a business community where people making sequential decisions say, “How many have even heard of Bellman’s equation?”, and almost none. Nobody uses Bellman’s equation.

The very last class is the full-blown look ahead. I use as my example Google Maps. If you want to plan a path to a destination, you have to plan all the way out. There’s a number of planning models that need to plan into the future. They don’t use a value function. They explicitly do a whole model into the future, and that’s done way more often than stepping out and doing a value function approximation.

So, the academics really love those more advanced techniques. When you go out into the real world, you’re mostly going to find one of three classes of policies: the simple rules like “order up to,” “buy low, sell high,” or “wear a coat.” For the more complex problems, I do a deterministic model that’s not too complicated. I throw in some tunable parameters and then tune the parameters. The third technique is the deterministic look-ahead, like Google Maps. Those are the big three.

I think that if you could actually make a list of all decisions everybody makes anywhere in any setting, 97% of those decisions would be made with those three classes of policies. Guess what? They are not emphasized in the books. This is where I’m heading. A lot of this thinking I’m going to attribute to you; you won’t find that particular discussion even in my big book, so that’s going to have to wait for the second edition.

This is my aha moment when I said, you know, let’s take the four classes of policies, let’s take the look-ahead, and split it into two: deterministic look-ahead and stochastic look-ahead. Now I’ve got five policies, and I said which of these are used most? Category one, widely used, are those first three: the policy function approximations like simple rules, the cost function approximations which are parameterized deterministic, and deterministic look-ahead. Those are the big three.

Now, there are times we need a stochastic look-ahead. For example, I’m ordering from China; it normally takes five weeks, but it could take seven. Well, if you go in and say, well, I’ll plan on seven, that’s actually a form of stochastic look-ahead called robust optimization using a probabilistic forecast. Because I planned on the most it might be rather than what it typically is.

Value function approximations, the topic of my earlier book, that’s down there at the bottom. I honestly think that’s a great tool for many problems. If you really need that, you better call an expert, but for day-to-day stuff, you’re just not going to end up using it. It’s way too hard to use.

Now, some people will talk about reinforcement learning. Reinforcement learning in the early days was just another name for approximate dynamic programming. They were just different words for the same thing. The ORL community has discovered just what I discovered; they found, wow, this doesn’t always work. If you go from the first edition of Sutton and Barto’s book, where all you’ll see is approximate dynamic programming, and go to the second edition, if you know what you’re looking for, you can find all four classes of policies in their second edition. But I still think most people, when they say they’re using reinforcement learning, mean approximate dynamic programming. The computer scientists are so much better than the rest of us at marketing their tools.

Conor Doherty: Well, thank you. I’m going to immediately throw it to you, Joannes. Does that align with your take on how we make decisions here at Lokad?

Joannes Vermorel: I mean, not quite, but to be fair, this split is, I believe, very technically correct in how you slice and dice the domain. I won’t challenge that point. And when I say the domain, I mean having the sort of agreed-upon intellectual model where we have state transition functions and a reward function, and then we want to optimize those decisions etc. So here, if we take it from this perspective, I would say yes, the way you describe it is correct.

But the way I personally approach the problem is through relatively different angles. My first perspective, even before considering the list of techniques, would be: what about the problem statement itself? The journey to the problem statement is crucial. It’s a little bit of unfair criticism of this book because it’s already 1100 pages long, and apparently, your publishers didn’t want a 3000-page book.

At Lokad, when we approach that, the first question we’d ask is how much we should approximate the state. People might think it’s a given, but it’s not a given. You are always modeling the real world, and you don’t model it to the position of every atom, so there’s a huge amount of leeway in what you decide is a state. Then you have the transition function, which again has a huge amount of leeway in deciding how you can go from one state to another version of this state.

I believe that it is part of solving the problem. If you make the wrong call at this stage, if you have a state that is way too granular or a transition function that is way too complex, your tooling will just fall apart afterward. So the first thing, for me, is getting the right call and having the right paradigm to do that. The same goes for the cost function or the reward function.

We have a classic case for clients that need to assess the cost of a stockout or the cost of giving a discount. If you give a discount once, you are giving away a portion of your margin. That’s okay, you can measure it, it’s fairly straightforward. But then you’re also creating a bad habit because people will expect this discount to be applied again. So you’re setting up a problem for yourself later.

This is very difficult to assess exactly the behavior, how much people will remember your previous discounts, and whatnot. That’s your transition function; you have to approximate that. My first thing would be the approximations before approximating the various cogs of the algorithmic process with this framework.

My perspective starts earlier with the very definition of the models. I don’t take the perspective that the model in general, what you want to optimize, is a given. For me, it’s part of the methodology. That would be the first thing. Sorry, I’m not done. The second thing is, but that’s more my computer science background, is to look at the dimension of the problem.

It is very different if you’re attacking a small problem, like a few thousand decisions such as a route in a city with a few hundred deliveries and whatnot. A thousand decisions, as far as I’m concerned, is a very small problem. We have problems where we are up to a billion variables. If we’re looking at a large supply chain, like a hypermarket, you can have in one hypermarket 100,000 SKUs. If you have a thousand hypermarkets, that gives you 100 million SKUs. For every SKU, you have half a dozen decisions, and you repeat that a few weeks into the future. You can end up with problems that are either super small, like route optimization, or very large, where they would not even fit in memory.

So for me, that would be the sort of thing where if I have to approach the problem, I would start to try to capture the key characteristics of the problem. Dimensionality is one of them. Another one, which is very important, will be how tough it is to navigate toward the better solution. If I take those two examples, route optimization is something very nonlinear, very fragile. You just move, swap two locations, and you can go from a super poor solution to a super good solution by just swapping two points. So your solution behaves a little bit like a crystal; it has this kind of fragility. It’s very easy to disrupt the solution and go from something good to something very terrible.

On the other hand, if I go to this spectrum of problems with my network of supermarkets, if I decide to put one unit that was supposed to be there somewhere else, the problem is very insensitive. You can go with a lot of leeway. You want to have something that is a lot more directionally correct. This spectrum goes from crystal properties to mud properties. Crystal properties are fragile and brittle, and they break easily, whereas mud properties are shapeless. As long as you’re directionally correct, it’s kind of okay. That would be the second consideration.

The third one would be the time characteristic you’re looking for. Time characteristics go from piloting robots in a warehouse, where you want constant time, constant memory answers within milliseconds. When people say you have 10 milliseconds to give an answer, if you don’t, we have all sorts of other problems, as opposed to another problem like purchasing overseas, where it’s going to take 10 weeks to come from China. If your calculation takes 24 hours, it’s no big deal. We can afford those 24 hours; we have no constraints whatsoever.

So that’s a little bit how I slice and dice the domain. I understand that the way I slice and dice the domain doesn’t say much about the algorithms that you want to use, but I use that as a way to eliminate what I will even consider as potential solutions for the sort of problems that I am interested in.

Warren Powell: I love that you’re coming from an application domain. One of the things I found when I started writing my supply chain analytics book, which is the first book I’ve ever written around a problem class, is that all of my other books have been just fundamentally methods books, and it’s a lot of fun.

Now, give my big book credit for one thing: I have one entire chapter, 90 pages long, dedicated to modeling but modeling in a very generic way. I absolutely appreciate your whole process you were describing. This is a state variable. In complex problems, I have some tutorials where I say we have five elements of the problem, beginning with state variables. But when I do modeling, I do the state variable last.

Furthermore, it really is iterative. You’re going to have a process where you go through the modeling. The state variable is just information. You go through the model and say, okay, I need this information, this information, this information—ah, there’s my state variable. But, for example, how do you make decisions? It depends on how you’re modeling uncertainty. How do you model uncertainty? It depends on how you’re making decisions.

So I describe the process of modeling uncertainty and making decisions like climbing two ladders. Don’t come in with some incredibly sophisticated method for making decisions under uncertainty if your model only has a very basic probability model. With our complex problems, we can make models of uncertainty as complicated as we’d like.

Generally, you’re not going to start off with the most complicated model. You’re going to start off with something more basic. Then you want to have something that makes decisions. You don’t need to do something incredibly complicated because it’s just a basic model. Once you’ve got a decent decision model, you can come back to your model of uncertainties because maybe now you want to get other metrics of risk.

Now you want to have decisions that reflect risk. So you’re climbing this ladder, and I’m sure your entire process at Lokad has been iterative. We always want the simplest model that solves the problem. The question is, what does it take to meet the business objectives? And that’s a learning process.

Joannes Vermorel: Absolutely. I do give credit to your book. I think there you’re listing, I think something like, I forget if it’s like 15 different types of uncertainty and that’s probably the longest list that I’ve seen collected and yes, that is a very real concern. When you say the uncertainty, people think, oh, you’re just talking about the inaccuracies of the point forecast. And I would say, certainly not, there are so many sources of uncertainty. That can be the price of the commodities you depend on that varies, that can be the labor force you depend on who goes on strike or just isn’t qualified or just not there.

It can be the possibility of having housing problems in your locations. This is looking at uncertainty just through the lens of the inaccuracies of your sales because that’s just one thing, demand for sales forecast, sales point forecast, it’s just extremely narrow. I completely agree with you. Being super iterative at Lokad is exactly what we do, which raises another question, a big concern, which is the productivity of the engineers you have to put on the case to iterate faster.

At Lokad, the way we typically approach those stochastic optimization problems is by identifying programming paradigms. We have a collection of these paradigms. They are not unified, more like a small library of stuff that you can use. These paradigms give you an angle to go relatively swiftly in the implementation of your solver. Here, I completely agree with the iterative process. The challenge we have from a business perspective is that my clients are very impatient.

We need to iterate very swiftly, but we are dealing with something complicated, involving hardcore algorithmics. They need to get their algorithm implemented in finite time. Another consideration that is never discussed is that many methods I’ve seen in books work if you have super smart university professors with a decade to implement the algorithm. In the real world, if you have 100 hours to get it implemented, some methods are incredibly difficult to get right at an implementation level. That’s why having those programming paradigms helps. They provide a way to code it in a manner that will work in production in a finite amount of time while iterating on the stuff.

Warren Powell: Toward the end of chapter 11, I think in the very last section, I have a subsection on the soft issues of evaluating policies. So in the book, every where I always write policy optimization is maximize over policies, expectation of something. Toward the end of chapter 11, which can be downloaded from the website, I have about five different qualities like methodological complexity is one of the qualities. When you look at a method, absolutely what you just said is very important, a computational tractability, transparency. We’ve all coded algorithms and the answer comes out something and we’re scratching our head and we, and the customer doesn’t understand and you want to be able to say, well, here’s why it came out that way because it might be a data error, it might be a rule change.

I mean, at Optimal Dynamics, we get this data from trucking companies and we’re running all the same war stories that you’re running into and when they don’t like an answer, they would like it to be fixed pretty quickly. One of the most powerful and important tools that I developed in my lab at the university is a graphics tool called Pilot View where it has two modules. One is a map where you can see flows and filter it in all kinds of fancy ways but the other, I call it my electron microscope, is where I can bring up individual drivers and individual loads and click on anything and see what driver was assigned to what load but not just what driver was assigned to what load but what loads we considered because if I have a thousand drivers and a thousand loads, I cannot consider all possible combinations of a thousand drivers times a thousand loads and it has nothing to do with the algorithm, it has everything to do with the network generator.

So we use fancy things but I may have a driver that I didn’t assign to a load, why not? Well, maybe because the penalty were too high, maybe because the cost was too high or maybe because one of my pruning rules simply didn’t consider it and we had that happening. And of course, when the when the customer is complaining you need an answer very quickly because when once you’re in the field that’s it for complicated algorithms.

Joannes Vermorel: I do relate enormously to that. I didn’t came up with this observation, I found it on the internet, don’t remember exactly who said it but the gist of it was, in order to debug an algorithm it requires you to be twice as smart as implementing the algorithm.

So if you go with an algorithm that is already, when you implement the algorithm, as smart as you can be, then that mean that once you will be in production and you want to debug, you have to be like twice as smart and that’s not even possible because implementing the algorithm was already you at your best. So you need a solution that is not you at your best so that when you want to debug it you can do it. Also, I very much agree with what you describe, which is this supporting tool. The role of instrumentation is absolutely fundamental and I think that’s also something, but it’s difficult. This book contains so much, I give you credit for that, it’s not exactly a book that is, that is short on insights, the role of instrumentation is paramount.

The classic optimization community in the deterministic sense, they would just say, how much CPU second do you need and what is the performance of the solution that you get and this is it. But when you enter this realm of stochastic optimization you will need extensively supporting instrumentation to understand what is going on. And I think that is something where also there is this sort of paradigmatic gaps in the way to look at it because it mean that it’s not just the tool that let you generate the decision, it’s also all the sort of instruments that you can bring on top of that so that you can make kind of sense of your decision-making process, not necessary just one decision but the decision-making process, and without that people would raise concerns and you’re ably stuck and you can’t just say, trust me, it’s good. In stochastic optimization that doesn’t work as well as in classic mathematical optimization.

Warren Powell: Yeah, it’s obviously a nice challenge working with industry. I’ve had that experience since I started my career. It wasn’t until the 1990s when I set up my lab and I hired some professional programmers, I mean all PhDs, but they had their PhDs and they were just there to code and without the two people that I had my lab just never would have gotten off the ground. It’s amazing how, I don’t know if you’ve ever found this, where your algorithm is presenting a solution and you don’t like it and the customer doesn’t like it and everybody’s sitting around the table scratching their head saying, what do we think is wrong and the number of times we’ve done this exercise and different people all have a theory that reflects their skill set. So there I am, oh my God, I think the algorithm could be fancier and this guy’s worrying about the data and this guy’s worrying about a programming error and the number of times we would sit in hypothesis size and we would all be wrong.

It’s really amazing. Obviously, I’d love to sit with you at some point and learn more about your issues of getting the raw data. In my industry of truckload trucking, we only work with carriers that are already working with some commercial TMS system, and that doesn’t mean it’s perfect, but it means we’re way higher. But it’s a challenge, it’s a lot of fun. One thing I wish we could do more of is challenge the academic community with the real-world problems, and I sort of gave up on the academic community.

They’re not in there to solve problems, they’re in there to prove theorems and write papers. I lived that world for almost 40 years, and I understand it, but I think it’s fundamentally flawed. So, one thing that I did have with my freight transportation companies is they were all willing to share data. That’s not true with shippers.

I’ve never encountered a shipper willing to share supply chain data. That is off the board, they’re not going to do it. I did a big supply chain project for Pratt and Whitney, the jet engine manufacturer, and it was being funded by the government and it was all being blessed by the company that owned them, called United Technologies, but they wouldn’t even listen to a proposal of sharing their data. They said, “Oh my god, that’s way too proprietary.”

And so, they happily got involved in the project where we had to write our own fancy data generator and make up random demands all over the world, and they said, “No, some of our suppliers are really secret, you know, we can’t even let somebody know that there’s a supplier who does that anywhere in the state of Connecticut.” They just said no, that would be, that would be too much information.

So, it’s hard to do these real-world problems in an environment where you can’t get the data. I’ve signed up with Rutgers University, I’m now an executive in residence at their supply chain management department, and I’m hoping to talk them into building a simulator with fake data, and trying to get them to at least work with simulated realistic problems.

Joannes Vermorel: I can very much relate to the series of problems that you faced. I think I was coming from the other spectrum of the supply chain world. At one side of the spectrum, you have trucking that is almost like the final relatively short-term decisions, even more extreme would be piloting robots, that’s one end of the spectrum.

At the other end of the spectrum, you have S&OP, sales and operation, super macro plan, corporate level, and whatnot. Then you have everything in between. My own journey was coming from the other end, the S&OP, very strategic, very forecast-oriented. The first few years of Lokad, decisions were not even involved, it was just pure forecasting.

Back to your concern, my problem was that I was seeing in academia, by the way, I’m a PhD dropout, so I didn’t make my PhD supervisor proud, and I dropped out shamelessly from my PhD to create Lokad. Academia focuses on forecasting accuracy, publishing 20,000 models for better sales forecasts since pretty much forever.

In the industry, we have exactly the situation that you describe, 10 people sit around the table, they look at the problem from their angle, and when we’re talking about forecast, which is pre-decision stage, just before acting the decision, first they want to make the forecast. People are wrestling to steer the forecast up or down.

In S&OP, you have salespeople who want to sandbag the forecast, so they want to have a very low sales forecast so that they can exceed expectations. Then you have manufacturing people who want to inflate the forecast because if they inflate the forecast, they will have more budget for their manufacturing assets, and thus whatever order they finally get to manufacture, if they have more capacity, it will be easier.

You have this tug of war where sales wants to throw the numbers to the ground, manufacturing wants to send it to the sky, and it’s not super rational. The interesting thing is that in academia, people would publish a paper where they find an incredibly sophisticated way, leveraging an obscure Russian theory, to remove 0.1% of bias.

Then you end up in this room with a tug of war where people are literally saying, “I want minus 50%,” and another guy says, “I want plus 50%.” That gives you the sort of disconnect. To access the data, it has always been incredibly painful.

Warren Powell: One forecasting question, and I’d love to know what you do with this. There’s a lot of math on taking history to forecast the future, but we know that a lot of times the future is likely to be very different than the past, for any one of a number of reasons, and especially, and I don’t know whether the future is going up or down, but I do know that it could be quite a bit different than anything I’ve seen in the past. May I ask how you deal with that?

Joannes Vermorel: Yeah, absolutely. So, the typical way is we want to introduce some kind of macro uncertainty that is not exactly justified. That sounds weird. You would think, okay, we have the demand forecast, fine, and we are going to say, let’s add a variable where we have a 4% chance every year to have a sharp decline of 30% of the demand, of the activity, of everything.

Then people say, “Wow, 30% decline in one year, that’s huge. Why would you ever consider that?” My take is, if you look at the 20th century, there were two World Wars, quite a series of other wars. Then more recently, we had global lockdowns and whatnot. So, saying that every 25 years there is an asteroid in your face that damages your industry, I don’t think it’s that far-fetched.

But people expect that they forecast something that they know, and here we say, no, you don’t really need to know exactly. You could just say, we are going to assume a big disruption, whatever it is, and then we are going to make up numbers. Those numbers are completely made up, 4% annual probability, 30%, you can change it, you can say five, and you can say 50%.

It forces you to consider your major disruption all the time. We were serving, for example, clients in aviation. People would say, “Oh, it’s not that frequent.” But it is frequent, because when you look at the industry, for example, the 737 Max of Boeing was grounded. For my client that was serving airplane and had dozens of those airplanes, that was a major issue.

The bottom line is to accept putting things that are incredibly pessimistic in your models. That’s usually very much a hard sell because it’s not consensual. The problem is not really that the math is lacking, it’s that it’s scary, and people don’t want to be scared. But if you do not prepare yourself for those big impactful events, then you’re going to be unprepared. That’s as simple as that.

The other thing is that would be one side, that’s a very pessimistic side, you need to look at those big disruptions, be okay with it, and embrace the fact that it will happen with 100% certainty if you give it enough time. That’s one aspect.

The other aspect is that most of my clients, when they look at uncertainty and decision uncertainty, they only see the bad outcome. I think the problem comes with variability. People equate bad outcomes. In manufacturing, people like Deming popularized the idea that you need to be consistent. It’s a cardinal virtue. You have to be absolutely consistent. It’s okay to do crappy products if you do crappy always the same. It’s going to be cheap, and people know what they can expect.

What is not okay is to do some good, some bad. No, you need to be absolutely consistent all the time. So, people equate manufacturing variability with something bad. But is it though? Once you exit the world of manufacturing, is variability such a bad thing? Not naturally.

A prime example would be fashion. Fashion, you create products that are hit or miss. If you can increase the variance of your hits and misses due to the fact that you have a low that is going to be a fat tail, it means that if you can increase the variance, yes, you will have more misses, but you may have hits that are an order of magnitude larger.

Variability in manufacturing is bad, but in the supply chain in general, it’s not that bad. If you can have a completely almost perfectly steady stream of opportunities, super steady, but if you’re disrupted, it’s going to kill you, versus something that is somewhat erratic, somewhat bumpy, but where there is a lot of constant risk taking that you carefully manage with decisions that are optimized under uncertainty so that when you make a mistake, it doesn’t kill you.

You may end up in a place where when the disruption comes, it is not nearly as impacting. For example, if you are in a business where you expect 98% of your products to be reconducted from one year to the next, if the law changes and 20% of your products are deemed illegal because you used the wrong product, the wrong process, the wrong whatever, that’s going to be a massive hit.

You were in a business where you had two percent of your products that were changing every year, and now you have 20% that gets phased out due to regulation. But if you’re in a business where every year you renew, let’s say 15% of your products, well okay, there is a year where you have 20%, but you will be able to recover much faster because you have this sort of appetite for novelties that you keep around.

Not all uncertainties are bad. Sometimes chasing a little bit of it is good. For example, most forecasting practitioners hate forecasting new products because they don’t have any history for the time series. If you look at time series forecasting literature, 99% of the time, people exclude products that have no history. From my perspective, forecasting the products that have no history is the most interesting thing.

That’s where the true battles of supply chain are done. It’s those products that are new, that might be hits, and might change the course for the company. So it’s a long answer to your question.

Warren Powell: I’ll make one closing comment. One of the things that I’ve appreciated most about my framework with my four classes of policies is it allows me to say, don’t worry about the decision. We’ll pick one of the four classes, we’ll pick something sensible, don’t sweat it. That’s not the big problem. The big problem is modeling uncertainty. If I can get people away from the complexities of making decisions under uncertainty and more focused on modeling the uncertainties, that’s the big win.

Joannes Vermorel: I completely agree with you. Large corporations, when facing uncertainty, one of the worst things that they can do is to make up rules to reduce uncertainty. You would invent rules just to simplify your problem. For example, they have read, UPS is only doing left turns in their circuits, and then they say, okay, so we ourselves are going to do only left turns because that simplifies something.

You see that when you had so much potential and options set, so much uncertainty to deal with, and I think one of the most adverse ways to approach the problem is to invent a whole series of constraints that are completely made-up constraints so that you have a problem that becomes more manageable. To segue into your frameworks, I would say that’s the wrong way because there are options to deal with the true problems.

So don’t start inventing constraints just for the sake of it, just because you’re scared that there won’t ever be any solution to consider your problem. There are plenty of solutions, so you need to delay inventing rules and constraints just for the sake of simplifying the resolution of the decision-making process.

Conor Doherty: Well, I’m still here, and that’s all right. I’ve made three or four separate pages of notes, but one of the things, and it’s following up on, you use the term managing financial risk, and I’ve written trade-offs, business concerns, evaluating performance, and managing financial risk.

So, Warren, as a sort of opportunity to summarize your framework and your approach to stochastic optimization, I know that your perspective is one of managing business concerns and managing the trade-offs that are endemic to decision making. So, take as long as you like, but how exactly does, whether it’s your lectures online, which I watch and are lovely, or the 1100-page book, how exactly does your framework manage the financial risk associated with, be it routing optimization or managing inventory or forecasting and managing inventory for products that we have no history for?

Warren Powell: Sure. First of all, I think one of the byproducts that Johannes and I have, we both work on real problems. And once you work on real problems, you’re going to come away with certain things that we all agree on, one of which is model first, then solve. You’ve got to understand the problem. You use the word risk, and that to me highlights you’ve got to talk about uncertainty, and this is way more complicated than a normal distribution.

The statistics people like, anytime you deal with uncertainty, the first thing they’re going to do is introduce a student to the normal distribution. They’ll say, okay, we have a mean, and we have a variance. There’s randomness around the mean, and they don’t seem to recognize that the biggest source of uncertainty is the mean. We don’t know what the mean is gonna be. The mean moves around. Now there’s noise around the mean, but it’s the movement of the mean that’s the worst.

And then these events that don’t really belong in a probability distribution, they’re contingencies. They’re like, look, I don’t know about a probability, but here’s something that might happen. What would I do given that contingency? And I don’t care about the probability of it. There are things that I think can happen, and I have to know what happens if that ship arrives a month late. What if this port shuts down? What if there’s an earthquake in Japan? There are these big events. I don’t necessarily need to know exactly what event, but I have to plan for contingencies.

The whole business of making decisions under uncertainty, one of the first things I like to say is, boy, there’s a lot of complicated math, but do you realize we humans make decisions under uncertainty all the time? And early in my career, when I was really struggling with my truck problems, I’m like, but the truck dispatchers are doing this already. We have to remind ourselves, one thing the human brain is astonishingly good at is making decisions under uncertainty.

A lot of these issues, people will say, oh, I don’t like stochastic. And yet that exact same person will plan for random events, uncertainties, and contingencies. It’s just something that’s built into the human brain because I guess we, animals, have had to deal with this for our entire evolution. The biggest challenge is not making decisions under uncertainty. The biggest challenge is teaching computers how to make decisions under uncertainty.

And so, I don’t think there’s ever going to be an end to this conversation. We do need to quantify, we do need to use computers because the idea of rooms full of people making decisions gets a little dated. In the trucking companies, we have a whole suite of models at Optimal Dynamics, from strategic down to the real time. But the product that is absolutely the foundation of what we’re selling is the one that does real-time dispatching because there isn’t a truckload executive anywhere in the United States who doesn’t think that the number one problem with his company is the dispatch floor, whether or not it’s true, that’s what their belief.

I have learned the idea of finding the right driver to move a load isn’t really the most important thing. What’s most important is picking the right load and that’s like Revenue management for Airlines. You have to plan it a little bit into the future, but it’s so hard finding the right driver that will get home and satisfy the DOT hours and everything that everybody gets hyperfocused on the driver dispatching problem.

But it’s really finding the right load because what’s hard about finding the right load is I may have to commit several days in the future or a week and I don’t know where my drivers are going to be and I don’t know what I can handle. So you have to be able to plan under uncertainty. The dispatchers know this and maybe they don’t have fancy tools but they do have this gut feeling of, “Hey, that’s a good place. I’ll probably have a driver there.”

I’ve seen people flat out say we don’t deal with any uncertainty because CEOs don’t understand stochastic. No, they don’t understand the word stochastic, they all understand uncertainty. Now, by the way, we have to move past their insistence on meeting the forecast.

I think one of the biggest problems with Transportation people, I’m talking about on the supply chain side, is they all have a budget for their transportation budget specifically truckload and none of them hit the budget. It’s always some optimistic estimate of their transportation spend and they always manage to spend more and it’s just something that comes with the space of being a supply chain guy who has to plan the transportation assets.

So, a lot of fun problems. I don’t think we’ll ever run out of things to say.

Joannes Vermorel: Yeah, and frankly to bounce back on your case of the human mind dealing with decision under certainty naturally, I completely agree. And I see this very peculiar situation where actually the toughest discussions are not with people that are uneducated in math or at the other end of the spectrum that are super educated in math. Those are the sweet spots, the completely uneducated and the super educated.

The difficult spot for me are the slightly educated because the funny thing is that it is actually fairly hard to make a computer deal with uncertainty. I very much agree and what does it mean for a person that is slightly educated in the art? It means Excel, Microsoft Excel.

And so the problem is, and I’ve seen that very frequently, is that they know a little so they know Excel and now there is this problem of looking at the universe through only the solution that you can think of. So you end up with the layman who doesn’t know anything about Excel so he just, you know, like playing poker, he has become good through habits. He doesn’t have the theories but he managed to play a poker game and perform decently.

And same thing with your picking the right load. I’m pretty sure that you will find plenty of people who have zero concept of probabilities but through experience have become very good players. They have this intuition although they don’t have the formalism.

And in between, you end up with people who know Excel and say, “Okay, I need to implement this in Excel.” And Excel is a terrible tool for that because Excel doesn’t deal with probabilities. Excel is not geared to do anything that is like Monte Carlo and Excel is the worst sort of tool to do that. Excel is great to do your accounting but absolutely terrible to deal with any kind of uncertainty.

And so my toughest situation is like people who are committed to the solution to Excel. If it’s outside the Excel and it can be outside either because it’s just a gut feeling which is more correct than the Excel calculation or because it’s too sophisticated and doesn’t fit anymore into the Excel calculation, there is this strong rejection.

So that’s very interesting and I very much relate and I have this kind of middle segments of people who have committed themselves to Microsoft Excel spreadsheets and that’s where it is very much a struggle.

And I think that most of when they say the CEO doesn’t like it, very frequently I have found that it is a projection of their own perception of the problem. CEOs are almost invariably, I mean past a certain size of company, people who are excellent at dealing with a huge amount of mess.

I think it’s very hard for anyone to reach a position of CEO of a company, let’s say a few hundreds of people and above, and not be just completely unfazed through the fact that the world is super chaotic. I mean, that’s your daily life, dealing with nonsense that is thrown at you all the time.

So my take is that I’ve seen very frequently that when people tell me, “Oh, it’s too complicated. The CEO would not understand or whatever,” no, it’s their own fear projected to that the CEO anyway has barely any time to understand anything about this company. So that’s just going to be one thing out of a thousand things that this person doesn’t understand about his own company and that’s just not going to be the last. So, it’s, I mean, that’s my take. What do you think?

Warren Powell: Yeah, well, often CEOs are, they come in from an entirely different level. They’re focused on the big picture of the finance, especially the larger companies. The fine details of what goes on on the operations floor probably was something they bypassed in their career.

I mean, in the old days, by the way, in my day when I was going to school, a lot of even people from Princeton, undergrads from Princeton, might go work for a Proctor and Gamble and they’d spend six months on the factory floor and then winding up the management ranks. And so they would be on a fast track but they were told to start at the bottom. That died in the 1980s.

In the 1980s, when I started teaching at Princeton, none of the Princeton undergrads ever went and worked for a company. The hot thing was to go work for management consulting firms. And so they would get their field experience working for a management consulting firm, go back for an MBA, come back, work for a few more years and then usually leave for a high-level executive position at one of the companies. So they bypass all those details.

The your point about Excel, when I’ve worked in the trucking industry, the only people I have ever encountered were truckers, I mean, I’m sorry, dispatchers and low-level managers. There was very little people who could do even basic work on an Excel spreadsheet. Whereas in supply chain, there’s a million of them all sitting there doing very basic stuff.

Now look at the books. As I’m getting involved in a real supply chain program at Rutgers, I’ve been going through all of these books and either the books are math games or they really tone it down. So not only do you have these people who think they can do anything inside an Excel spreadsheet, the books only teach them stuff that can be done inside an Excel spreadsheet.

And so I think we have more than just an Excel spreadsheet problem. We have to think about who’s going to be solving these problems and using it. I very much align with you as like, “Well, what we need is good tools where under the hood the tools can be quite sophisticated but they need to be easy to use.”

At Optimal Dynamics, boy do we focus on trying to make our tools easy to use. But underneath the hood, as long as it works, people really do want the best possible solution. Supply chain, I feel like, you know, as I start peeking at it and looking over the people’s shoulders and saying, “There’s this interesting supply chain world but what’s going on is you’ve got,” I know I saw a statistic, “93% of people do their planning in a spreadsheet.”

Well, you’re limited to what you can do in a spreadsheet. And so when you start talking about, you know, even running simple simulations, but I mean I can do a simple inventory simulation spreadsheet, but let’s start talking about introducing long lead times in multiple suppliers and, well, you know, that gets beyond the ability of a spreadsheet really quickly. It also gets beyond the ability of the people who are coding that spread spreadsheet and who think that they can do it on their own.

I have a former PhD student, she’s now our chief analytics officer, hyper bright, but she spent eight years doing operations planning in Kimberly Clark in Brazil. Long story behind that and at one point she was struggling with, you know, the usual inventory planning problems. So she called in, she used to work briefly at McKenzie, so she called in her old friends from McKenzie and guess what, McKenzie only knew what was in the textbooks and she knew right away that they had no clue what they’re talking about and kicked them right out. We’re not teaching even the best and the brightest how to solve problems. I’m not talking about doing weird math, I’m talking about doing practical stuff, the kind of modeling that absolutely should be done to solve the problem. It’s not being taught anywhere.

Conor Doherty: If I may, it is being taught somewhere. Shameless plug.

Joannes Vermorel: Shameless plug. At Lokad we have started to teach this in half a dozen of universities that mostly are around Paris. We have also started a whole series of public workshops for problem-solving situations in supply chain and one of the biggest effort that we have, our biggest investment that we have to make is to create data sets.

So we create and publish the relevant data sets, and indeed, my own opinion was that creating a fully synthetic data set is just too hard, so we just have to fully anonymize existing customers’ data with their blessing. We take real data, make it completely anonymized, preserve the weird patterns, and wrap it into relatively small well-organized data sets so that students can tackle the problem without spending three months dealing with loose angles on the data. I very much agree, and by the way, my two parents started at Procter & Gamble, so I can very much relate to the sentiment.

Warren Powell: So you’re teaching, what type of student are you teaching to in the schools around Paris?

Joannes Vermorel: Oh, it’s very classic. The French system has two years of prep school, so that’s basically a national exam. You do two years, national exams, everybody gets ranks, and your grades get published in the newspaper, so if you get bad grades, it’s in the newspaper, that’s no pressure. Then, there is what we call the Grandes Écoles, but you can think of it as the mini French Ivy League. People go into those engineering schools. So I’m only talking about three segments: engineering schools, business schools, and administrative schools. Here, I’m only talking about the engineering schools.

Warren Powell: Engineering, okay. So at Princeton, I taught the engineers. Now that I’m getting involved with Rutgers, it will be the first time in a business school and I’ve already been gently warned that of all the different categories of students in the business school, those who choose Supply Chain management tend to be kind of bottom of the list in terms of technical skills. The ones higher up go into finance and so there’s a dribble-down effect. I haven’t started, I’m not going to teach a course, I’m teaching the teachers, but I will depend on them to say, “Look, Warren, we’re just not going to be able to get away with this.”

One thing that I’m focusing on is, I say, “Look, there’s one very important part of my framework that does not involve any math. It involves the following three questions and you cannot answer these, you cannot build a model without these three questions.” Even if you’re not going to build a model, you should still, if you want to solve a problem, answer the following three questions: what are your metrics, what types of decisions are you making, what are the types of uncertainties?

In plain English is the way we phrase it here, and so I’m like no math, but these are the questions where if I want to build a math model, I still have to have the answers to those questions. So I’ve decided, you go to the business community and you ask about metrics, they all know about metrics, they got lists and lists of metrics. Then get to the decisions and say, “Do you have a little red book with a list of what decisions you make?” and they give you blank stares.

So after this talk, Joannes, I’m starting to generate a series of thoughts and notes that I’m going to share with the other Rutgers faculty. It’s a Google Docs document that can be publicly edited and you’ll see me developing different categories. I just started the section on decisions and I’m going to send it to you too because I think you’ll have fun. This is not for a book or anything, this is just chatter, this is my way of teaching the teachers because I can’t tell them what to do, the professors have to say, “Oh, that’s a good idea, I think I’ll use that.” If they don’t do that, then the idea doesn’t get in the class, but I have to trust their knowledge of what the students are doing. There is this one course on operations analysis, and that’s the course where they deal with inventory problems. I think you can imagine what’s being taught there, it’s a very basic presentation and I’m like, “I’m sorry, shouldn’t we at least tell them how to, even in a spreadsheet, you can simulate a very basic inventory problem.”

So I’ll send you the link to the Google Doc. One of the things that I would love to compile, and I haven’t, I’m just starting to think about this, but I haven’t done Supply Chain management my whole career, I’ve worked on a much wider set of problems. I want to come up with a list of decisions. This is not going to be a small thing; decisions come in many flavors and categories, it’s not all inventory. There’s finance, there’s informational decisions, and so it’s going to end up, I’d like to put it in a spreadsheet and then I would love to send it around and just invite people, “Hey, what’s a category of mine or what are examples of decisions?” because I had this catch line that says, “If you want to run a better anything, call it a supply chain, you have to make better decisions.” I’ve never heard anybody disagree with that, everybody’s like, “Yeah, that’s right.” Well, if you want to make better decisions, what are your list of decisions? And then I get these blank stares.

So I’m going to start off with a very non-quantitative approach, so this appeals to your sense of, you got to model first and I think a non-quantitative MBA, this is a good challenge for them because answering and then of course the uncertainties, entire careers are built on identifying sources of uncertainty for supply chains, but I really love the one about the decisions. Shouldn’t we all be able to know what decisions we’re making? I understand why in business they don’t because it’s like, “Well, that’s somebody else’s problem, I’ll just evaluate how well he does.” So business, it’s entirely the language of metrics, but shouldn’t there be a little red book somewhere that has the decisions?

Joannes Vermorel: Yeah, I very much agree and decisions are very difficult because large companies tend to bury decisions and decision-making processes under workflows and processes. In fact, they don’t even see a decision because there is a rule that is being applied that is actually a decision-making process. And it’s already so ambient that they don’t see it anymore, it is just steering the company. It can be a bad policy, it exists, it steers the business, it effectively delivers potentially thousands of decisions a day, and nobody even sees it. Once it has been in place for a while, this thing doesn’t even have a command. There is nobody in charge of that just because it’s like having fresh air in the building, it just happens, people don’t even know exactly why it is, it just is.

I very much agree with your idea of decision. It is difficult because people have the wrong notion about decisions. They think a decision is something where there is a meeting, there is tension, and there is a boss, arguments will be presented and the boss will decide. That’s one type of decision, but there are much more mundane decisions that are much more consequential. When they are so mundane that you don’t even see them anymore, that’s very intriguing.

Warren Powell: Because, as you said, they’ve already chosen the policy, so once you fix the policy, it stops becoming a decision. And actually decision is not the decision the decision is what policy.

Now let me just offer one other Insight. You know I talk about the complicated policies in the simple policies. One line that I’m pretty sure I use in my big book is the price of Simplicity is tunable parameters. Virtually any simple policy like SS policies, have tunable parameters, and tuning is hard. Tuning is one of these universal types of decisions. It’s an active learning problem.

I’m giving a talk at a supply chain analytics conference workshop at Rutgers in June, and one of the things I’ve got a whole section on is that there’s something that absolutely everybody, no matter what field you’re in, has got to understand. There’s a class of sequential decision problem that’s called learning.

It goes under many names. It can be intelligent trial and error, stochastic search, or multi-arm bandit, but they’re learning problems. There’s nothing physical. When you have something physical, things get more complicated, but there are a lot of problems where it’s not physical, it’s just learning.

You try this, that worked or it didn’t work, I’ll try something else. It’s a sequential decision, but the only thing you carry from one time to the next is what you learned and your belief about what works best.

Sequential learning should be taught at the undergraduate level, not in one course, but throughout the curriculum like statistics is taught in different styles. Active learning should be taught to anybody outside of English majors.

Unless you’re a raw humanity student, name any field where you don’t have to do some form of intelligent trial and error. This is a fundamentally human process, and there are tools for it that are simple, so you can get people going.

There are basic policies called UCB policies that you can teach people in a minute. You just say, look, you have discrete choices, here’s how good I think each one is, but here’s my uncertainty.

There’s a simple exercise that says if all you do is base it on how good you think it is, that can be seriously suboptimal. You want to aim a little bit higher, you want to aim at how good something might be. That’s an insight you can teach in a minute, and yet there are subtleties that make it much richer and this really needs to be taught.

Joannes Vermorel: I very much agree. That’s very funny because from a machine learning background, Lokad was mostly forecasting machine learning.

The typical situation was that those simplistic policies with tunable parameters were never tuned in practice. When you finally get your hands on the dataset of a company, it’s complicated, but at Lokad, we finally get the dataset.

You apply your learning algorithms and realize that there is very little to learn because the company has been operating with an incredibly rigid autopilot for so long that you can have billions of dollars or euros of history and yet so little to learn from because you’ve done only the same thing over and over with zero variations.

One of the challenges we face is that frequently, when we want to learn, we have to start to explore and add a little noise. This noise is just for the sake of learning.

You have to make sure that it’s not too expensive, but the idea of deviating from what is considered optimal, they have no clue whether it’s optimal, but it is certainly the default, the status quo, the practice.

Deviating from the practice to explore randomly for something as abstract as collecting information on what the landscape looks like when you deviate from what you usually do, is very baffling.

There are very few people who have gone to MBAs and whatnot that can comprehend this idea of dropping tiny drops. If you’re a large company, even a small dose of exploration, if you operate at scale, will give you a lot of information over time.

In manufacturing, deviation is bad. You want to be as rigid and consistent as possible. But in active learning, if you do that in the supply chain world and you’re so rigid and this policy that remains immutable, you barely learn anything.

That is a very strange concept. Being introduced to the idea of active learning, that you can cherry-pick your deviations to maximally inform you, so it’s not just doing something random but with an intent to learn something, is crucial.

Warren Powell: The insights you just articulated are so fundamentally important that they should be taught everywhere in all kinds of fields, not just in analytic fields.

You can teach that at an advanced analytic level and at a basic level, depending on the students. I don’t know why this isn’t being taught.

I’m writing an article for Princeton locally to say, hey look, 40 years of Princeton, guess what course we’re not teaching. About half the university is involved in departments where there’s some opportunity to do trial and error thinking of some form.

We can go on for several more hours.

Conor Doherty: I am going to jump back in and close one loop. One of the shameless plugs, just to clarify, when you’re talking about supply chain students learning, the workshops you mentioned are actually publicly available on our website.

In terms of learning how to code for supply chain problems, we have publicly available, completely free resources on our website, docs.lokad.com. These are guided exercises designed by our supply chain scientists to mimic the sort of decision trees you’re talking about.

If you want to evaluate performance, supplier analysis, we have a free guided tutorial for that where you get to see all the snippets of code designed specifically for these kinds of problems as opposed to a rough approximation in an Excel spreadsheet.

Warren Powell: I know that at Lokad you have your own programming language. So I found that very interesting. I love the resources that you’re making available. We’re trying to do something along those lines for trucking, but trucking is a very different business.

We’re trying to put very educational stuff. We would never attempt to put something like that. First of all, we don’t have that simple bit code and there’s nobody inside the truckload industry.

One thing that’s kind of neat about truckload trucking is, well, we don’t have a lot of competition. It’s not the scope of the truckload business which in the United States is about $800 billion a year. I mean, it’s a big market, but it’s a tiny fraction of supply chains.

Supply chain is a true ocean, whereas truckload trucking is a sea or something like that. But I’m going to bring your resources to the attention of the department at Rutgers because I think that could be very interesting.

I have to deal with the fact that these are business students who I need to get to learn their students. They also have a department of industrial engineering and I have a feeling that’s going to be more engineering level.

I actually think that the two departments should work together because that first step of those three questions, answering those three questions, is really hard. You really have to know what you’re talking about. So you need good, smart, management consulting type thinking to answer those questions.

Once you’ve answered them, now you need a different skill set to turn them into analytics and use a computer to help. So I’m hoping at Rutgers, I know people in industrial engineering quite well. The group that I don’t know as well is the department of supply chain management. They seem to like what I’m saying and I’m going to try to make the pitch that we pull them together.

Conor Doherty: I think that’s going for quite a while now and I have exhausted all of my questions. But it is custom at Lokad to offer the final word to the guest. So Warren, I’ll allow you to close with any call to action or shameless plugs. We don’t mind those here.

Joannes Vermorel: There is a clear call to action, which is to buy the book. It’s a very solid good book.

Warren Powell: I recommend that people do not start with the big book, but they start with a book that they can download for free.

Tinyurl.com/SDAmodeling, that’s the book I wrote for the undergraduate class. It’s called Sequential Decision Analytics and Modeling. I work with a publisher, they don’t pay me anything, but they allow me to offer the PDF for free, the published version.

This is the book. It uses a teach-by-example style. So, other than chapter one, which says here’s the universal framework and there’s inventory examples in there, every chapter other than chapter 7 is just different examples, all written in the exact same style with an emphasis on modeling.

So, I have my five elements: State, decision, information, transition, objective. I always start with a narrative, a plain English narrative, and then I have the five elements. Then I say a word about modeling uncertainty, not very much. Then I say something about, here’s a way that we might make decisions.

By the time I’ve gotten to chapter 7, I have given examples of all four classes of policies. So, chapter 7 says, let’s pause for a bit, let’s look at what we’ve just done. The remaining chapters, 8 through 14, are simply more advanced examples, including the beer game.

The beer game is my opportunity to do a multi-agent problem. One of my favorite chapters in my big book is the last chapter on multi-agent. I wrote that chapter and said, if I were starting my career over today, multi-agent would be so much fun.

And of course, in the supply chain world, everything’s multi-agent. It almost defines the problem that you don’t have one. Like in my trucking work, even though there’s different managers, we roughly act as if the trucking company is a single agent.

Supply chain, you can’t. It just doesn’t work. You’ve got to model the fact that you’ve got all these interacting components, which opens the door to modeling who knows what. So now, you’re modeling organization of information, not just ordering inventory.

This is such a fabulously rich area. I look at 70 years of textbook writing and realize that our books seem to be so far behind what’s required to really solve the problem. I’m a little surprised by that. It’s a neat opportunity. I wish I was a couple decades younger.

So, it’s great that you’re doing this TV series. I’m going to definitely promote this. I’d certainly like to send this link around, but I will also point people at your website because I love your academic style.

Optimal Dynamics is a great company. I can’t do much of a shameless plug because it’s really focused on truckload carriers, but I will give Lokad the shameless plug. I like your style. I talk about you guys because it says, they’ve got a very academic point of view.

I love how you like to share. Academics love to share. Yes, we’d like to make some money, but we still can’t help sharing our ideas and being very proud of it, as you should. I appreciate that because I’ve gone through your website with care and it’s helping me learn things and pick up on your style.

Conor Doherty: Thank you very much. I don’t have any other questions. Joannes, I’ll say thank you very much for your time, Warren. Thank you so much for yours. And thank you all for watching. We’ll see you next time.