How to Solve a Difficult Forecasting Problem

The Boulder, The Cliff, and The Baby

Imagine you are faced with this very urgent problem: A large boulder is teetering on the edge of a cliff, at the bottom of which sits a baby, at risk of being crushed. How do you solve this problem?

One solution for the teetering boulder is this:

The Boulder, The Cliff, and The Baby

Imagine you are faced with this very urgent problem: A large boulder is teetering on the edge of a cliff, at the bottom of which sits a baby, at risk of being crushed. How do you solve this problem?

One solution for the teetering boulder is this:

Use large ropes and cables to fasten the boulder to the top of the cliff, buying some time while you build a large infrastructure of concrete and metal to support the boulder from below.

Sounds great — the boulder is secured and the baby is now safe. But is this really such a great solution? What if instead we just did this:

Remove the baby from the bottom of the cliff.

While this does not solve the issue of the teetering boulder, it has done something better. It has made the teetering boulder irrelevant — no longer a problem that needs to be solved! (Once the baby is in a safe spot, who cares if the boulder falls?)

An Example from Forecasting

Too often, in dealing with our urgent business forecasting problems, we go for the first type of costly and time-consuming solution. Sometimes it may not be obvious that there are alternative approaches. Or sometimes we may have hired an unscrupulous consultant who will (of course) suggest a costly and time-consuming answer.

Consider the apparent problem of generating highly granular forecasts, such as by customer/item for a manufacturer, or store/item for a retailer. There can be millions of time series at this most granular level. It may appear that we need to forecast all of them. So we buy terabytes of storage and the fastest processors to be able to model and forecast each of these millions of series. But did we really have to do all this? Is this approach really going to give us the best answer to the ultimate business problem, which is meeting customer demand in a cost effective manner?

A manufacturer who fulfills customer demand out of network of distribution centers (DCs), probably doesn’t have to care about the individual demands of individual customers. As long as forecasts at the DC/item level are accurate enough to keep an appropriate level of inventory in each DC, who cares what individual customers are demanding? Unless a customer dominates the demand for an item at the DC (consuming a high percentage of the DC volume in that item), there may be no good reason to try to forecast that customer/item combination.

You could make a similar argument for replenishable items in retail, where multiple stores are supplied from a central warehouse. Instead of forecasting each store/item combination every week, just forecast the total demand for each item at the warehouse level, and use inventory policies (min/max, 2-bin, etc.) to replenish the store shelves. As long as you have forecasted well enough at the warehouse/item level, you should not have to worry about store/item forecasting.

The point it, don’t waste time solving a difficult problem (like customer/item or store/item forecasts) if it doesn’t need to be solved. Not only are highly granular forecasts going to be less accurate than forecasts at an aggregated intermediate level (like DC/item), they take considerably more time and resources to generate.

So make your life easy. Whenever possible, eliminate the need to do forecasting.