Is your company using demand forecasting in your planning process? And are you happy with those results? Based on our interviews with hundreds of companies worldwide, we have found that among midsize manufacturers and distributors, many still use error prone spreadsheets or forecast based on historical sales data. Among these firms, nearly all tell us that their results aren’t as good as they want. More importantly, they are still making costly inventory and purchasing mistakes based on bad demand inputs. We have noted three KPI’s that will help any organizatio
Is your company using demand forecasting in your planning process? And are you happy with those results? Based on our interviews with hundreds of companies worldwide, we have found that among midsize manufacturers and distributors, many still use error prone spreadsheets or forecast based on historical sales data. Among these firms, nearly all tell us that their results aren’t as good as they want. More importantly, they are still making costly inventory and purchasing mistakes based on bad demand inputs. We have noted three KPI’s that will help any organization immediately improve the utility and value of its forecasting.
Accuracy is not the whole story…
Traditionally, forecasters are primarily concerned with improving accuracy. The industry standard metric is the “MAPE,” an acronym for the impressive sounding mean absolute percent error. While a lot of traditional forecasting focuses on reducing this measure, it really tells us nothing about two other key issues:
1) Is our situation really forecastable to begin with?
2) What do we really need to measure to get a good result?
Without all three pieces of information, your forecasting results are likely hard to explain and you may be spending a lot of time worrying about data that isn’t helping your accuracy.
Demand Forecasting in Halo 15.1 can immediately improve forecasting effectiveness by monitoring accuracy, forecast-ability and data cost-effectiveness in a prebuilt dashboard. Our customers generate more believable and explainable forecasts in hours, identify root causes of missed forecasts, and reduce the risk of bad decisions based on bad forecasts by 50 to 90% (our white paper provides more detail).
Three measures to improve your forecast quality using Halo 15.1
Forecast-ability – Before measuring accuracy, we should really look at whether we have adequate data to even try a forecast. A simple approach is based on the Coefficient of Variation (or “CoV”), a measure of how dispersed data across a frequency distribution. If the data are too dispersed, our forecast precision will be low. But this is not always bad. In many scenarios, for example, we are simply trying to avoid extreme product shortfalls – and even a forecast with large dispersion of sales values can help us identify those situations.
Halo 15.1 provides a simple but powerful Forecast-ability visualization. The plot shows the CoV calculated with our sales distribution data compared with our forecast errors. To read it, the error values are plotted on the vertical axis and the CoV values are plotted on the horizontal axis. You can see that, on this raw file of grocery chain sale data, as the CoV increases so does the error. In effect, the more volatile our sales, the harder it is to forecast.
We can learn more from this chart. The line through the chart represents a base case forecast result using a simple moving average. Anything above the line tells us where our forecast approach was better than a moving average. Anything below the line tells us where we did worse than expected. With a little further analysis, we found that we were better with forecasts against mature products vs. new products – which naturally had fewer data points and were still being positioned in the market with different promotions and pricing campaigns.
We can create a KPI called, by industry terminology, our Forecast Value Add (FVA). This is a measure of how much benefit you get from trying to improve the forecasts with managerial adjustments, external data use and other tactics. The approach generates pre-post forecasts using historical data combined with/without other factors. By monitoring subsequent performance, the model indicates what factors are worth investing more resources in – or, just as valuable, which ones may be ineffective, misleading, or really not worth the cost. You can then monitor those results by seeing how our CoV points fall “above the line” on our plot.
Our takeaway with the CoV graph is that we need to segment our product forecasts into meaningful groups based on our data quality, the expected volatility, and the precision required of the forecast for our specific business decisions. For a new product, I may be willing to accept a less forecastable result as long as there is some patterns in our data.
Now back to accuracy
Halo 15.1 actually provides 10 different types of accuracy metrics. The MAPE is easily interpreted and broadly accepted as a good metric. The goal is to manage forecast planning to continuously try to reduce the MAPE, for example, by introducing management assumptions about the impacts of future events ranging from supply chain disruptions to promotional campaigns.
Still, how good is good? Is an 80% accuracy level any good? Again, it all depends. Some industries, regions and products are simply more forecast worthy than others. Is it any good to be the best at forecasting in an highly volatile area where everyone’s models stink? You need a relevant basis of comparison if you are going to use your forecast to try to build a more competitive market position.
Halo has worked to create a benchmark of forecasts results across specific industry, company size, product segments and includes an option to compare your results against a peer set via Halo 15.1.
This is of real value when you are moving to a new forecasting process and want a read on your initial performance.
Demand forecasting is an incredibly valuable process when done effectively. Too many companies are still mired in poor methods – and a lack of awareness about the key factors that drive forecast usability. Forecast-ability and comparative accuracy our two of those factors. With Halo’s approach, your supply chain team can plan better and be more confident about their future revenue streams.
