We use cookies, including third-party cookies from Google to serve personalized ads through AdSense, to operate this site and understand how it is used. By continuing to browse, you accept this use. See our Privacy Policy and Terms of Use for details, including how to opt out of personalized advertising.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
    Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
    23 Min Read
    Flat editorial illustration: The article is about automating the collection, preparation, and delivery of marketing data across p
    From Data to Dashboards: 5 Marketing Reporting Automation Software Picks
    12 Min Read
    Flat editorial illustration: The article contrasts passive follower totals with a behavioral path from attention and trust to cli
    Creator Analytics Beyond Follower Count: Seven Metrics That Measure Audience Behavior
    11 Min Read
    Flat editorial illustration: The article connects AI and marketing analytics with managerial judgment grounded in consumer behavi
    How an Online MBA in Marketing Connects AI and Data Analytics
    11 Min Read
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers -- AI-generated illustration
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers
    7 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: The Perils of Forecasting Benchmarks
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Data Management > Best Practices > The Perils of Forecasting Benchmarks
Best PracticesPredictive Analytics

The Perils of Forecasting Benchmarks

mvgilliland
mvgilliland
3 Min Read
The Perils of Forecasting Benchmarks
Illustration generated with Qwen Image.
SHARE

Benchmarks of forecasting performance are available from several sources, including professional organizations and journals, academic research, and private consulting/benchmarking organizations. But there are several reasons why industry forecasting benchmarks should not be used for setting your own forecasting performance objectives.

1) Can you trust the data?

Are the numbers based on rigorous audits of company data or responses to a survey? If they are based on unaudited survey responses, do the respondents actually know the answers or are they just guessing?

2) Is measurement consistent across the respondents?

More Read

Is Performance Management Pushed or Postponed in an Ailing Economy?
Is Performance Management Pushed or Postponed in an Ailing Economy?
Big Data Accountability (Part 1)
NPV Considerations for Open Source Big Data Technologies
Simple Tools for Building a Recommendation Engine
Please put the shower curtain inside the bathtub!

Are all organizations forecasting at the same level of granularity, such as by product, customer or region? Are they forecasting in the same time interval, such as weekly or monthly? Are they forecasting by the same lead time offset, such as three weeks or three months in advance? Are they using the same metric? It is important to note that even metrics as similar sounding as MAPE, weighted MAPE, and symmetric MAPE can deliver very different values from the same data.

3) Finally, and most important, is the comparison relevant?

Does the benchmark company have equally forecastable data?

Consider this worst-case example:

Suppose a benchmark study shows that Company X has the lowest forecast error. Consultants and academics then converge on Company X to study its forecasting process and publish reports touting Company X’s best practices. You read these reports and begin to copy Company X’s best practices at your own organization.

However, upon further review using FVA analysis, it is discovered that Company X had very easy-to-forecast demand, and it would have had even lower error if it had just used a naive forecast. In other words, Company X’s so-called best practices just made the forecast worse.

This example is not far-fetched. Organizations at the top of the benchmark lists are probably there because they have the easiest-to-forecast demand. Many organizational practices, even purported best practices, may only make the forecast worse.

Benchmarks tell you the accuracy that best-in-class companies are able to achieve. But…they do not tell you whether their forecasting environment is similar to yours or worthy of your admiration. Without that information, industry benchmarks are largely irrelevant and should not be used to evaluate your performance or set performance objectives.

 
Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

Flat editorial illustration: The article connects downstream report failures, data-quality gaps, and conflicting metric definitio
Signs Your Data Architecture Needs an Outside Review
Big Data Exclusive
Data-Driven Pultrusion: Best GFRP Rebar Machines & ROI
Data-Driven Pultrusion: Best GFRP Rebar Machines & ROI
Big Data Exclusive
Flat editorial illustration: The article contrasts tools for gathering fresh interviews with tools for analyzing existing qualita
10 AI Market Research Tools That Turn Qualitative Data into Insights
Artificial Intelligence Big Data Exclusive
Flat editorial illustration: The article compares proxy MCP servers by what an AI agent can do through them: manage proxy account
Proxy MCP Servers: 7 Options and What Each Lets an AI Agent Do
Artificial Intelligence Big Data Data Collection

Stay Connected

1.2KFollowersLike
33.7KFollowersFollow
222FollowersPin

You Might also Like

Big Analytics: Closing the 'Clue Gap' with Big Data
AnalyticsCulture/LeadershipData MiningData QualityData VisualizationData WarehousingPredictive AnalyticsText AnalyticsUnstructured Data

Big Analytics: Closing the ‘Clue Gap’ with Big Data

4 Min Read
Demand for Data-Savvy Cybersecurity Professionals Grows In 2021
Big Data

Demand for Data-Savvy Cybersecurity Professionals Grows In 2021

9 Min Read
Intuit uses attribution analysis, behavioral targeting, and funnel-aware messaging
MarketingPredictive Analytics

Intuit uses attribution analysis, behavioral targeting, and funnel-aware messaging

9 Min Read
How Nate Silver Won the Election with Data Science
AnalyticsData MiningPredictive Analytics

How Nate Silver Won the Election with Data Science

10 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots
Artificial Intelligence for eCommerce: A Closer Look
Artificial Intelligence for eCommerce: A Closer Look
Artificial Intelligence

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-26 SmartData Collective. All Rights Reserved.
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?