The Impact of Forecasting Improvement on Return on Shareholder Value
John T. Mentzer; Journal of Business Forecasting

Improving Salesforce Forecasting
Mark A. Moon and John T. Mentzer; Journal of Business Forecasting

Benchmarking Sales Forecasting Management
John T. Mentzer, Carol C. Bienstock, and Kenneth B. Kahn; Business Horizons

Webcast: Three Keys to Successful S&OP
Wednesday, October 26, 1pm Eastern, 10am Pacific
Principal Speakers
Tim Vaio, Managing VP,
Hitachi Consulting
Seema Phull, Director Process & Technology,
Enterasys Networks
Click to Register

Conference:
The Dynamic Supply Chain-- Strategies for Implementing Sustainable Competitive Advantage
Nov 17-18, NY
Hear Glen Margolis, Founder & EVP Steelwedge Software, Carol Ptak, Former APICS CEO and Author, Others
Learn more

Previous Webcasts--
State-of-the-Art Sales Forecasting Management
September 13, 1pm Eastern, 10am Pacific
Principal Speakers -
Professor J. Tom Mentzer, University of Tennessee
Todd Jones, Sr. Director
Sales Operations, QLogic
Click here to view recording

Forecasting and Consensus Planning in a Rapidly Changing Environment
June 15, 2005, 1pm Eastern, 10am Pacific
Principal Speaker -
Christine Pfefferle, Director of Global Demand & Order Management, Tellabs
Click here to view recording

Extending SAP for Global, Integrated S&OP
May 25, 2005
Stu Reekie, Air Products
Click here to view recording

Enterprise Forecasting--Dubunking the Myths

By Craig Burkhead, Chief Scientist, Steelwedge Software

In twenty years of working enterprise planning, I've seen and heard more than my fair share of strange claims by vendors, and even stranger statements by prospects.  The plan for this month, is to bring up just a handful of the more common myths or misconceptions that I've heard over the years.

Of course, as they say, YMMV * ,   so   these are generalities .  Perhaps you really do have 98% corporate accuracy, 5 min lead times and salespeople who don't game their numbers.  For the rest, hopefully   you'll  find this worthwhile …   * Your Mileage May Vary

1 – ”You Can't Use Early Sales Opportunities in a Planning Process, the Data is a Mess”

Early stage sales opportunities are like Delphic oracles – vague and wrong, but in this case, they're worth hearing. There are four challenges to mapping early stage opportunities to your forecast process.

First, for many companies there's a level of detail issue. At the early stages, opportunities often are not configured, and therefore the exact product list isn't known, or defined in terms of ‘product line' or ‘product family'.

Unconfigured opportunities present their own unique challenges. One best in class approach is to use a variation on planning BOMs, called statistical BOMs (or sBOMs), which are assemblies of items with statistically determined component count (aka attach rates). By using a series of rules to map unconfigured opportunities to their corresponding sBOMs defined by prior configurations, we can adjust component detail forecasts in a statistically meaningful fashion.

Next, the gaming issue. Yes, we all know that salespeople can sandbag, but that's a subject that's better discussed in the answer to another myth in this article.

The final issue revolves around the high degree of uncertainty in early opportunities. Of course they have high uncertainty, but that does not mean they are devoid of value. One approach is to capitulate – use early stage opportunities as qualitative information. Simply provide a mechanism, such as an alert, when these early stages are on the distant radar horizon. Another approach is to treat stage-weighted opportunity revenue in regression. If a reasonable correlation is present, then your forecast will be positively affected. If the number of early opportunities drops, you'll see a corresponding drop in forecast as advance warning.

2 – My Algorithms Can Beat Up Your Algorithms

As they say, there are lies, damned lies and statistics. Forecasting companies routinely argue over ‘bake-offs', or forecasting competitions that pit one statistical methodology or algorithm against another. If you examine the state of the art in forecasting techniques over the last several decades, there have been only minor refinements. Most time series and regression techniques have been around for centuries.

So where has the research gone? Largely in two areas: integration and collaboration.

Only in the last several years has ubiquitous computing provided the infrastructure to allow forecasting systems to access data from beyond the thin veil that separates silos. The best algorithm at forecasting demand will pale
in comparison to one which can see and react to pipeline opportunities.
The ability to integrate disparate data sources, recognize and compensate for gaming, using heuristics to adapt to patterns represents the new state of the art.

Secondly, the best forecasting methodology is the one which recognizes that the best business intelligence is institutional intelligence.By providing a mechanism to allow a wide variety of users to contribute and add value to the planning process, recording both their quantitative (numbers) and qualitative (notes) inputs, and then grading their impact on performance, today complex manufacturers will become more productive and profitable.

3 – ”Demand Forecast…Sales Goals; Same Thing, Right?”

A disconcerting and common error. A demand forecast is the result of a statistically complex algorithm, based upon actual shipments or demand, while a sales goal is but a gleam in your Sales VPs eye.

The two are linked, but with different intent. A forecast ideally is generated close to the level of detail in which you operationally require the information. If you sell products, then you need a product forecast. A corporate forecast simply defers the problem to how to disaggregate the numbers.

A sales goal is the result of a plan, which is the result of a process. The goal probably started as the sum of many forecasts, then underwent a revision process at aggregate. Sales goals are typically managed in a top-down fashion, where adjustments of ‘increase third quarter by 20%' are not uncommon.

For a forecast to be accurate, it shouldn't have any bias, i.e. half the time over, half under forecasting. If a sales goal came in below actuals half the time, heads would roll.

World class planning processes should use both demand forecasts and sales goals to get world class numbers.

4 – ”We Can't Ask Sales to Collaborate for our Plan, They'll just Game the Numbers'”

Collaboration is simply in concept – ask the question, record the answer, measure the effect and reward or punish accordingly. This works with sales, as it works for any other contributor, with only two minor changes: First, if sales people are doing their job, they're selling, not mucking with forecasts, so make it easy to get their input. Companies like SalesForce.com are doing a wonderful job on this front; Second, if they do sandbag their numbers, by using regression to compare their estimates against actuals, we can compensate accordingly. Rather than suspecting that sales people are gaming, we can demonstrate in a statistically sound fashion that this is the case, and to what extent. Don't fight gaming or bias, its part of human nature; instead compensate statistically to create a better plan.

5 – ”We have a Corporate Accuracy of 98%, What could Possibly be Wrong?'

Actually, quite a lot. Accuracy at aggregate, while desirable, does not automatically translate into accuracy at detail.

If you're producing apples and oranges, a surplus of apples won't help filling orders for oranges. Companies need to measure accuracy at the level that they can operationally respond. For some, that's at product, possibly product-location, perhaps even product-location-customer. Once measured, these values can be aggregated, but the level of measurement must always be kept in mind.

Second, consider when the measurement was made. Your metric should take into account the ‘time-fence' (typically lead time) by which you are able to operationally respond to that forecast. If I predict that next month you'll sell 100 units of something with a two month lead time, I may be 100% accurate, but I'm not helping you.

Finally, check your metrics. Some planners use metrics that make them look better (now why would they do that?) A standard formula for percent error is PE = (forecast – demand) / demand. What do you do when demand is zero? Do you simply count this as 0% error – that's cheating. In this case, the correct answer is PE = 0 only if both forecast and demand are zero, otherwise, it's 100%. And, please, don't use PE = (forecast – demand) / forecast. That's measuring accuracy against your forecast, not demand.

Forecast accuracy is critical for world class planning; just know what you're measuring.