Why is accurate demand forecasting so difficult, and is progress on the horizon?

Will Green is news editor of Supply Management
8 June 2023

It’s the stuff of nightmares for procurement and supply chain professionals. You wake up in a cold sweat with one thing on your mind: what are you going to do about the $10bn worth of inventory sitting idle in a warehouse?

For some, this is more than bad dream, and several high-profile companies have been caught out by erroneous forecasting.

While predicting demand is a challenge for any industry, it is FMCG that has found itself at the battlefront recently. Last year, Walmart reported excess inventory of $1.5bn, but this sizeable figure was dwarfed by Nike which announced it had excess goods worth $9.7bn sitting in warehouses.

Walmart was among the firms that had chartered ships to import sufficient goods in the face of pandemic-related supply disruptions. CEO Doug McMillon said at the time: “If we can just wave a magic wand, we’d make it go away today.” Meanwhile, Nike finance chief financial officer Matthew Friend said the company planned to “tighten up” its procurement and “liquidate excess inventory more aggressively”.

The turbulent times we live in are the cause of retailers’ problems, according to consultancy ADS Procurement & Supply managing director Adam Smith. “Given the Covid-induced global supply chain issues, organisations moved from just-in-time to just-in-case inventory management approaches,” he says. “Plus, there was a boom in consumer spending post-Covid lockdowns, so increasing inventory seemed an obvious and sensible approach. Unfortunately, many organisations did not anticipate that price inflation would reach the level it has, resulting in a significant dampening of demand for non-essential goods.”

That move to just-in-case is perfectly understandable when you consider the perils of under-ordering and stockouts, which Walmart, Nike and Adidas are likely to have been anxious to avoid. Research by Bain & Company found stockouts to be “the most annoying of all episodes” that shoppers commonly encounter in grocery. “Stockouts are also a high-risk situation in other corners of retail, of course, potentially causing the disappointed shopper to transfer their allegiance elsewhere,” the report says. “Supply chain weaknesses might mean that a consumer doesn’t even bother to start their shopping journey with a certain retailer.”

The role of data

Data has certainly adopted an evermore important role since the 1960s, and a whole host of software programmes is now available to crunch the numbers. Research suggests the market for demand planning software is only growing. The market was worth $3.97bn in 2022 but is expected to grow at a rate of 10.3% a year until 2030, when it will reach an estimated value of $8.68bn, according to Grand View Research. Given the march of technology, you would expect the accuracy of forecasting to have improved. But a survey of warehouse operators by ProGlove in January found just 39% felt they could accurately predict trends and patterns for the 2022-2023 holiday season, while 51% said forecasting demand was their biggest inventory management concern.

A 2019 report on forecasting techniques by academics at the Wharton School, part of the University of Pennsylvania, found predictions tended to be swayed by people making “adjustments” – in other words, tinkering with the results. Quoting Leonardo de Vinci, the report says: “The greatest deception men suffer is from their own opinions.” The study reviewed research covering forecasting in areas such as economics, energy, transport and population, and concluded: “Have the substantial improvements in knowledge on forecasting over the past half century led to more accurate forecasts in government and business? The short answer is no. Instead, forecast accuracy appears to have declined over the period.”

However, in certain areas – weather, political elections, crime, medicine and engineering – the report did concede accuracy “appears to be improving”. Smith points to a lengthening of processes between source to receipt, “thus the ability to react quickly to demand swings remains difficult”.

“While technology has improved the ability to communicate, the transportation of physical goods has become more challenging due to multiple factors,” he says. “These include manufacturing instabilities, logistics delays, trade barriers due to geopolitical events and economic policy, and a seemingly reduced workforce.”

Information and technology

Perhaps a little over-optimistically, researchers at the Singapore Institute of Manufacturing Technology found forecasting is improved when firms share retail data, and claim this can be done without commercial compromise. “Although it is known that sharing information improves the overall efficiency of a supply chain, information such as pricing or promotional strategy is often kept proprietary for competitive reasons,” the report says. “It is shown that simply sharing the retail-level forecasts – this does not reveal the exact business strategy, due to the effect of omnichannel sales – yields nearly all the benefits of sharing all pertinent information that influences FMCG demand.”

Yet despite the technology available, “supply chain leaders haven’t done themselves any favours by clinging to manual systems and antiquated software”, according to McKinsey. Research by the consultancy found 73% of supply chain executives were using spreadsheets for planning, while the second most popular method was SAP Advanced Planning and Optimization, which it described as a “popular but antiquated supply chain planning application that SAP introduced in 1998 and will stop supporting in 2027”. It is evident that companies hang on to older systems despite the substantial benefits new technology offers “because of the time and money needed to replace them,” says the report.

Based on its research, McKinsey has crystallised best practice when implementing new planning software into three key stages:

  • Process redesign: understanding pain-points, defining future processes and deciding on performance indicators,
  • Vendor selection: outlining high-level business requirements, clear evaluation criteria, and two or three must-have use cases, and
  • Implementation road map: prioritise features, divide implementation into ‘sprints’ and ensure user buy-in with testing and training.

Over reliance on technology

“When planning IT implementations fail, 60% of the time it’s in one of three ways: they aren’t completed on time, are over budget, or don’t deliver the expected outcomes,” says McKinsey. “The failings are indicators that processes are poorly designed or lack needed capabilities, or the change was poorly managed.” Smith adds a note of caution to this, saying: “Many organisations have perhaps been guilty of trusting technology too much, and while it can play a key role in streamlining the administrative burden, it cannot react particularly well to unusual and unexpected circumstances.”

It would appear that despite all the advantages promised by technology, there is no replacement for boots on the ground. “Put simply, more resources, either in the shape of systems or people, need to be deployed to monitor and forecast demand patterns,” says Smith. “There must surely be the capability to scan the internet for historical statistics and news stories, and identify patterns that preceded previous demand slumps.

Perhaps AI could be put to work to absorb this data and create triggers to predict future demand patterns, with this intel then used by supply chain managers and planners.” This, he adds, connects with the talent shortage procurement has undergone for many years. Because, “if we are going to remodel our supply chains, then people need to be recruited to drive this activity”.

CIPS Knowledge
Find out more with CIPS Knowledge:
  • best practice insights
  • guidance
  • tools and templates