MASTERING LUMPY DEMAND - By EUGENIO CORNACCHIA and JOSEPH SHAMIR

Overview – The Growing Problem of Lumpy Demand

Many companies these days are dealing with lumpy demand. What’s causing the problem? To start, product proliferation. Customers have far morechoices than ever before. This divides demand into smaller and smaller buckets. But the real issue for most companies is that they are replenishing more frequently. Faster and more frequent replenishment cycles are creating smaller demand buckets, leading to lumpy demand and many more slow movers.

So the “long tail of demand” that we will describe in this paper now stretches across far more products than ever before. Why is lumpy demand a problem for supply chain managers? Lumpy demand is much more challenging than high volume mainstream business. Demand becomes more unpredictable. Supply chain noise increases. Demand signals are harder to read. As you would expect, forecasting and inventory management in this environment is more challenging, especially since traditional inventory management systems weren't designed for lumpy demand.

As Lora Cecere of AMR Research in “Of Long Tails and Supply Chains” (January 4, 2008), concludes:

"The deterministic, replenishment logic traditionally found in advanced planning and scheduling (APS) technologies are not a good fit for the long end of the tail. Bottom line, in this scenario, traditional inventory techniques—safety stock logic based on normal demand distribution — just don’t work."

The result is that inventory mixes are wrong. Some products are being over served. Others are being under served. Managers find that they have too many of the items they don’t need and not enough of the items most in demand. This situation creates a significant opportunity for companies to improve both their top and bottom line. Later we will discuss how, but first let’s take a closer look at lumpy demand and the long tail that causes it.

What’s Causing “Long Tails” to Grow? - The “long tail’ concept was originally developed to describe the fractured demand commonly seen in Internet companies like iTunes. But companies with more traditional supply chains also experience the long tail effect, along with lumpy, unpredictable demand patterns. Three main business trends are causing long tails to grow:

• Product proliferation – Most companies have more products and product variants than 30 years ago. Generally, the percentage increase in the number of SKUs is much larger than the increase in sales, significantly decreasing sales per SKU. So product proliferation translates into more variability at the individual SKU level. Many more products means many more buckets between which to split the demand.
• More frequent replenishment and more granular forecasting – While more frequent deliveries allow companies to be more responsive and react to changes in demand more easily; they also mean shorter time periods are being served. Shorter time buckets means living with more demand variability. The same SKU may look like a “fast mover” (relatively stable demand) if demand behavior is observed in monthly buckets, but looks more like a “slow mover” if observed in weekly buckets, and typically appears “lumpy” at the daily level.
• A total supply chain focus - There is more collaboration between manufacturers, distributors and retailers and more vendor-managed inventory (VMI) now. Many manufacturers who used to focus on keeping their big regional distribution warehouses stocked are now minimizing out-of-stocks further downstream, often at the end node of demand. As the focus of replenishment planning shifts from the primary distribution centers to secondary distribution centers and retail shelves,demand is increasingly disaggregated into smaller demand streams. Demand variability and slow-moving behavior increases.

These trends are creating long tail demand for many companies, and there is also a business objective trend that makes this problem even more challenging:

• Higher customer service expectations – Companies are striving for higher levels of customer service. Whether measured by “perfect order fill rates” or “moment of truth” retail shelf product availability, customer expectations and industry standards have been steadily rising. This helps explain why inventories have not gone down more significantly despite the advances in supply chain planning and execution achieved during the last 10 years.

So several trends, but especially shorter replenishment cycles, are causing longer tails at most companies. Let’s look at a few examples, but first let’s define more precisely where the long tail begins.

Where the “Long Tail” Begins - From a supply chain perspective, the tail starts where demand becomes ”intermittent”. This commonly accepted threshold between “head” and “tail” defines the line between fast and slow-moving products. This is the point where there are at least as many zero-demand periods as non-zero demand periods, or a 50% probability of having a zero demand time bucket.

If demand frequency is distributed according to a Poisson distribution, a 50% probability of zero demand corresponds to an order frequency of approximately 0.7 line-orders per time bucket. Since real demand is usually over-dispersed with respect to Poisson, we can be sure that there are more zero-demand periods than non-zero demand periods below the 0.7 line-orders per period threshold.

Long Tail Examples at “Brick and Mortar” Companies 

Here are a few examples of long tails from actual Fortune 1000 companies.

The left graph plots SKUs for a large automotive aftermarket parts business. Even where a weekly time bucket has been used, 98% of the SKUs and 62% of the sales revenue are in the tail. This may not be so surprising. After all, aftermarket parts businesses are known for lots of SKUs. Nonetheless, the shear size of this long tail is impressive. In the example on the right, what may be more surprising is that even for a globally branded “fast moving” consumer packaged goods company, the long tail consumes 86% of the SKUs and nearly half (46%) of the revenue. This is a long tail for a business that most people would not normally consider “long tail” prone. One important reason is that, like many consumer goods companies, this company now has a replenishment control frequency that is daily, not weekly. The majority of our clients no longer work with longer time buckets, either because of the need for rapid demand sensing or to support a high frequency replenishment process. We studied several large brick and mortar companies and found similar situations elsewhere. Here are some sample results, including the two companies depicted above.

Notice that even at the company with the smallest tail, a food and beverage company with very strong brands, nearly half (44%) of the SKUs and more than one-third (36%) of the revenue was in the tail. For everyone else the situation was much worse. Most every company, these days, seems to have a long tail. Let’s look at the implications.

Problem #1 - Intermittent Demand - When we defined the tail previously, we said a basic implication of dealing with SKUs in the tail is that the demand is lumpy and/or intermittent. There are many zero-demand periods. In addition, demand variability is very high and demand probability distributions are highly skewed. This “intrinsic variability” has very little to do with forecast accuracy. For instance, if an SKU has average sales of one unit every ten days, and the daily forecast is close to 0.1, the forecast is quite accurate overall. The problem is that it is very difficult to predict in which day the next demand will occur. So any attempt to improve “forecast accuracy” will likely be costly and useless, since it will not lead to any relevant reduction in demand variability. A much more effective approach would be to analyze the full probability distribution and create a reliable statistical description of how demand behaves in the right end tail.

Problem #2 - Traditional Systems Weren't Designed for the Tail - In a lumpy and/or intermittent demand environment where demand variability is high and the demand distribution is skewed, classic demand and inventory models do not perform well.

As seen in the sample data on the right, the tail creates a significant mismatch between the requirements for high service levels and traditional enterprise solutions.

As Lora Cecere of AMR concludes: “If the products have a skewed distribution and it is your desire to meet a higher service level, then the deterministic technologies of APS and ERP are not designed to meet your needs.”

Problem #3 – The Bullwhip Effect - A final issue in the tail arises from the bullwhip effect. Even if advanced, “integrated” replenishment logic is used, which shouldn’t in principle create any bullwhip effect, the presence of many slow-moving items produces significant gaps between what is planned and actual results. Once users realize that inventories are out-of-line and too many out-of stocks occur, they usually make extensive manual interventions, often over-reacting and creating overstocks. Hence, inadequate technology for managing the slow-moving and lumpy items in the long tail causes performance gaps that then lead to inappropriate manual intervention. This manually induced bullwhip effect shifts working capital from active stock to slow or even dead stock.

The Bottom Line - Misaligned Inventory

Let’s recap. Lumpy demand has an intrinsic variability that traditional enterprise systems can’t handle, creating a very “nervous” and unstable operating environment. The inventory mix, and therefore the service levels, across the network get out of whack, with neither reflecting business objectives. Planners try to compensate for this lack of performance by manually adding lots of inventory, which typically accumulates in the tail. Meanwhile the faster movers in the “head” are underserved. Without good statistical demand and inventory models, there is neither the right inventory, nor the desired service levels. The bottom line business impact is simple. The tail consumes lots of working capital, without delivering the desired service levels.

Here is what the situation looks like when you plot optimized stock versus current stock:

For the automotive aftermarket company represented in the graph on the left, you can see that inventories are largely misaligned, with a general tendency to carry overstocks across the entire product range. In the right graph, a well-known branded CPG vendor shows another common problem: not only are many items clearly over-stocked, but many others are also heavily under-stocked. They are not delivering an acceptable service level and are causing a high percentage of lost sales and margin. In other words, they are underserving many high margin items that should be receiving more inventory. They are failing to assure high service levels where there is the highest profitability. Both companies’ inventories are seriously misaligned.

Successfully Managing Long Tail Inventories

Successfully managing long tail inventories requires two things that are simple to articulate, yet harder to execute. Companies with long tails need:

• accurate demand and inventory models to support reliable service level and inventory management
• disciplined processes that eliminate manual interventions and bullwhip behavior

Successful inventory management in a long tail environment does not require sophisticated optimization. The optimization can actually be rather straightforward, once the demand and inventory behavior has been accurately and robustly modeled. The problem with many software packages is that they try to take shortcuts that lead to incorrect modeling and therefore incorrect results. If shortcuts lead to poor models of demand and inventory behavior, no amount of optimization can solve the problem. Many inventory optimization software solutions were primarily designed to solve strategic supply chain design problems. These are large combinatorial problems that require high scalability and therefore also dramatic simplifications. In order to solve the problem, they either completely ignore or grossly simplify the structure of the demand distribution tail. The optimization is often based on quite sophisticated optimization algorithms, but by oversimplifying the demand model they are actually optimizing a model that does not match the real world behavior with a sufficient degree of detail.

This approach may work well for supply chain network design, when the main purpose is answering strategic questions such as “How many DCs should I use?” or ”Where should they be located?” But for other questions instead, such as “Where should I set the Push/Pull Boundary for an assembly or component?” or “How do I take advantage of Risk Pooling?”, they may not find the right answers when dealing with “long tail” situations.

Normal Distributions Don’t Work in the Long Tail

One shortcut that most applications still use is to assume that demand conforms to a normal distribution, ignoring the huge difference in the right hand tail behavior among different distributions. A commonly stated justification is: “In practice all tails are more or less the same”. In reality, this isn’t true at all.

Let us recall a paragraph, written by Bob Brown in “Advanced Service Parts Inventory Control”, and also quoted by Lora Cecere:

"It is a coincidence, but in the range of 90% to 95% probability of ...(not having)... a shortage in the next replenishment cycle, most of the common distributions require very nearly the same value for the safety factor, so the distinction among alternative forms of the distribution is not so critical. But if you want to aim for 98% to 99.5% service, there can be a very large difference between the safety stock theoretically required under a skewed distribution and that required if the forecast errors were normally distributed."

The figure below shows exactly what Bob Brown was describing. We compare a normal distribution with a Negative Binomial, a generalized Poisson, and the proprietary distribution used by ToolsGroup. In this example, by the way, the normal behavior has been “improved” by adding to the zero point the cumulative probabilities of all negative demand values, which clearly wouldn’t make sense in the real world. Nevertheless it begins to diverge starting around 90% in-stock probability or even lower.

The above probability distributions refer to a lead-time (LT) demand with an average of 5 units and a standard deviation of 2.7 units. So this can’t even be considered a pure case of “lumpy” demand, since the probability of zero demand is less than 4%. Therefore this example actually sits in the head, not in the tail. Nevertheless, the “right hand tails” rapidly diverge for “in-stock probability” above 90% and show huge differences for “in-stock probabilities” close to or above 99%. Of course, such differences are even larger for slower-moving SKUs. So the bottom line is that items in the long tail cannot be treated with a normal demand distribution.

Another Shortcut: The Two Model Approach

In the last decade or so, several software vendors have realized the serious inadequacy of using the normal demand distribution for modeling slow-moving item behavior. For those who have tried to solve the problem, the most common remedy has been to create a second model, normally based on one of the many derivations of Croston’s method,explicitly dedicated to “slow movers”. Croston’s method was originally created in the 1970’s and is based more on empirical considerations than on a sound scientific basis. It has proven useful to avoid crazy forecast behaviors of intermittent demand items, but nothing more than that. It doesn’t provide any reliable way to set safety stocks and target a specific service level, particularly for slow moving items.

Finally, the two model approach requires an “a-priori” classification of the SKU’s, separating the “normal” ones from the “lumpy” ones. Products are split into two separate categories, making use of two entirely different modeling techniques for the two categories. All observed real world phenomena present an infinite variety of different behaviors, not just two extremes. Inventory modeling is no exception to this rule.

Succeeding in a Long Tail Environment

It is evident that customer demand has become more geographically scattered, occurs in smaller quantities and is for more specialized products. Nobody can deny this is a clear challenge. Can we really understand the implications on the inventory behavior of a large number of items well enough to achieve “service level excellence”. Using the right technology, the answer can be yes. If you want to fine tune your inventories to achieve very high customer service levels and the important benefits it brings, you have to master the shape of the demand distribution and the right-hand tail. You must avoid modeling shortcuts and approximations that can prevent you from getting close to your target.

Achieving a high service level in the long tail world requires inventory modeling technology that can master demand behavior and probability distributions across a very wide variety of demand patterns requires the following capabilities:

• A very reliable demand modeling technology that automatically adjusts all relevant statistical parameters, in a seamless way across a wide range of SKU behaviors, understanding all elements of demand uncertainty and considering not only forecast errors, but also the “intrinsic demand variability” caused by a certain order-line frequency and order-line size distribution
• Advanced inventory modeling that, by eliminating many of the gross approximations of traditional inventory management theory, can provide very reliable descriptions of statistical inventory behavior.
• Logic for demand signal propagation that combines the superior capabilities of both demand modeling and inventory modeling, to model the impact of replenishment parameters and constraints at each echelon of the supply chain.
• A solution that assures very good adherence between plans and actual performance, reducing the need for emergency interventions, and thus allowing disciplined inventory and service planning processes.