Forecasting With Model Uncertainty: Representations and Risk Reduction

We consider forecasting with uncertainty about the choice of predictor variables. The researcher wants to select a model, estimate the parameters, and use the parameter estimates for forecasting. We investigate the distributional properties of a number of different schemes for model choice and parameter estimation, including: in‐sample model selection using the Akaike information criterion; out‐of‐sample model selection; and splitting the data into subsamples for model selection and parameter estimation. Using a weak‐predictor local asymptotic scheme, we provide a representation result that facilitates comparison of the distributional properties of the procedures and their associated forecast risks. This representation isolates the source of inefficiency in some of these procedures. We develop a simulation procedure that improves the accuracy of the out‐of‐sample and split‐sample methods uniformly over the local parameter space. We also examine how bootstrap aggregation (bagging) affects the local asymptotic risk of the estimators and their associated forecasts. Numerically, we find that for many values of the local parameter, the out‐of‐sample and split‐sample schemes perform poorly if implemented in the conventional way. But they perform well, if implemented in conjunction with our risk‐reduction method or bagging.     

Neural Network Earnings per Share Forecasting Models: A Comparative Analysis of Alternative Methods

In this paper, we present a comparative analysis of the forecasting accuracy of univariate and multivariate linear models that incorporate fundamental accounting variables (i.e., inventory, accounts receivable, and so on) with the forecast accuracy of neural network models. Unique to this study is the focus of our comparison on the multivariate models to examine whether the neural network models incorporating the fundamental accounting variables can generate more accurate forecasts of future earnings than the models assuming a linear combination of these same variables. We investigate four types of models: univariate‐linear, multivariate‐linear, univariate‐neural network, and multivariate‐neural network using a sample of 283 firms spanning 41 industries. This study shows that the application of the neural network approach incorporating fundamental accounting variables results in forecasts that are more accurate than linear forecasting models. The results also reveal limitations of the forecasting capacity of investors in the security market when compared to neural network models.     

Revenue Management for Intermodal Transportation: The Role of Dynamic Forecasting

We study a joint capacity leasing and demand acceptance problem in intermodal transportation. The model features multiple sources of evolving supply and demand, and endogenizes the interplay of three levers—forecasting, leasing, and demand acceptance. We characterize the optimal policy, and show how dynamic forecasting coordinates leasing and acceptance. We find (i) the value of dynamic forecasting depends critically on scarcity, stochasticity, and volatility; (ii) traditional mean‐value equivalence approach performs poorly in volatile intermodal context; (iii) mean‐value‐based forecast may outperform stationary distribution‐based forecast. Our work enriches revenue management models and applications. It advances our understanding on when and how to use dynamic forecasting in intermodal revenue management.     

Sales Forecasting with Financial Indicators and Experts' Input

We present a method for forecasting sales using financial market information and test this method on annual data for US public retailers. Our method is motivated by the permanent income hypothesis in economics, which states that the amount of consumer spending and the mix of spending between discretionary and necessity items depend on the returns achieved on equity portfolios held by consumers. Taking as input forecasts from other sources, such as equity analysts or time‐series models, we construct a market‐based forecast by augmenting the input forecast with one additional variable, lagged return on an aggregate financial market index. For this, we develop and estimate a martingale model of joint evolution of sales forecasts and the market index. We show that the market‐based forecast achieves an average 15% reduction in mean absolute percentage error compared with forecasts given by equity analysts at the same time instant on out‐of‐sample data. We extensively analyze the performance improvement using alternative model specifications and statistics. We also show that equity analysts do not incorporate lagged financial market returns in their forecasts. Our model yields correlation coefficients between retail sales and market returns for all firms in the data set. Besides forecasting, these results can be applied in risk management and hedging.     

Improving the accuracy and operational predictability of project cost forecasts: an adaptive combination approach

Conducting an early warning forecast to detect potential cost overrun is essential for timely and effective decision-making in project control. This paper presents a forecast combination model that adaptively identifies the best forecast and optimises various combinations of commonly used project cost forecasting models. To do so, a forecast error simulator is formulated to visualise and quantify likely error profiles of forecast models and their combinations. The adaptive cost combination (ACC) model was applied to a pilot project for numerical illustration as well as to real world projects for practical implementation. The results provide three valuable insights into more effective project control and forecasting. First, the best forecasting model may change in individual projects according to the project progress and the management priority (i.e. accuracy, outperformance or large errors). Second, adaptive combination of simple, index-based forecasts tends to improve forecast accuracy, while mitigating the risk of large errors. Third, a post-mortem analysis of seven real projects indicated that the simple average of two most commonly used cost forecasts can be 31.2% more accurate, on average, than the most accurate alternative forecasts.     

Mechanisms to Induce Buyer Forecasting: Do Suppliers Always Benefit from Better Forecasting?

In this study, we consider a supplier's contract offerings to a buyer who may obtain improved forecasts for her demand over time. We investigate how the supplier can take advantage of the buyer's better forecasts and what kind of contracts he should offer to the buyer in order to maximize his profits. We model a natural forecast evolution where the buyer can obtain a more accurate forecast closer to the selling season. We assume there is information asymmetry between the buyer and the supplier at all times in that the buyer understands her demand better than the supplier. Three types of contracts that the supplier can offer are considered: (1) one where a contract is offered before the buyer has a chance to obtain improved forecasts, (2) one where a contract is offered after the buyer has obtained improved forecasts, and (3) a contingent (dynamic) contract which offers an initial contract to the buyer before she obtains improved forecasts, followed by a later contract (contingent on the initial contract) offered after improved forecasts have been obtained. We consider two scenarios: (1) where the supplier is certain that the buyer can obtain more accurate forecasts over time, and (2) where the supplier is uncertain about the buyer's forecasting capability (or forecasting cost). In the first scenario, we show that among the three types of contracts, the contingent contract is always the most profitable for the supplier. Furthermore, using the contingent contract, the supplier always benefits from higher accuracy of the buyer's demand forecasts. In the second scenario, we explicitly model the supplier's level of certainty about the buyer's capability of obtaining better forecasts, and explore how the supplier can design contracts to induce the buyer to obtain better forecasts when she is capable.     

The Value of Weather Information for E‐Commerce Operations

To be efficient, logistics operations in e‐commerce require warehousing and transportation resources to be aligned with sales. Customer orders must be fulfilled with short lead times to ensure high customer satisfaction, and the costly under‐utilization of workers must be avoided. To approach this ideal, forecasting order quantities with high accuracy is essential. Many drivers of online sales, including seasonality, special promotions and public holidays, are well known, and they have been frequently incorporated into forecasting approaches. However, the impact of weather on e‐commerce operations has not been rigorously analyzed. In this study, we integrate weather data into the sales forecasting of the largest European online fashion retailer. We find that sunshine, temperature, and rain have a significant impact on daily sales, particularly in the summer, on weekends, and on days with extreme weather. Using weather forecasts, we have significantly improved sales forecast accuracy. We find that including weather data in the sales forecast model can lead to fewer sales forecast errors, reducing them by, on average, 8.6% to 12.2% and up to 50.6% on summer weekends. In turn, the improvement in sales forecast accuracy has a measurable impact on logistics and warehousing operations. We quantify the value of incorporating weather forecasts in the planning process for the order fulfillment center workforce and show how their incorporation can be leveraged to reduce costs and increase performance. With a perfect information planning scenario, excess costs can be reduced by 11.6% compared with the cost reduction attainable with a baseline model that ignores weather information in workforce planning.     

A Comparison of Unconstraining Methods to Improve Revenue Management Systems

Asuccessful revenue management system requires accurate demand forecasts for each customer segment. The forecasts are used to set booking limits for lower value customers to ensure an adequate supply for higher value customers. The very use of booking limits, however, constrains the historical demand data needed for an accurate forecast. Ignoring this interaction leads to substantial penalties in a firm's potential revenues. We review existing unconstraining methods and propose a new method that includes some attractive properties not found in the existing methods. We evaluate several of the common unconstraining methods against our proposed method by testing them on intentionally constrained simulated data. Results indicate our proposed method outperforms other methods in two of three data sets. We also test the revenue impact of our proposed method, expectation maximization (EM), and “no unconstraining” on actual booking data from a hotel/casino. We show that performance varies with the initial starting protection limits and a lack of unconstraining leads to significant revenue losses.     

Improving Risk Management: From Lame Excuses to Principled Practice

The three classic pillars of risk analysis are risk assessment (how big is the risk and how sure can we be?), risk management (what shall we do about it?), and risk communication (what shall we say about it, to whom, when, and how?). We propose two complements as important parts of these three bases: risk attribution (who or what addressable conditions actually caused an accident or loss?) and learning from experience about risk reduction (what works, and how well?). Failures in complex systems usually evoke blame, often with insufficient attention to root causes of failure, including some aspects of the situation, design decisions, or social norms and culture. Focusing on blame, however, can inhibit effective learning, instead eliciting excuses to deflect attention and perceived culpability. Productive understanding of what went wrong, and how to do better, thus requires moving past recrimination and excuses. This article identifies common blame‐shifting “lame excuses” for poor risk management. These generally contribute little to effective improvements and may leave real risks and preventable causes unaddressed. We propose principles from risk and decision sciences and organizational design to improve results. These start with organizational leadership. More specifically, they include: deliberate testing and learning—especially from near‐misses and accident precursors; careful causal analysis of accidents; risk quantification; candid expression of uncertainties about costs and benefits of risk‐reduction options; optimization of tradeoffs between gathering additional information and immediate action; promotion of safety culture; and mindful allocation of people, responsibilities, and resources to reduce risks. We propose that these principles provide sound foundations for improving successful risk management.     

Combining Judgmental and Statistical Forecasts: An Application to Earnings Forecasts

This study investigated the accuracy of combinations of statistical and judgmental forecasts of annual accounting earnings. Combined forecasts were generated as equally weighted (i.e., simple averages) and unequally weighted combinations of individual forecasts from time-series models of quarterly and annual earnings (statistical forecasts) and security analysts' forecasts of quarterly and annual earnings (judgmental forecasts). The effect of the number of individual forecasts combined on the accuracy of the combined forecasts was also examined. The empirical results indicated that, on the average, combined forecasts were more accurate than individual forecasts. The results also indicated that although analysts' forecasts are based on a wider information set, the accuracy of their forecasts could be improved by combining them with forecasts generated from statistical models. Even if the best individual forecast could be identified in advance, gains in accuracy could be achieved by using combinations of two other forecasting methods. Several of the combined forecasts were superior to the most accurate individual forecast. Forecasts combined by using unequal weights derived from a regression model proved more accurate than equally weighted combinations. Forecasting accuracy improved and the variability of accuracy across different combinations decreased as the number of forecasts in the combination increased.     

Improving Revenue Management Decision Making for Airlines by Evaluating Analyst‐Adjusted Passenger Demand Forecasts*

To maximize revenue, airline revenue management analysts (RMAs) attempt to protect the right number of seats for late‐booking, high‐revenue‐generating passengers from low‐valued leisure passengers. Simulation results in the past showed that a major airline can generate approximately $500 million per year through efficient RM operations. Accurate passenger demand forecasts are required, because reduced forecast error significantly improves revenue. RMAs often adjust the system forecasts to improve revenue opportunity. Analysis of system forecast performance and analyst adjustment is complex, because one must account for all unseen demands throughout the life of a flight. This article proposes a method to account for unseen demand and evaluate forecast performance (adjusted or unadjusted) through a forecast monitoring system. Initial results from one major airline's origin‐destination market data justify the value of RMA forecasting adjustments.     

Priority Rules for Multi‐Task Due‐Date Scheduling under Varying Processing Costs

We study the scheduling of multiple tasks under varying processing costs and derive a priority rule for optimal scheduling policies. Each task has a due date, and a non‐completion penalty cost is incurred if the task is not completely processed before its due date. We assume that the task arrival process is stochastic and the processing rate is capacitated. Our work is motivated by both traditional and emerging application domains, such as construction industry and freelance consulting industry. We establish the optimality of Shorter Slack time and Longer remaining Processing time (SSLP) principle that determines the priority among active tasks. Based on the derived structural properties, we also propose an effective cost‐balancing heuristic policy and demonstrate the efficacy of the proposed policy through extensive numerical experiments. We believe our results provide operators/managers valuable insights on how to devise effective service scheduling policies under varying costs.     

Distribution Planning to Optimize Profits in the Motion Picture Industry

We consider the distribution planning problem in the motion picture industry. This problem involves forecasting theater‐level box office revenues for a given movie and using these forecasts to choose the best locations to screen a movie. We first develop a method that predicts theater‐level box office revenues over time for a given movie as a function of movie attributes and theater characteristics. These estimates are then used by the distributor to choose where to screen the movie. The distributor's location selection problem is modeled as an integer programming‐based optimization model that chooses the location of theaters in order to optimize profits. We tested our methods on realistic box office data and show that it has the potential to significantly improve the distributor's profits. We also develop some insights into why our methods outperform existing practice, which are crucial to their successful practical implementation.     

Forecast Updating and Supplier Coordination for Complementary Component Purchases

We study a supply chain where an original equipment manufacturer (OEM) buys subassemblies, comprised of two complementary sets of components, from a contract manufacturer (CM). The OEM provides a demand forecast at the time when the CM must order the long lead‐time set of components, but must decide whether or not to provide updated forecasts as a matter of practice. Forecast updates affect the CM's short lead‐time purchase decision, and the anticipation of updates may also affect the long lead‐time purchase decision. While the OEM and CM both incur lost sales costs, the OEM can decide whether or not to share the overage costs otherwise fully borne by the CM. We investigate when the OEM is better served by committing to provide updated forecasts and/or committing to share overage costs. For a distribution‐free, two‐stage forecast‐update model, we show that (1) the practice of providing forecast updates may be harmful to the OEM and (2) at the OEM's optimal levels of overage risk sharing, the CM undersupplies relative to the supply chain optimal quantity. For a specific forecast‐update model, we computationally investigate conditions under which forecast updating and risk sharing are in the best interest of the OEM.     

Dynamic Supply Risk Management with Signal‐Based Forecast,Multi‐Sourcing,and Discretionary Selling

We examine the critical role of advance supply signals—such as suppliers’ financial health and production viability—in dynamic supply risk management. The firm operates an inventory system with multiple demand classes and multiple suppliers. The sales are discretionary and the suppliers are susceptible to both systematic and operational risks. We develop a hierarchical Markov model that captures the essential features of advance supply signals, and integrate it with procurement and selling decisions. We characterize the optimal procurement and selling policy, and the strategic relationship between signal‐based forecast, multi‐sourcing, and discretionary selling. We show that higher demand heterogeneity may reduce the value of discretionary selling, and that the mean value‐based forecast may outperform the stationary distribution‐based forecast. This work advances our understanding on when and how to use advance supply signals in dynamic risk management. Future supply risk erodes profitability but enhances the marginal value of current inventory. A signal of future supply shortage raises both base stock and demand rationing levels, thereby boosting the current production and tightening the current sales. Signal‐based dynamic forecast effectively guides the firm's procurement and selling decisions. Its value critically depends on supply volatility and scarcity. Ignoring advance supply signals can result in misleading recommendations and severe losses. Signal‐based dynamic supply forecast should be used when: (a) supply uncertainty is substantial, (b) supply‐demand ratio is moderate, (c) forecast precision is high, and (d) supplier heterogeneity is high.     

Impact of Bias Magnification on Supply Chain Costs: The Mitigating Role of Forecast Sharing

The impact of forecast error magnification on supply chain cost has been well documented. Unlike past studies that measure forecast error in terms of forecast standard deviation, our study extends research to consider the impact of forecast bias, and the complex interaction between these variables. Simulating a two‐stage supply chain using realistic cost data we test the impact of bias magnification comparing two scenarios: one with forecast sharing between retailer and supplier, and one without. We then corroborate findings via survey data. Results show magnification of forecast bias to have a considerably greater impact on supply chain cost than magnification of forecast standard deviation. Particularly damaging is high bias in the presence of high forecast standard deviation. Forecast sharing is found to mitigate the impact of forecast error, however, primarily at higher levels of forecast standard deviation. At low levels of forecast standard deviation the benefits are not significant suggesting that engaging in such mitigation strategies may be less effective when there is little opportunity for improvement in accuracy. Furthermore, forecast sharing is found to be much less effective against high levels of bias. This is an important finding as managers often deliberately bias their forecasts and underscores the importance of exercising caution even with forecast sharing, particularly for forecasts that have inherently large errors. The findings provide a deeper understanding of the impact of forecast errors, suggest limitations of forecast sharing, and offer implications for research and practice alike.     

Managing Functional Biases in Organizational Forecasts: A Case Study of Consensus Forecasting in Supply Chain Planning

To date, little research has been done on managing the organizational and political dimensions of generating and improving forecasts in corporate settings. We examine the implementation of a supply chain planning process at a consumer electronics company, concentrating on the forecasting approach around which the process revolves. Our analysis focuses on the forecasting process and how it mediates and accommodates the functional biases that can impair the forecast accuracy. We categorize the sources of functional bias into intentional, driven by misalignment of incentives and the disposition of power within the organization, and unintentional, resulting from informational and procedural blind spots. We show that the forecasting process, together with the supporting mechanisms of information exchange and elicitation of assumptions, is capable of managing the potential political conflict and the informational and procedural shortcomings. We also show that the creation of an independent group responsible for managing the forecasting process, an approach that we distinguish from generating forecasts directly, can stabilize the political dimension sufficiently to enable process improvement to be steered. Finally, we find that while a coordination system—the relevant processes, roles and responsibilities, and structure—can be designed to address existing individual and functional biases in the organization, the new coordination system will in turn generate new individual and functional biases. The introduced framework of functional biases (whether those biases are intentional or not), the analysis of the political dimension of the forecasting process, and the idea of a coordination system are new constructs to better understand the interface between operations management and other functions.     

ON THE IMPLICATIONS OF SPECIFICATION UNCERTAINTY IN FORECASTING*

Forecasters typically select a statistical forecasting model from among a set of alternative models. Subsequently, forecasts are generated with the chosen model and reported to management (forecast consumers) as if specification uncertainty did not exist (i.e., as if the chosen model were the “true” model of the forecast variable). In this note, a well-known Bayesian model-comparison procedure is used to illustrate some of the ambiguities and distortions of forecasts that do not reflect specification uncertainty. It is shown that a single selected forecasting model (however chosen) will generally misstate measures of forecast risk and lead to point and interval forecasts that are misplaced from a decision-theoretic point of view.     

Using Expert Judgments to Explore Robust Alternatives for Forest Management under Climate Change

We develop and apply a judgment‐based approach to selecting robust alternatives, which are defined here as reasonably likely to achieve objectives, over a range of uncertainties. The intent is to develop an approach that is more practical in terms of data and analysis requirements than current approaches, informed by the literature and experience with probability elicitation and judgmental forecasting. The context involves decisions about managing forest lands that have been severely affected by mountain pine beetles in British Columbia, a pest infestation that is climate‐exacerbated. A forest management decision was developed as the basis for the context, objectives, and alternatives for land management actions, to frame and condition the judgments. A wide range of climate forecasts, taken to represent the 10–90% levels on cumulative distributions for future climate, were developed to condition judgments. An elicitation instrument was developed, tested, and revised to serve as the basis for eliciting probabilistic three‐point distributions regarding the performance of selected alternatives, over a set of relevant objectives, in the short and long term. The elicitations were conducted in a workshop comprising 14 regional forest management specialists. We employed the concept of stochastic dominance to help identify robust alternatives. We used extensive sensitivity analysis to explore the patterns in the judgments, and also considered the preferred alternatives for each individual expert. The results show that two alternatives that are more flexible than the current policies are judged more likely to perform better than the current alternatives on average in terms of stochastic dominance. The results suggest judgmental approaches to robust decision making deserve greater attention and testing.     

The futility of forecasting

Long-range forecasting is an integral part of planning, but relying on its accuracy may be a mistake. The landscape is strewn with often wildly inaccurate forecasts. This article studies performances of some forecasts, analyses factors contributing to forecast error, and suggests ways in which management may deal with the uncertainty resulting from faulty forecasting performances.