Judgmental Adjustments of Previously Adjusted Forecasts*

Forecasts are important components of information systems. They provide a means for knowledge sharing and thus have significant decision‐making impact. In many organizations, it is quite common for forecast users to receive predictions that have previously been adjusted by providers or other users of forecasts. Current work investigates some of the factors that may influence the size and propensity of further adjustments on already‐adjusted forecasts. Two studies are reported that focus on the potential effects of adjustment framing (Study 1) and the availability of explanations and/or original forecasts alongside the adjusted forecasts (Study 2). Study 1 provides evidence that the interval forecasts that are labeled as “adjusted” are modified less than the so‐called “original/unadjusted” predictions. Study 2 suggests that the provision of original forecasts and the presence of explanations accompanying the adjusted forecasts serve as significant factors shaping the size and propensity of further modifications. Findings of both studies highlight the importance of forecasting format and user perceptions with critical organizational repercussions.     

ON THE IMPLICATIONS OF SPECIFICATION UNCERTAINTY IN FORECASTING*

Forecasters typically select a statistical forecasting model from among a set of alternative models. Subsequently, forecasts are generated with the chosen model and reported to management (forecast consumers) as if specification uncertainty did not exist (i.e., as if the chosen model were the “true” model of the forecast variable). In this note, a well-known Bayesian model-comparison procedure is used to illustrate some of the ambiguities and distortions of forecasts that do not reflect specification uncertainty. It is shown that a single selected forecasting model (however chosen) will generally misstate measures of forecast risk and lead to point and interval forecasts that are misplaced from a decision-theoretic point of view.     

Improving the accuracy and operational predictability of project cost forecasts: an adaptive combination approach

Conducting an early warning forecast to detect potential cost overrun is essential for timely and effective decision-making in project control. This paper presents a forecast combination model that adaptively identifies the best forecast and optimises various combinations of commonly used project cost forecasting models. To do so, a forecast error simulator is formulated to visualise and quantify likely error profiles of forecast models and their combinations. The adaptive cost combination (ACC) model was applied to a pilot project for numerical illustration as well as to real world projects for practical implementation. The results provide three valuable insights into more effective project control and forecasting. First, the best forecasting model may change in individual projects according to the project progress and the management priority (i.e. accuracy, outperformance or large errors). Second, adaptive combination of simple, index-based forecasts tends to improve forecast accuracy, while mitigating the risk of large errors. Third, a post-mortem analysis of seven real projects indicated that the simple average of two most commonly used cost forecasts can be 31.2% more accurate, on average, than the most accurate alternative forecasts.     

The Value of Weather Information for E‐Commerce Operations

To be efficient, logistics operations in e‐commerce require warehousing and transportation resources to be aligned with sales. Customer orders must be fulfilled with short lead times to ensure high customer satisfaction, and the costly under‐utilization of workers must be avoided. To approach this ideal, forecasting order quantities with high accuracy is essential. Many drivers of online sales, including seasonality, special promotions and public holidays, are well known, and they have been frequently incorporated into forecasting approaches. However, the impact of weather on e‐commerce operations has not been rigorously analyzed. In this study, we integrate weather data into the sales forecasting of the largest European online fashion retailer. We find that sunshine, temperature, and rain have a significant impact on daily sales, particularly in the summer, on weekends, and on days with extreme weather. Using weather forecasts, we have significantly improved sales forecast accuracy. We find that including weather data in the sales forecast model can lead to fewer sales forecast errors, reducing them by, on average, 8.6% to 12.2% and up to 50.6% on summer weekends. In turn, the improvement in sales forecast accuracy has a measurable impact on logistics and warehousing operations. We quantify the value of incorporating weather forecasts in the planning process for the order fulfillment center workforce and show how their incorporation can be leveraged to reduce costs and increase performance. With a perfect information planning scenario, excess costs can be reduced by 11.6% compared with the cost reduction attainable with a baseline model that ignores weather information in workforce planning.     

Provider–user differences in perceived usefulness of forecasting formats

This paper aims to examine potential differences in perceived usefulness of various forecasting formats from the perspectives of providers and users of predictions. Experimental procedure consists of asking participants to assume the role of forecast providers and to construct forecasts using different formats, followed by requesting usefulness ratings for these formats (Phase 1). Usefulness of the formats are rated again in hindsight after receiving individualized performance feedback (Phase 2). In the ensuing role switch exercise, given new series and external predictions, participants are required to assign usefulness ratings as forecast users (Phase 3). In the last phase, participants are given performance feedback and asked to rate the usefulness in hindsight as users of predictions (Phase 4). Results reveal that regardless of the forecasting role, 95% prediction intervals are considered to be the most useful format, followed by directional predictions, 50% interval forecasts, and lastly, point forecasts. Finally, for all formats and for both roles, usefulness in hindsight is found to be lower than usefulness prior to performance feedback presentation.     

Combining Judgmental and Statistical Forecasts: An Application to Earnings Forecasts

This study investigated the accuracy of combinations of statistical and judgmental forecasts of annual accounting earnings. Combined forecasts were generated as equally weighted (i.e., simple averages) and unequally weighted combinations of individual forecasts from time-series models of quarterly and annual earnings (statistical forecasts) and security analysts' forecasts of quarterly and annual earnings (judgmental forecasts). The effect of the number of individual forecasts combined on the accuracy of the combined forecasts was also examined. The empirical results indicated that, on the average, combined forecasts were more accurate than individual forecasts. The results also indicated that although analysts' forecasts are based on a wider information set, the accuracy of their forecasts could be improved by combining them with forecasts generated from statistical models. Even if the best individual forecast could be identified in advance, gains in accuracy could be achieved by using combinations of two other forecasting methods. Several of the combined forecasts were superior to the most accurate individual forecast. Forecasts combined by using unequal weights derived from a regression model proved more accurate than equally weighted combinations. Forecasting accuracy improved and the variability of accuracy across different combinations decreased as the number of forecasts in the combination increased.     

A Congruence Construct of User Involvement

The MIS literature has devoted considerable attention to the relationship between user involvement and MIS success; unfortunately, this research has produced conflicting results. Recent research on a discrepancy model of user involvement has provided a framework for reconciling studies showing positive, negative, or no impact of involvement on user satisfaction. In this discrepancy model, individual differences between perceived and desired levels of involvement define three conditional states or frames of reference that govern the relationship between involvement and end-user satisfaction. The discrepancy model indicates that studies of user involvement need to control for individual differences by dividing respondents into three groups corresponding to these conditional states. Building upon the discrepancy concept, this paper presents a congruence construct of user involvement (i.e., a measure of involvement “relative” to an individual's desire to get involved) as an alternative way of modeling this contingency relationship. The reliability and validity of the involvement congruence construct are assessed. Perceived and congruence constructs of involvement are compared as predictors of end-user computing satisfaction. The results suggest that involvement congruence is a better predictor than perceived involvement and may offer theoretical as well as empirical advantages over the use of component measures. Furthermore, the congruence method of modeling the discrepancy effect has research design advantages (i.e., it does not require dividing respondents by frame of reference).     

Forecast horizon aggregation in integer autoregressive moving average (INARMA) models

This paper addresses aggregation in integer autoregressive moving average (INARMA) models. Although aggregation in continuous-valued time series has been widely discussed, the same is not true for integer-valued time series. Forecast horizon aggregation is addressed in this paper. It is shown that the overlapping forecast horizon aggregation of an INARMA process results in an INARMA process. The conditional expected value of the aggregated process is also derived for use in forecasting. A simulation experiment is conducted to assess the accuracy of the forecasts produced by the aggregation method and to compare it to the accuracy of cumulative h-step ahead forecasts over the forecasting horizon. The results of an empirical analysis are also provided.     

The futility of forecasting

Long-range forecasting is an integral part of planning, but relying on its accuracy may be a mistake. The landscape is strewn with often wildly inaccurate forecasts. This article studies performances of some forecasts, analyses factors contributing to forecast error, and suggests ways in which management may deal with the uncertainty resulting from faulty forecasting performances.     

Aligning supply chain collaboration using Analytic Hierarchy Process

The significance of collaboration among supply chain members has been sufficiently stressed in the recent literature as a powerful tool for increasing accuracy of demand forecasts and for consequent cost reductions. Since it has been recognized that naïve forecasting is no longer cost efficient, Supply Chain (SC) members have found it very important to exchange relevant information that will help improve accuracy of demand forecasting. This information differs widely in terms of their characteristics. For example, some information (e.g. historic sales data) that is cheap to exchange may not contribute to a great increase in forecast accuracy. Similarly, some information may not be very reliable (e.g. demand forecast by individual SC members). In general, there is a trade-off in the kind of information required and the kind of information exchanged. This study analyses these trade-offs using an Analytic Hierarchy Process (AHP) model. The model is then implemented based on case studies conducted in two manufacturing firms. The AHP model ranks available information in terms of their contributions to improve forecast accuracy, and can provide vital clues to SC partners for preparing exchangeable data. From the case studies using AHP model, it was proved that using the preferred SC data, the firms could enhance forecasts accuracy. This in turn can help the firms to make decisions on SC collaborative arrangements for information exchange.     

On Bayesian composite forecasting

Often decision makers have several forecasts of an uncertain and operationally relevant random variable. A rich literature now exists which argues that in this situation the decision maker should consider forming a forecast as a weighted average of each of the individual forecasts. In this paper, composite forecasting is discussed in a Bayesian context. The ability of the user to control the impact of the data on his composite weights is illustrated by an example using expert opinion forecasts of US hog prices.     

Composite Forecasting with Dirichlet Priors*

In this paper, composite forecasting is considered from a Bayesian perspective. A forecast user combines two or more forecasts of an operationally relevant random variable. We consider the case where outperformance is modeled as a realization from a multinomial process. The user has prior beliefs about the probability that a particular method outperforms all others, information which is summarized by the Dirichlet distribution. An empirical example with hog prices in the United States illustrates the method.     

Mechanisms to Induce Buyer Forecasting: Do Suppliers Always Benefit from Better Forecasting?

In this study, we consider a supplier's contract offerings to a buyer who may obtain improved forecasts for her demand over time. We investigate how the supplier can take advantage of the buyer's better forecasts and what kind of contracts he should offer to the buyer in order to maximize his profits. We model a natural forecast evolution where the buyer can obtain a more accurate forecast closer to the selling season. We assume there is information asymmetry between the buyer and the supplier at all times in that the buyer understands her demand better than the supplier. Three types of contracts that the supplier can offer are considered: (1) one where a contract is offered before the buyer has a chance to obtain improved forecasts, (2) one where a contract is offered after the buyer has obtained improved forecasts, and (3) a contingent (dynamic) contract which offers an initial contract to the buyer before she obtains improved forecasts, followed by a later contract (contingent on the initial contract) offered after improved forecasts have been obtained. We consider two scenarios: (1) where the supplier is certain that the buyer can obtain more accurate forecasts over time, and (2) where the supplier is uncertain about the buyer's forecasting capability (or forecasting cost). In the first scenario, we show that among the three types of contracts, the contingent contract is always the most profitable for the supplier. Furthermore, using the contingent contract, the supplier always benefits from higher accuracy of the buyer's demand forecasts. In the second scenario, we explicitly model the supplier's level of certainty about the buyer's capability of obtaining better forecasts, and explore how the supplier can design contracts to induce the buyer to obtain better forecasts when she is capable.     

The efficacy of using judgmental versus quantitative forecasting methods in practice

In an era where forecasts drive entire supply chains forecasting is seen as an increasingly critical organizational capability. However, business forecasting continues to rely on judgmental methods despite large advancements in information technology and quantitative method capability, prompting calls for research to help understand the reasons behind this practice. Our study is designed to contribute to this knowledge by profiling differences between firms identified as primary users of either judgmental or quantitative forecasting methods. Relying on survey data from 240 firms we statistically analyzed differences between these categories of users based on a range of organizational and forecasting issues. Our study finds large differences in forecast error rates between the two groups, with users of quantitative methods significantly outperforming users of judgmental methods. The former are found to be equally prevalent regardless of industry, firm size, and product positioning strategy, documenting the benefits of quantitative method use in a variety of settings. By contrast, the latter are found to have significantly lower access to quantifiable data and to use information and technology to a lesser degree.     

Measurement biases in user involvement research

Considerable research has been conducted to demonstrate user involvement's effect on information system success. User involvement and system success typically have been measured by asking users for their perceptions of these variables. This paper reports on a field study conducted to investigate the possibility that this approach to measuring study variables tends to overstate the benefits of user involvement. The link of user involvement to user satisfaction is found to be significantly weaker when user involvement is assessed by systems analysts than when it is self reported. Further, this difference is found to be greater for systems with few users than for systems with many users. The findings suggest that common method variance and self-serving bias may have overstated the apparent benefits of user involvement in past research on information systems. Suggestions for future research are presented.     

Some Empirical Findings on Short-Term Forecasting: Technique Complexity and Combinations

The purpose of this research is to determine if prior findings that favor simple forecasting techniques and technique combinations hold true in a short-term forecasting environment, where demand data can be quite volatile. Twenty-two time series of daily data from a real business setting are used to test one-period ahead forecasts, the epitome of short-term forecasting. The time series vary systematically as to data volatility and forecast difficulty. Forecast accuracy is measured in terms of both mean absolute percentage error (MAPE) and mean percentage error (MPE).     

Neural Network Earnings per Share Forecasting Models: A Comparative Analysis of Alternative Methods

In this paper, we present a comparative analysis of the forecasting accuracy of univariate and multivariate linear models that incorporate fundamental accounting variables (i.e., inventory, accounts receivable, and so on) with the forecast accuracy of neural network models. Unique to this study is the focus of our comparison on the multivariate models to examine whether the neural network models incorporating the fundamental accounting variables can generate more accurate forecasts of future earnings than the models assuming a linear combination of these same variables. We investigate four types of models: univariate‐linear, multivariate‐linear, univariate‐neural network, and multivariate‐neural network using a sample of 283 firms spanning 41 industries. This study shows that the application of the neural network approach incorporating fundamental accounting variables results in forecasts that are more accurate than linear forecasting models. The results also reveal limitations of the forecasting capacity of investors in the security market when compared to neural network models.     

Sales Forecasting with Financial Indicators and Experts' Input

We present a method for forecasting sales using financial market information and test this method on annual data for US public retailers. Our method is motivated by the permanent income hypothesis in economics, which states that the amount of consumer spending and the mix of spending between discretionary and necessity items depend on the returns achieved on equity portfolios held by consumers. Taking as input forecasts from other sources, such as equity analysts or time‐series models, we construct a market‐based forecast by augmenting the input forecast with one additional variable, lagged return on an aggregate financial market index. For this, we develop and estimate a martingale model of joint evolution of sales forecasts and the market index. We show that the market‐based forecast achieves an average 15% reduction in mean absolute percentage error compared with forecasts given by equity analysts at the same time instant on out‐of‐sample data. We extensively analyze the performance improvement using alternative model specifications and statistics. We also show that equity analysts do not incorporate lagged financial market returns in their forecasts. Our model yields correlation coefficients between retail sales and market returns for all firms in the data set. Besides forecasting, these results can be applied in risk management and hedging.     

Restrictiveness and guidance in support systems

Restrictiveness and guidance have been proposed as methods for improving the performance of users of support systems. In many companies computerized support systems are used in demand forecasting enabling interventions based on management judgment to be applied to statistical forecasts. However, the resulting forecasts are often ‘sub-optimal’ because many judgmental adjustments are made when they are not required. An experiment was used to investigate whether restrictiveness or guidance in a support system leads to more effective use of judgment. Users received statistical forecasts of the demand for products that were subject to promotions. In the restrictiveness mode small judgmental adjustments to these forecasts were prohibited (research indicates that these waste effort and may damage accuracy). In the guidance mode users were advised to make adjustments in promotion periods, but not to adjust in non-promotion periods. A control group of users were not subject to restrictions and received no guidance. The results showed that neither restrictiveness nor guidance led to improvements in accuracy. While restrictiveness reduced unnecessary adjustments, it deterred desirable adjustments and also encouraged over-large adjustments so that accuracy was damaged. Guidance encouraged more desirable system use, but was often ignored. Surprisingly, users indicated it was less acceptable than restrictiveness.     

Tests of Conditional Predictive Ability

We propose a framework for out‐of‐sample predictive ability testing and forecast selection designed for use in the realistic situation in which the forecasting model is possibly misspecified, due to unmodeled dynamics, unmodeled heterogeneity, incorrect functional form, or any combination of these. Relative to the existing literature (Diebold and Mariano (1995) and West (1996)), we introduce two main innovations: (i) We derive our tests in an environment where the finite sample properties of the estimators on which the forecasts may depend are preserved asymptotically. (ii) We accommodate conditional evaluation objectives (can we predict which forecast will be more accurate at a future date?), which nest unconditional objectives (which forecast was more accurate on average?), that have been the sole focus of previous literature. As a result of (i), our tests have several advantages: they capture the effect of estimation uncertainty on relative forecast performance, they can handle forecasts based on both nested and nonnested models, they allow the forecasts to be produced by general estimation methods, and they are easy to compute. Although both unconditional and conditional approaches are informative, conditioning can help fine‐tune the forecast selection to current economic conditions. To this end, we propose a two‐step decision rule that uses current information to select the best forecast for the future date of interest. We illustrate the usefulness of our approach by comparing forecasts from leading parameter‐reduction methods for macroeconomic forecasting using a large number of predictors.