Acquisition Management Under Fluctuating Raw Material Prices

We study a continuous‐review acquisition problem, in which the raw material price follows a discrete‐state Markov process and demand is compound Poisson. We show that one optimal policy is of the order‐up‐to type. Under our mean reversion and time continuity conditions, we further show that the order‐up‐to levels are decreasing at the current price level. At the same time, our computational study verifies that both conditions are indispensable for the monotonicity result. The study also hints at the connection between discrete‐ and continuous‐state price processes.     

Equilibrium in Continuous‐Time Financial Markets: Endogenously Dynamically Complete Markets

We prove existence of equilibrium in a continuous‐time securities market in which the securities are potentially dynamically complete: the number of securities is at least one more than the number of independent sources of uncertainty. We prove that dynamic completeness of the candidate equilibrium price process follows from mild exogenous assumptions on the economic primitives of the model. Our result is universal, rather than generic: dynamic completeness of the candidate equilibrium price process and existence of equilibrium follow from the way information is revealed in a Brownian filtration, and from a mild exogenous nondegeneracy condition on the terminal security dividends. The nondegeneracy condition, which requires that finding one point at which a determinant of a Jacobian matrix of dividends is nonzero, is very easy to check. We find that the equilibrium prices, consumptions, and trading strategies are well‐behaved functions of the stochastic process describing the evolution of information. We prove that equilibria of discrete approximations converge to equilibria of the continuous‐time economy.     

Bootstrap Methods for Markov Processes

The block bootstrap is the best known bootstrap method for time‐series data when the analyst does not have a parametric model that reduces the data generation process to simple random sampling. However, the errors made by the block bootstrap converge to zero only slightly faster than those made by first‐order asymptotic approximations. This paper describes a bootstrap procedure for data that are generated by a Markov process or a process that can be approximated by a Markov process with sufficient accuracy. The procedure is based on estimating the Markov transition density nonparametrically. Bootstrap samples are obtained by sampling the process implied by the estimated transition density. Conditions are given under which the errors made by the Markov bootstrap converge to zero more rapidly than those made by the block bootstrap.     

Optimal Sourcing and Lead‐Time Reduction under Evolutionary Demand Risk

We develop a real‐options model for optimizing production and sourcing choices under evolutionary supply‐chain risk. We model lead time as an endogenous decision and calculate the cost differential required to compensate for the risk exposure coming from lead time. The shape of the resulting cost‐differential frontier reveals the term structure of supply‐chain risk premiums and provides guidance as to the potential value of lead‐time reduction. Under constant demand volatility, the break‐even cost differential increases in volatility and lead time at a decreasing rate, making incremental lead‐time reduction less valuable than full lead‐time reduction. Stochastic demand volatility increases the relative value of incremental lead‐time reduction. When demand has a heavy right tail, the value of lead‐time reduction depends on how extreme values of demand are incorporated into the forecasting process. The cost‐differential frontier is invariant to discount rates, making the cost of capital irrelevant for choosing between lead times. We demonstrate the managerial implications of the model by applying it first to the classic Sport‐Obermeyer case and then to a supplier‐selection problem faced by a global manufacturer.     

Long‐Term Risk: An Operator Approach

We create an analytical structure that reveals the long‐run risk‐return relationship for nonlinear continuous‐time Markov environments. We do so by studying an eigenvalue problem associated with a positive eigenfunction for a conveniently chosen family of valuation operators. The members of this family are indexed by the elapsed time between payoff and valuation dates, and they are necessarily related via a mathematical structure called a semigroup. We represent the semigroup using a positive process with three components: an exponential term constructed from the eigenvalue, a martingale, and a transient eigenfunction term. The eigenvalue encodes the risk adjustment, the martingale alters the probability measure to capture long‐run approximation, and the eigenfunction gives the long‐run dependence on the Markov state. We discuss sufficient conditions for the existence and uniqueness of the relevant eigenvalue and eigenfunction. By showing how changes in the stochastic growth components of cash flows induce changes in the corresponding eigenvalues and eigenfunctions, we reveal a long‐run risk‐return trade‐off.     

Mixed Hitting‐Time Models

We study mixed hitting‐time models that specify durations as the first time a Lévy process—a continuous‐time process with stationary and independent increments—crosses a heterogeneous threshold. Such models are of substantial interest because they can be deduced from optimal‐stopping models with heterogeneous agents that do not naturally produce a mixed proportional hazards structure. We show how strategies for analyzing the identifiability of the mixed proportional hazards model can be adapted to prove identifiability of a hitting‐time model with observed covariates and unobserved heterogeneity. We discuss inference from censored data and give examples of structural applications. We conclude by discussing the relative merits of both models as complementary frameworks for econometric duration analysis.     

The Effect of Distribution Processes on Replenishment Lead Time and Inventory

We investigate the interrelationship of distribution center picking policies and supply chain inventory performance. In particular, we show how the picking sequence in the upstream supply chain link affects the inventory levels of items being replenished to a downstream link for a common “ship‐when‐full” trailer dispatching policy. Perturbing the picking sequence affects items’ inventory levels asymmetrically which causes the aggregate system inventory to vary. We separate the items in replenishment into those units in transit and those awaiting shipment from the upstream distribution step: we call the latter the residual replenishment. We show that the process governing aggregate residual replenishment is Markov and has a stationary distribution that is discrete uniform. Computing the item‐level residual distribution is intractable and so we develop analytical models from which we derive hypotheses for the effectiveness of stable vs. random picking sequences, how item residual replenishment varies with stable picking sequences, and how the aggregate inventory level changes with picking sequence. These suggest a heuristic sequencing algorithm for minimizing inventory, which performs well in simulation tests over a large testbed of parameter sets.     

A Systematic Approach to Determining the Identifiability of Multistage Carcinogenesis Models

Multistage clonal expansion (MSCE) models of carcinogenesis are continuous‐time Markov process models often used to relate cancer incidence to biological mechanism. Identifiability analysis determines what model parameter combinations can, theoretically, be estimated from given data. We use a systematic approach, based on differential algebra methods traditionally used for deterministic ordinary differential equation (ODE) models, to determine identifiable combinations for a generalized subclass of MSCE models with any number of preinitation stages and one clonal expansion. Additionally, we determine the identifiable combinations of the generalized MSCE model with up to four clonal expansion stages, and conjecture the results for any number of clonal expansion stages. The results improve upon previous work in a number of ways and provide a framework to find the identifiable combinations for further variations on the MSCE models. Finally, our approach, which takes advantage of the Kolmogorov backward equations for the probability generating functions of the Markov process, demonstrates that identifiability methods used in engineering and mathematics for systems of ODEs can be applied to continuous‐time Markov processes.     

Managing Perishables with Time and Temperature History

We address the use and value of time and temperature information to manage perishables in the context of a retailer that sells a random lifetime product subject to stochastic demand and lost sales. The product's lifetime is largely determined by the temperature history and the flow time through the supply chain. We compare the case in which information on flow time and temperature history is available and used for inventory management to a base case in which such information is not available. We formulate the two cases as Markov Decision Processes and evaluate the value of information through an extensive simulation using representative, real world supply chain parameters.     

Competing on Time: An Integrated Framework to Optimize Dynamic Time‐to‐Market and Production Decisions

This study develops a comprehensive framework to optimize new product introduction timing and subsequent production decisions faced by a component supplier. Prior to market entry, the supplier performs process design activities, which improve manufacturing yield and the chances of getting qualified for the customer's product. However, a long delay in market entry allows competitors to enter the market and pass the customer's qualification process before the supplier, reducing the supplier's share of the customer's business. After entering the market and if qualified, the supplier also needs to decide how much to produce for a finite planning horizon by considering several factors such as manufacturing yield and stochastic demand, both of which depend on the earlier time‐to‐market decision. To capture this dependency, we develop a sequential, nested, two‐stage decision framework to optimize the time‐to‐market and production decisions in relation to each other. We show that the supplier's optimal market entry and qualification timing decision need to be revised in real time based on the number of qualified competitors at the time of market‐entry decision. We establish the optimality of a threshold policy. Following this policy, at the beginning of each decision epoch, the supplier should optimally stop preparing for qualification and decide whether to enter the market if her order among qualified competitors exceeds a predetermined threshold. We also prove that the supplier's optimal production policy is a state‐dependent, base‐stock policy, which depends on the time‐to‐market and qualification decisions. The proposed framework also enables a firm to quantify how market conditions (such as price and competitor entry behavior) and operating conditions (such as the rate of learning and inventory/production‐related costs) affect time‐to‐market strategy and post‐entry production decisions.     

Hierarchical Multi‐skill Resource Assignment in the Telecommunications Industry

We formulate a discrete time Markov decision process for a resource assignment problem for multi‐skilled resources with a hierarchical skill structure to minimize the average penalty and waiting costs for jobs with different waiting costs and uncertain service times. In contrast to most queueing models, our application leads to service times that are known before the job is actually served but only after it is accepted and assigned to a server. We formulate the corresponding Markov decision process, which is intractable for problems of realistic size due to the curse of dimensionality. Using an affine approximation of the bias function, we develop a simple linear program that yields a lower bound for the minimum average costs. We suggest how the solution of the linear program can be used in a simple heuristic and illustrate its performance in numerical examples and a case study.     

Time Horizon and Cooperation in Continuous Time

We study social dilemmas in (quasi‐) continuous‐time experiments, comparing games with different durations and termination rules. We discover a stark qualitative contrast in behavior in continuous time as compared to previously studied behavior in discrete‐time games: cooperation is easier to achieve and sustain with deterministic horizons than with stochastic ones, and end‐game effects emerge, but subjects postpone them with experience. Analysis of individual strategies provides a basis for a simple reinforcement learning model that proves to be consistent with this evidence. An additional treatment lends further support to this explanation.     

Dynamic Preference for Flexibility

We consider a decision maker who faces dynamic decision situations that involve intertemporal trade‐offs, as in consumption–savings problems, and who experiences taste shocks that are transient contingent on the state of the world. We axiomatize a recursive representation of choice over state contingent infinite horizon consumption problems, where uncertainty about consumption utilities depends on the observable state and the state follows a subjective Markov process. The parameters of the representation are the subjective process that governs the evolution of beliefs over consumption utilities and the discount factor; they are uniquely identified from behavior. We characterize a natural notion of greater preference for flexibility in terms of a dilation of beliefs. An important special case of our representation is a recursive version of the Anscombe–Aumann model with parameters that include a subjective Markov process over states and state‐dependent utilities, all of which are uniquely identified.     

How Excessive Stage Time Reduction in NPD Negatively Impacts Market Value

We empirically investigate how time reductions in particular product development stages impact market value. Using longitudinal project data from 107 firms, we compare stage times prior to and following investments in new product development process changes. Our analysis reveals a predominance of focus on time reduction in the late stages of product development. We also find support for the existence of an inverted‐U relationship between market performance and time reductions for some of these stages: beta testing and technical implementation. Therefore, while time reductions can improve time to market, we observe a clear limit to the benefits associated with stage time reductions at particular stages. We also investigate the role of strategic contextual factors such as the extent to which a firm's patented innovations rely upon a variety, as opposed to a limited range, of diverse technology classes. The extent of this technology‐span impacts optimal stage time reductions. We perform an in‐depth post hoc analysis with a small set of firms to uncover how they should invest in stage time reduction given our empirical results. The post hoc analysis highlights that some firms are likely overinvesting in stage time reductions and destroying market value.     

Collusion With Persistent Cost Shocks

We consider a dynamic Bertrand game in which prices are publicly observed and each firm receives a privately observed cost shock in each period. Although cost shocks are independent across firms, within a firm costs follow a first‐order Markov process. We analyze the set of collusive equilibria available to firms, emphasizing the best collusive scheme for the firms at the start of the game. In general, there is a trade‐off between productive efficiency, whereby the low‐cost firm serves the market in a given period, and high prices. We show that when costs are perfectly correlated over time within a firm, if the distribution of costs is log‐concave and firms are sufficiently patient, then the optimal collusive scheme entails price rigidity: firms set the same price and share the market equally, regardless of their respective costs. When serial correlation of costs is imperfect, partial productive efficiency is optimal. For the case of two cost types, first‐best collusion is possible if the firms are patient relative to the persistence of cost shocks, but not otherwise. We present numerical examples of first‐best collusive schemes.     

Flexible Backup Supply and the Management of Lead‐Time Uncertainty

We consider replenishment decisions for a constant rate demand environment from a supplier with uncertain lead times. We study the potential use of a flexible backup supplier as an emergency response to accurate lead‐time information arriving at (or close after) the beginning of the demand interval and well after an original order with the stochastic lead‐time supplier has been placed. The emergency response decisions involve whether to order and how much from the flexible backup supplier, with the objective of minimizing the cost of meeting demand. We derive the optimal emergency‐response policy and clearly outline its implications on the optimized safety lead time of the original order placement and on the cost of meeting demand. We examine the impact on the use of the flexible backup supplier of factors like the arrival time of accurate lead‐time information and the response lead time of the backup supplier. We further study the potential benefits of the use of the flexible backup supplier in a dual role: as one of the two suppliers in a redundant supply system assigned to originally meet the demand and as an emergency response to later‐arriving lead‐time information. Our numerical studies illustrate the benefits from the use of the flexible backup supplier as an emergency response, but for reasonable purchase premiums and short lead times of flexible backup supply options, their use in a dual (regular and emergency response) role often leads to improved performance over safety lead‐time single and uncertain lead‐time supplier‐replenishment strategies. The benefits of the backup supply options are accentuated the higher the lead‐time uncertainty of the stochastic lead‐time supplier is.     

Designing Personalized Treatment: An Application to Anticoagulation Therapy

In this study, we develop an analytical framework for personalizing the anticoagulation therapy of patients who are taking warfarin. Consistent with medical practice, our treatment design consists of two stages: (i) the initiation stage, modeled using a partially‐observable Markov decision process, during which the physician learns through systematic belief updates about the unobservable patient sensitivity to warfarin, and (ii) the maintenance stage, modeled using a Markov decision process, during which the physician relies on his formed belief about patient sensitivity to determine the stable, patient‐specific, warfarin dose to prescribe. We develop an expression for belief updates in the POMDP, establish the optimality of the myopic policy for the MDP, and derive conditions for the existence and uniqueness of a myopically optimal dose. We validate our models using a real‐life patient data set gathered at the Hematology Clinic of the Jewish General Hospital in Montreal. The proposed analytical framework and case study enable us to develop useful clinical insights, for example, concerning the length of the initiation period and the importance of correctly assessing patient sensitivity.     

Deterministic Approximation of Stochastic Evolution in Games

This paper provides deterministic approximation results for stochastic processes that arise when finite populations recurrently play finite games. The processes are Markov chains, and the approximation is defined in continuous time as a system of ordinary differential equations of the type studied in evolutionary game theory. We establish precise connections between the long‐run behavior of the discrete stochastic process, for large populations, and its deterministic flow approximation. In particular, we provide probabilistic bounds on exit times from and visitation rates to neighborhoods of attractors to the deterministic flow. We sharpen these results in the special case of ergodic processes.     

Route vs. Segment: An Experiment on Real‐Time Travel Information in Congestible Networks

We report the results of an experimental study of route choice in congestible networks with a common origin and common destination. In one condition, in each round of play network users independently committed themselves at the origin to a three‐segment route; in the other condition, they chose route segments sequentially at each network junction upon receiving en route information. At the end of each round, players received ex‐post complete information about the distribution of the route choices. Although the complexity of the network defies analysis by common users, traffic patterns in both conditions converged rapidly to the equilibrium solution. We account for the observed results by a Markov adaptive learning model postulating regret minimization and inertia. We find that subjects' learning behavior was similar across conditions, except that they exhibited more inertia in the condition with en route information.     

Modeling Finite‐Time Failure Probabilities in Risk Analysis Applications

In this article, we introduce a framework for analyzing the risk of systems failure based on estimating the failure probability. The latter is defined as the probability that a certain risk process, characterizing the operations of a system, reaches a possibly time‐dependent critical risk level within a finite‐time interval. Under general assumptions, we define two dually connected models for the risk process and derive explicit expressions for the failure probability and also the joint probability of the time of the occurrence of failure and the excess of the risk process over the risk level. We illustrate how these probabilistic models and results can be successfully applied in several important areas of risk analysis, among which are systems reliability, inventory management, flood control via dam management, infectious disease spread, and financial insolvency. Numerical illustrations are also presented.