A Markov Chain Model for an EMS System with Repositioning

We propose and analyze a two‐dimensional Markov chain model of an Emergency Medical Services system that repositions ambulances using a compliance table policy, which is commonly used in practice. The model is solved via a fixed‐point iteration. We validate the model against a detailed simulation model for several scenarios. We demonstrate that the model provides accurate approximations to various system performance measures, such as the response time distribution and the distribution of the number of busy ambulances, and that it can be used to identify near‐optimal compliance tables. Our numerical results show that performance depends strongly on the compliance table that is used, indicating the importance of choosing a well‐designed compliance table.     

A reinforcement-learning approach for admission control in distributed network service systems

In the distributed network service systems such as streaming-media systems and resource-sharing systems with multiple service nodes, admission control (AC) technology is an essential way to enhance performance. Model-based optimization approaches are good ways to be applied to analyze and solve the optimal AC policy. However, due to “the curse of dimensionality”, computing such policy for practical systems is a rather difficult task. In this paper, we consider a general model of the distributed network service systems, and address the problem of designing an optimal AC policy. An analytical model is presented for the system with fixed parameters based on semi-Markov decision process (SMDP). We design an event-driven AC policy, and the stationary randomized policy is taken as the policy structure. To solve the SMDP, both the state aggregation approach and the reinforcement-learning (RL) method with online policy optimization algorithm are applied. Then, we extend the problem by considering the system with time-varying parameters, where the arrival rates of requests at each service node may change over time. In view of this situation, an AC policy switching mechanism is presented. This mechanism allows the system to decide whether to adjust its AC policy according to the policy switching rule. And in order to maximize the gain of system, that is, to obtain the optimal AC policy switching rule, another RL-based algorithm is applied. To assess the effectiveness of SMDP-based AC policy and policy switching mechanism for the system, numerical experiments are presented. We compare the performance of optimal policies obtained by the solutions of proposed methods with other classical AC policies. The simulation results illustrate that higher performance and computational efficiency could be achieved by using the SMDP model and RL-based algorithms proposed in this paper.     

Inventory Rationing in a Make‐to‐Stock System with Batch Production and Lost Sales

We address an inventory rationing problem in a lost sales make‐to‐stock (MTS) production system with batch ordering and multiple demand classes. Each production order contains a single batch of a fixed lot size and the processing time of each batch is random. Assuming that there is at most one order outstanding at any point in time, we first address the case with the general production time distribution. We show that the optimal order policy is characterized by a reorder point and the optimal rationing policy is characterized by time‐dependent rationing levels. We then approximate the production time distribution with a phase‐type distribution and show that the optimal policy can be characterized by a reorder point and state‐dependent rationing levels. Using the Erlang production time distribution, we generalize the model to a tandem MTS system in which there may be multiple outstanding orders. We introduce a state‐transformation approach to perform the structural analysis and show that both the reorder point and rationing levels are state dependent. We show the monotonicity of the optimal reorder point and rationing levels for the outstanding orders, and generate new theoretical and managerial insights from the research findings.     

Looking Upstream: Optimal Policies for a Class of Capacitated Multi‐Stage Inventory Systems

We consider a multi‐stage inventory system with stochastic demand and processing capacity constraints at each stage, for both finite‐horizon and infinite‐horizon, discounted‐cost settings. For a class of such systems characterized by having the smallest capacity at the most downstream stage and system utilization above a certain threshold, we identify the structure of the optimal policy, which represents a novel variation of the order‐up‐to policy. We find the explicit functional form of the optimal order‐up‐to levels, and show that they depend (only) on upstream echelon inventories. We establish that, above the threshold utilization, this optimal policy achieves the decomposition of the multidimensional objective cost function for the system into a sum of single‐dimensional convex functions. This decomposition eliminates the curse of dimensionality and allows us to numerically solve the problem. We provide a fast algorithm to determine a (tight) upper bound on this threshold utilization for capacity‐constrained inventory problems with an arbitrary number of stages. We make use of this algorithm to quantify upper bounds on the threshold utilization for three‐, four‐, and five‐stage capacitated systems over a range of model parameters, and discuss insights that emerge.     

OPTIMAL AND NEAR OPTIMAL CONTROL OF A TWO-PART-TYPE STOCHASTIC MANUFACTURING SYSTEM WITH DYNAMIC SETUPS

This paper deals with a manufacturing system consisting of a single machine subject to random failures and repairs. The machine can produce two types of parts. When the production is switched from one part type to the other, a random setup time is incurred at a constant cost rate. The objective is to track the demand, while keeping the work-in-process as close as possible to zero for both products. The problem is formulated as an optimal stochastic control problem. The optimal policy is obtained numerically by discretizing the continuous time continuous state opti-mality conditions using a Markov chain approximation technique. The discretized optimality conditions are shown to correspond to an infinite horizon, discrete time, discrete state dynamic programming problem. The optimal setup policy is shown to have two different structures depending on the parameters of the system. A heuristic policy is proposed to approximate the optimal setup policy. Simulation results show that the heuristic policy is a very good approximation for sufficiently reliable systems.     

Dynamic Pricing and Inventory Management with Regular and Expedited Supplies

We consider a periodic‐review inventory system with regular and expedited supply modes. The expedited supply is faster than the regular supply but incurs a higher cost. Demand for the product in each period is random and sensitive to its selling price. The firm determines its order quantity from each supply in each period as well as its selling price to maximize the expected total discounted profit over a finite or an infinite planning horizon. We show that, in each period if it is optimal to order from both supplies, the optimal inventory policy is determined by two state‐independent thresholds, one for each supply mode, and a list price is set for the product; if only the regular supply is used, the optimal policy is a state‐dependent base‐stock policy, that is, the optimal base‐stock level depends on the starting inventory level, and the optimal selling price is a markdown price that decreases with the starting inventory level. We further study the operational impact of such supply diversification and show that it increases the firm's expected profit, reduces the optimal safety‐stock levels, and lowers the optimal selling price. Thus that diversification is beneficial to both the firm and its customers. Building upon these results, we conduct a numerical study to assess and compare the respective benefit of dynamic pricing and supply diversification.     

具有多元马氏需求的多产品多阶段库存优化模型

本文研究一类新的多产品库存控制策略,即具有多元马氏需求特征的多产品多阶段的订货点订货量(Q, R, SS)策略,该策略考虑市场需求在不同产品之间具有多元马氏转移特征,并考虑缺货因素设置安全库存。论文首先建立了多产品多阶段的多元马氏需求预测模型,并通过该模型确定了各种产品需求之间的关系。同时,在该模型的理论基础上,提出了多产品多阶段的总期望成本模型及其最优(Q, R, SS)策略,进而结合算例给出模型的最优策略的数值解。     

Coordinated Logistics: Joint Replenishment with Capacitated Transportation for a Supply Chain

In this study, we consider the integrated inventory replenishment and transportation operations in a supply chain where the orders placed by the downstream retailer are dispatched by the upstream warehouse via an in‐house fleet of limited size. We first consider the single‐item single‐echelon case where the retailer operates with a quantity based replenishment policy, (r,Q), and the warehouse is an ample supplier. We model the transportation operations as a queueing system and derive the operating characteristics of the system in exact terms. We extend this basic model to a two‐echelon supply chain where the warehouse employs a base‐stock policy. The departure process of the warehouse is characterized in distribution, which is then approximated by an Erlang arrival process by matching the first two moments for the analysis of the transportation queueing system. The operating characteristics and the expected cost rate are derived. An extension of this system to multiple retailers is also discussed. Numerical results are presented to illustrate the performance and the sensitivity of the models and the value of coordinating inventory and transportation operations.     

INFLATION STABILIZATION AND WELFARE: THE CASE OF A DISTORTED STEADY STATE

This paper considers the appropriate stabilization objectives for monetary policy in a micro‐founded model with staggered price‐setting. Rotemberg and Woodford (1997) and Woodford (2002) have shown that under certain conditions, a local approximation to the expected utility of the representative household in a model of this kind is related inversely to the expected discounted value of a conventional quadratic loss function, in which each period's loss is a weighted average of squared deviations of inflation and an output gap measure from their optimal values (zero). However, those derivations rely on an assumption of the existence of an output or employment subsidy that offsets the distortion due to the market power of monopolistically competitive price‐setters, so that the steady state under a zero‐inflation policy involves an efficient level of output. Here we show how to dispense with this unappealing assumption, so that a valid linear‐quadratic approximation to the optimal policy problem is possible even when the steady state is distorted to an arbitrary extent (allowing for tax distortions as well as market power), and when, as a consequence, it is necessary to take account of the effects of stabilization policy on the average level of output. We again obtain a welfare‐theoretic loss function that involves both inflation and an appropriately defined output gap, though the degree of distortion of the steady state affects both the weights on the two stabilization objectives and the definition of the welfare‐relevant output gap. In the light of these results, we reconsider the conditions under which complete price stability is optimal, and find that they are more restrictive in the case of a distorted steady state. We also consider the conditions under which pure randomization of monetary policy can be welfare‐improving, and find that this is possible in the case of a sufficiently distorted steady state, though the parameter values required are probably not empirically realistic. (JEL: D61, E52, E61)     

Limiting Exposure to Collateral Liability Associated With ISO 14001 EMS Audits

The purpose of this article is to provide suggestions on how to limit the collateral liability associated with ISO 14001 environmental management system (EMS) auditing activities. Due to the detailed, invasive nature of EMS audits, companies performing such activities may expose themselves to collateral risks. Collateral risk is liability not directly associated with ISO 14001 activities, but borne through the unrelated process of EMS audit activities. Examples of collateral risks include issues of non-compliance, unintended discovery of evidence in civil tort claims and exposure to regulatory sanctions. Therefore, well-defined management of EMS audits represents an important aspect of limiting risk. This article identifies several strategies for managing and reducing these legal risks. These strategies include the rigorous management of compliance audits, concisely defining audit scopes, training auditors and auditees, understanding and using audit privileges as appropriate, exploring the use of self-declaration (for ISO 14001 conformance purposes), and using reliable document control methods. While completely eliminating liability represents a laudable but challenging goal, these methods should reduce collateral exposure to corporate and personal liability.     

Experimental Results Indicating Lattice‐Dependent Policies May Be Optimal for General Assemble‐To‐Order Systems

We consider an assemble‐to‐order (ATO) system with multiple products, multiple components which may be demanded in different quantities by different products, possible batch ordering of components, random lead times, and lost sales. We model the system as an infinite‐horizon Markov decision process under the average cost criterion. A control policy specifies when a batch of components should be produced, and whether an arriving demand for each product should be satisfied. Previous work has shown that a lattice‐dependent base‐stock and lattice‐dependent rationing (LBLR) policy is an optimal stationary policy for a special case of the ATO model presented here (the generalized M‐system). In this study, we conduct numerical experiments to evaluate the use of an LBLR policy for our general ATO model as a heuristic, comparing it to two other heuristics from the literature: a state‐dependent base‐stock and state‐dependent rationing (SBSR) policy, and a fixed base‐stock and fixed rationing (FBFR) policy. Remarkably, LBLR yields the globally optimal cost in each of more than 22,500 instances of the general problem, outperforming SBSR and FBFR with respect to both objective value (by up to 2.6% and 4.8%, respectively) and computation time (by up to three orders and one order of magnitude, respectively) in 350 of these instances (those on which we compare the heuristics). LBLR and SBSR perform significantly better than FBFR when replenishment batch sizes imperfectly match the component requirements of the most valuable or most highly demanded product. In addition, LBLR substantially outperforms SBSR if it is crucial to hold a significant amount of inventory that must be rationed.     

Analysis and optimization of an ambulance offload delay and allocation problem

Ambulance offload delays have recently become one of the most significant operational challenges for Emergency Medical Services (EMS) providers. Offload delays occur when an ambulance arriving at a hospital Emergency Department (ED) is blocked until a bed becomes available for the patient. To formally investigate the effect of patient routing decisions on EMS offload delays, we introduce a stylized queueing network model with blocking. Following a decomposition approach, we develop an approximation scheme to find explicit solutions that can be used to find proper patient allocation policies to multiple hospitals in a region. We introduce a Markov chain representation for a single ED network and solve for its exact steady state distribution. A comprehensive numerical study is carried out to validate the approximation approaches and to gain insight into ambulance offload delays. By keeping the total offload delays at minimal levels, we observe that it is better to load larger EDs more heavily than smaller ones due to resource pooling.     

Myopic Analysis for Multi‐Echelon Inventory Systems with Batch Ordering and Nonstationary/Time‐Correlated Demands

We provide an exact myopic analysis for an N‐stage serial inventory system with batch ordering, linear ordering costs, and nonstationary demands under a finite planning horizon. We characterize the optimality conditions of the myopic nested batching newsvendor (NBN) policy and the myopic independent batching newsvendor (IBN) policy, which is a single‐stage approximation. We show that echelon reorder levels under the NBN policy are upper bounds of the counterparts under both the optimal policy and the IBN policy. In particular, we find that the IBN policy has bounded deviations from the optimal policy. We further extend our results to systems with martingale model of forecast evolution (MMFE) and advance demand information. Moreover, we provide a recursive computing procedure and optimality conditions for both heuristics which dramatically reduces computational complexity. We also find that the NBN problem under the MMFE faced by one stage has one more dimension for the forecast demand than the one faced by its downstream stage and that the NBN policy is optimal for systems with advance demand information and stationary problem data. Numerical studies demonstrate that the IBN policy outperforms on average the NBN policy over all tested instances when their optimality conditions are violated.     

Ambulance allocation for maximal survival with heterogeneous outcome measures

This paper proposes new models for locating emergency medical services (EMS) by incorporating survival functions for capturing multiple-classes of heterogeneous patients. The Maximal Expected Survival Location Model for Heterogeneous Patients (MESLMHP) aims to maximize the overall expected survival probability of multiple-classes of patients, whereby different classes could be defined according to agreed patient categories based on response time targets, or by capturing differing medical conditions each with a corresponding survival function. Furthermore, we propose and demonstrate an approximation approach to solving the extended stochastic version of MESLMHP, which utilizes queuing theory to permit the modeling of congestion and utilization at each ambulance station, and does not require assumptions to be made on the utilization of ambulances. Both models are demonstrated using data from the ambulance service in Wales. We show that our multiple outcome measures and survival-maximizing approach, rather than one based on average response time targets alone or a single patient class provides more effective EMS ambulance allocations.     

Heuristics for Base‐Stock Levels in Multi‐Echelon Distribution Networks

We study inventory optimization for locally controlled, continuous‐review distribution systems with stochastic customer demands. Each node follows a base‐stock policy and a first‐come, first‐served allocation policy. We develop two heuristics, the recursive optimization (RO) heuristic and the decomposition‐aggregation (DA) heuristic, to approximate the optimal base‐stock levels of all the locations in the system. The RO heuristic applies a bottom‐up approach that sequentially solves single‐variable, convex problems for each location. The DA heuristic decomposes the distribution system into multiple serial systems, solves for the base‐stock levels of these systems using the newsvendor heuristic of Shang and Song (2003), and then aggregates the serial systems back into the distribution system using a procedure we call “backorder matching.” A key advantage of the DA heuristic is that it does not require any evaluation of the cost function (a computationally costly operation that requires numerical convolution). We show that, for both RO and DA, changing some of the parameters, such as leadtime, unit backordering cost, and demand rate, of a location has an impact only on its own local base‐stock level and its upstream locations’ local base‐stock levels. An extensive numerical study shows that both heuristics perform well, with the RO heuristic providing more accurate results and the DA heuristic consuming less computation time. We show that both RO and DA are asymptotically optimal along multiple dimensions for two‐echelon distribution systems. Finally, we show that, with minor changes, both RO and DA are applicable to the balanced allocation policy.     

Dynamic Policies for Optimal LEO Satellite Launches

Low‐earth orbit satellite (LEO) systems continue to provide mobile communication services. The issue of cost containment in system maintenance is a critical factor for continued operation. Satellite finite life‐times follow a stochastic process, and since satellite replenishment cost is the most significant on‐going cost of operation, finding optimal launch policies is of paramount importance. This paper formulates the satellite launch problem as a Markovian decision model that can be solved using dynamic programming. The policy space of the system is enormous and traditional action space dominance rules do not apply. In order to solve the dynamic program for realistic problem sizes, a novel procedure for limiting the state space considered in the dynamic program is developed. The viability of the proposed solution procedure is demonstrated in example problems using realistic system data. The policies derived by the proposed solution procedure are superior to those currently considered by LEO system operators, and result in substantial annual cost savings.     

Unsold Versus Unbought Commitment: Minimum Total Commitment Contracts with Nonzero Setup Costs

We study a minimum total commitment (MTC) contract embedded in a finite‐horizon periodic‐review inventory system. Under this contract, the buyer commits to purchase a minimum quantity of a single product from the supplier over the entire planning horizon. We consider nonstationary demand and per‐unit cost, discount factor, and nonzero setup cost. Because the formulations used in existing literature are unable to handle our setting, we develop a new formulation based on a state transformation technique using unsold commitment instead of unbought commitment as state variable. We first revisit the zero setup cost case and show that the optimal ordering policy is an unsold‐commitment‐dependent base‐stock policy. We also provide a simpler proof of the optimality of the dual base‐stock policy. We then study the nonzero setup cost case and prove a new result, that the optimal solution is an unsold‐commitment‐dependent (s, S) policy. We further propose two heuristic policies, which numerical tests show to perform very well. We also discuss two extensions to show the generality of our method's effectiveness. Finally, we use our results to examine the effect of different contract terms such as duration, lead time, and commitment on buyer's cost. We also compare total supply chain profits under periodic commitment, MTC, and no commitment.     

EFFECTS OF RANDOM SHIFTS OF TESTING EQUIPMENT ON PROCESS CONTROL SYSTEM DESIGN AND SELECTION OF PROCESS CONTROL POLICIES*

This paper studies issues associated with designing process control systems when the testing equipment is subjected to random shifts. We consider a production process with two states: in control and out of control. The process may shift randomly to the out‐of‐control state over time. The process is monitored by periodically sampling finished items from the process. The equipment used to test sampled items also is assumed to have two states and may shift randomly during the testing process. We formulate a cost model for finding the optimal process control policy that minimizes the expected unit time cost. Numerical results show that shifts of the testing equipment may significantly affect the performance of a process control policy. We also studied the effects of the testing equipment's shifts on the selection of process control policies.     

Decision Making and the Price Setting Newsvendor: Experimental Evidence

We present an experimental study of the price‐setting newsvendor problem, which extends the traditional framework by allowing the decision maker to determine both the selling price and the order quantity of a given item. We compare behavior under this model with two benchmark conditions where subjects have a single decision to make (price or quantity). We observe that subjects deviate from the theoretical benchmarks when they are tasked with a single decision. They also exhibit anchoring behavior, where their anchor is the expected demand when quantity is the decision variable and is the initial inventory level when price is the decision variable. When decision makers set quantity and price concurrently, we observe no significant difference between the normative (i.e., expected profit‐maximizing) prices and the decision makers’ price choices. Quantity decisions move further from the normative benchmarks (compared to when subjects have a single decision to make) when the ratio of cost to price is less than half. When this ratio is reversed, there is no significant difference between order levels in single‐ and multi‐task settings. In the multidecision framework, we also observe a tendency to match orders and expected demand levels, which subjects can control using prices.     

Storing Fresh Produce for Fast Retrieval in an Automated Compact Cross‐Dock System

We study temporary storage of fresh produce in a cross‐dock center. In order to minimize cooling cost, compact storage systems are used. A major disadvantage of these systems is that additional retrieval time is needed, caused by necessary reshuffles due to the improper storage sequence of unit loads. In practice therefore, a dedicated storage policy is used in which every storage lane in the system accommodates only one product. However, this policy does not use the planned arrival time information of the outbound trucks. To exploit this information, this study proposes a mathematical model for a shared storage policy that minimizes total retrieval time. The policy allows different products to share the same lane. In order to solve real‐sized problems, an effective and efficient heuristic is proposed, based on a greedy construction and an improvement part, which provides near optimal solutions. The gaps between the results of the heuristic and the lower bound are mostly less than 1%. The resulting shared storage policy is generally robust against disturbances in arrival or departure times. We compare our shared storage heuristic with dedicated storage to determine which policy performs best under which circumstances. For most practical cases, shared storage appears to outperform dedicated storage, with a shorter response time and better storage lane utilization.