Analyses of the Markov modulated fluid flow with one-sided ph-type jumps using coupled queues and the completed graphs

In this paper, we analyze Markov modulated fluid flow processes with one-sided ph-type jumps using the completed graph and also through the limits of coupled queueing processes to be constructed. For the models, we derive various results on time-dependent distributions and distributions of first passage times, and present the Riccati equations that transform matrices of the first return times to 0 satisfy. The Riccati equations enable us to compute the transform matrices using Newton’s method which is known very fast and stable. Finally, we present some duality results between the model with ph-type downward jumps and the model with ph-type upward jumps. This paper contains extended results of Ahn (2009) and probabilistic interpretations given by the completed graphs.     

Total shift during the first passages of Markov-modulated Brownian motion with bilateral ph-type jumps: Formulas driven by the minimal solution matrix of a Riccati equation

This article describes our study of the total shift during the first passages (one-sided and two-sided exit times) of Markov-modulated Brownian motion with bilateral ph-type jumps, which is referred to as MMBM. The total shift is defined as the value of a so-called shift process at the first passage epochs of the MMBM. The shift process, introduced by Bean and O’Reilly, behaves like a continuous Markovian fluid process; that is, it increases or decreases linearly with slopes regulated by the underlying Markov process that determines the path of the MMBM. Hence, the notion of total shift, which includes the first passage times of the MMBM as special cases, is useful for describing various performance measures of systems modeled by the MMBM. In this article, we present formulas for the Laplace–Stieltjes transform matrices of the total shift during various first passages of the MMBM. In particular, a Riccati equation is derived so that a matrix associated with the Laplace–Stieltjes transform of the total shift during the first return time of the MMBM is its minimal non-negative solution matrix. With this solution matrix, the Laplace–Stieltjes transform matrices can be obtained without much additional work. Furthermore, it is shown that the Riccati equation satisfies the conditions for the Newton scheme to have quadratic convergence, which enables us to use algorithms with quadratic convergence, such as Newton’s method and the Stochastic Doubling Algorithm, to compute the presented matrix-driven formulas. For the analyses, we take an approach based on approximating the MMBM with a sequence of scaled Markov-modulated fluid flows with bilateral ph-type jumps, referred to as MMFF, that weakly converge to the MMBM. Another contribution of this article is that duality results are derived in relation to the MMBM, which is an extension of the duality theorems developed by Ahn and Ramaswami for an MMFF without a jump.     

Factor stochastic volatility with time varying loadings and Markov switching regimes

We generalize the factor stochastic volatility (FSV) model of Pitt and Shephard [1999. Time varying covariances: a factor stochastic volatility approach (with discussion). In: Bernardo, J.M., Berger, J.O., Dawid, A.P., Smith, A.F.M. (Eds.), Bayesian Statistics, vol. 6, Oxford University Press, London, pp. 547–570.] and Aguilar and West [2000. Bayesian dynamic factor models and variance matrix discounting for portfolio allocation. J. Business Econom. Statist. 18, 338–357.] in two important directions. First, we make the FSV model more flexible and able to capture more general time-varying variance–covariance structures by letting the matrix of factor loadings to be time dependent. Secondly, we entertain FSV models with jumps in the common factors volatilities through So, Lam and Li's [1998. A stochastic volatility model with Markov switching. J. Business Econom. Statist. 16, 244–253.] Markov switching stochastic volatility model. Novel Markov Chain Monte Carlo algorithms are derived for both classes of models. We apply our methodology to two illustrative situations: daily exchange rate returns [Aguilar, O., West, M., 2000. Bayesian dynamic factor models and variance matrix discounting for portfolio allocation. J. Business Econom. Statist. 18, 338–357.] and Latin American stock returns [Lopes, H.F., Migon, H.S., 2002. Comovements and contagion in emergent markets: stock indexes volatilities. In: Gatsonis, C., Kass, R.E., Carriquiry, A.L., Gelman, A., Verdinelli, I. Pauler, D., Higdon, D. (Eds.), Case Studies in Bayesian Statistics, vol. 6, pp. 287–302].     

Distribution of busy period in stochastic fluid models

We consider the busy period in a stochastic fluid flow model with infinite buffer where the input and output rates are controlled by a finite homogeneous Markov process. We derive an explicit expression for the distribution of the busy period and we obtain an algorithm to compute it which exhibits nice numerical properties.

     

ALGORITHMS FOR RETURN PROBABILITIES FOR STOCHASTIC FLUID FLOWS

Abstract

We consider several known algorithms and introduce some new algorithms that can be used to calculate the probability of return to the initial level in the Markov stochastic fluid flow model. We give the physical interpretations of these algorithms within the fluid flow environment. The rates of convergence are explained in terms of the physical properties of the fluid flow processes. We compare these algorithms with respect to the number of iterations required and their complexity. The performance of the algorithms depends on the nature of the process considered in the analysis. We illustrate this with examples and give appropriate recommendations.     

Fluid Flow Models and Queues—A Connection by Stochastic Coupling

We establish in a direct manner that the steady state distribution of Markovian fluid flow models can be obtained from a quasi birth and death queue. This is accomplished through the construction of the processes on a common probability space and the demonstration of a distributional coupling relation between them. The results here provide an interpretation for the quasi-birth-and-death processes in the matrix-geometric approach of Ramaswami and subsequent results based on them obtained by Soares and Latouche.     

Critical Scale for a Continuous AIMD Model

A scaled version of the general AIMD model of transmission control protocol (TCP) used in Internet traffic congestion management leads to a Markov process x(t) representing the time dependent data flow that moves forward with constant speed on the positive axis and jumps backward to γx(t), 0 < γ < 1 according to a Poisson clock whose rate α(x) depends on the interval swept in between jumps. We give sharp conditions for Harris recurrence and analyze the convergence to equilibrium on multiple scales (polynomial, fractional exponential, exponential) identifying the critical case xα(x) ～ β. Criticality has different behavior according to whether it occurs at the origin or infinity. In each case, we determine the transient (possibly explosive), null—and positive—recurrent regimes by comparing β to ( ? ln?γ)^{? 1}.     

Robust Model Selection for Stochastic Processes

We address the problem of robust model selection for finite memory stochastic processes. Consider m independent samples, with most of them being realizations of the same stochastic process with law Q, which is the one we want to retrieve. We define the asymptotic breakdown point γ for a model selection procedure and also we devise a model selection procedure. We compute the value of γ which is 0.5, when all the processes are Markovian. This result is valid for any family of finite order Markov models but for simplicity we will focus on the family of variable length Markov chains.     

A phase expansion for non-Markovian availability models with time-based aperiodic rejuvenation and checkpointing

Abstract

This paper presents a stochastic framework, consisting of stochastic reward net (SRN) for capturing the transient behaviors of the system and its related non-Markovian state transition diagram, to model an operational software system that undergoes aperiodic time-based rejuvenation and checkpointing schemes, and further to investigate whether there exists the optimal rejuvenation schedule that maximizes the system steady-state availability. A phase expansion approach is adopted to solve the non-Markovian availability models, which are actually neither the semi-Markov processes nor the Markov regenerative processes. Our numerical results show an appropriate rejuvenation trigger timing range, resulting in the positive improvement effect on the system availability of a database system, and that there exists the optimal rejuvenation trigger timing maximizing the system availability.     

Empirical Performance and Asset Pricing in Hidden Markov Models

Abstract

To improve the empirical performance of the Black-Scholes model, many alternative models have been proposed to address leptokurtic feature, volatility smile, and volatility clustering effects of the asset return distributions. However, analytical tractability remains a problem for most alternative models. In this article, we study a class of hidden Markov models including Markov switching models and stochastic volatility models, that can incorporate leptokurtic feature, volatility clustering effects, as well as provide analytical solutions to option pricing. We show that these models can generate long memory phenomena when the transition probabilities depend on the time scale. We also provide an explicit analytic formula for the arbitrage-free price of the European options under these models. The issues of statistical estimation and errors in option pricing are also discussed in the Markov switching models.     

Piecewise deterministic Markov processes applied to fatigue crack growth modelling

In this paper, we use a particular piecewise deterministic Markov process (PDMP) to model the evolution of a degradation mechanism that may arise in various structural components, namely, the fatigue crack growth. We first derive some probability results on the stochastic dynamics with the help of Markov renewal theory: a closed-form solution for the transition function of the PDMP is given. Then, we investigate some methods to estimate the parameters of the dynamical system, involving Bogolyubov's averaging principle and maximum likelihood estimation for the infinitesimal generator of the underlying jump Markov process. Numerical applications on a real crack data set are given.     

A threshold autoregressive model for wholesale electricity prices

Summary.  We introduce a discrete time model for electricity prices which accounts for both transitory spikes and temperature effects. The model allows for different rates of mean reversion: one for weather events, one around price jumps and another for the remainder of the process. We estimate the model by using a Markov chain Monte Carlo approach with 3 years of daily data from Allegheny County, Pennsylvania. We show that our model outperforms existing stochastic jump diffusion models for this data set. Results also demonstrate the importance of model parameters corresponding to both the temperature effect and the multilevel mean reversion rate.     

A new characterization of the jump rate for piecewise-deterministic Markov processes with discrete transitions

Piecewise-deterministic Markov processes form a general class of non diffusion stochastic models that involve both deterministic trajectories and random jumps at random times. In this paper, we state a new characterization of the jump rate of such a process with discrete transitions. We deduce from this result a non parametric technique for estimating this feature of interest. We state the uniform convergence in probability of the estimator. The methodology is illustrated on a numerical example.     

Costing Mixed Coxian Phase-type Systems with Poisson Arrivals

Previously, we developed a modeling framework which classifies individuals with respect to their length of stay (LOS) in the transient states of a continuous-time Markov model with a single absorbing state; phase-type models are used for each class of the Markov model. We here add costs and obtain results for moments of total costs in (0, t], for an individual, a cohort arriving at time zero and when arrivals are Poisson. Based on stroke patient data from the Belfast City Hospital we use the overall modelling framework to obtain results for total cost in a given time interval.     

Convergence in the Wasserstein Metric for Markov Chain Monte Carlo Algorithms with Applications to Image Restoration

Abstract

In this paper, we show how the time for convergence to stationarity of a Markov chain can be assessed using the Wasserstein metric, rather than the usual choice of total variation distance. The Wasserstein metric may be more easily applied in some applications, particularly those on continuous state spaces. Bounds on convergence time are established by considering the number of iterations required to approximately couple two realizations of the Markov chain to within ε tolerance. The particular application considered is the use of the Gibbs sampler in the Bayesian restoration of a degraded image, with pixels that are a continuous grey-scale and with pixels that can only take two colours. On finite state spaces, a bound in the Wasserstein metric can be used to find a bound in total variation distance. We use this relationship to get a precise O(N log N) bound on the convergence time of the stochastic Ising model that holds for appropriate values of its parameter as well as other binary image models. Our method employing convergence in the Wasserstein metric can also be applied to perfect sampling algorithms involving coupling from the past to obtain estimates of their running times.     

Bayesian fractional polynomials

This paper sets out to implement the Bayesian paradigm for fractional polynomial models under the assumption of normally distributed error terms. Fractional polynomials widen the class of ordinary polynomials and offer an additive and transportable modelling approach. The methodology is based on a Bayesian linear model with a quasi-default hyper-g prior and combines variable selection with parametric modelling of additive effects. A Markov chain Monte Carlo algorithm for the exploration of the model space is presented. This theoretically well-founded stochastic search constitutes a substantial improvement to ad hoc stepwise procedures for the fitting of fractional polynomial models. The method is applied to a data set on the relationship between ozone levels and meteorological parameters, previously analysed in the literature.     

Consistency of Bayesian linear model selection with a growing number of parameters

Linear models with a growing number of parameters have been widely used in modern statistics. One important problem about this kind of model is the variable selection issue. Bayesian approaches, which provide a stochastic search of informative variables, have gained popularity. In this paper, we will study the asymptotic properties related to Bayesian model selection when the model dimension p is growing with the sample size n. We consider p≤n and provide sufficient conditions under which: (1) with large probability, the posterior probability of the true model (from which samples are drawn) uniformly dominates the posterior probability of any incorrect models; and (2) the posterior probability of the true model converges to one in probability. Both (1) and (2) guarantee that the true model will be selected under a Bayesian framework. We also demonstrate several situations when (1) holds but (2) fails, which illustrates the difference between these two properties. Finally, we generalize our results to include g-priors, and provide simulation examples to illustrate the main results.     

The generalized linear chirp process

The linear chirp process is an important class of time series for which the instantaneous frequency changes linearly in time. Linear chirps have been used extensively to model a variety of physical signals such as radar, sonar, and whale clicks (see 1, 5 and 6). We introduce the stochastic linear chirp model and then define the generalized linear chirp (GLC) process as a special case of the G-stationary process studied by Jiang et al. (2006) to model data with time-varying frequencies. We then define GLC(p,q) processes and show that the relationship between stochastic linear chirp processes and GLC(p,q) processes is analogous to that between harmonic and ARMA models. The new methods are then applied to both simulated and actual data sets.     

On conditional variance estimation in nonparametric regression

In this paper we consider a nonparametric regression model in which the conditional variance function is assumed to vary smoothly with the predictor. We offer an easily implemented and fully Bayesian approach that involves the Markov chain Monte Carlo sampling of standard distributions. This method is based on a technique utilized by Kim, Shephard, and Chib (in Rev. Econ. Stud. 65:361–393, 1998) for the stochastic volatility model. Although the (parametric or nonparametric) heteroscedastic regression and stochastic volatility models are quite different, they share the same structure as far as the estimation of the conditional variance function is concerned, a point that has been previously overlooked. Our method can be employed in the frequentist context and in Bayesian models more general than those considered in this paper. Illustrations of the method are provided.     

A stochastic graph process for epidemic modelling

A stochastic graph process with a Markov property is introduced to model the flow of an infectious disease over a known contact network. The model provides a probability distribution over unobserved infectious pathways. The basic reproductive number in compartmental models is generalized to a dynamic reproductive number based on the sequence of outdegrees in the graph process. The cumulative resistance and threat associated with each individual is also measured based on the cumulative indegree and outdegree of the graph process. The model is applied to the outbreak data from the 2001 foot‐and‐mouth (FMD) outbreak in the United Kingdom. The Canadian Journal of Statistics 40: 55–67; 2012 © 2012 Statistical Society of Canada