Estimation in a model for a semi-Markov process with covariates under right-censoring

A semi-Markov model with covariates is proposed for a multi-state process with a finite number of states such that the transition probabilities between the states and the distribution functions of the duration times between the occurrence of two states depend on a discrete covariate. The hazard rates for the time elapsed between two successive states depend on the covariate through a proportional hazards model involving a set of regression parameters, while the transition probabilities depend on the covariate in an unspecified way. We propose estimators for these parameters and for the cumulative hazard functions of the sojourn times. A difficulty comes from the fact that when a sojourn time in a state is right-censored, the next state is unknown. We prove that our estimators are consistent and asymptotically Gaussian under the model constraints.     

Smoothed nonparametric estimation in window censored semi-Markov processes

Consider a process that jumps among a finite set of states, with random times spent in between. In semi-Markov processes transitions follow a Markov chain and the sojourn distributions depend only on the connecting states. Suppose that the process started far in the past, achieving stationary. We consider non-parametric estimation by modelling the log-hazard of the sojourn times through linear splines; and we obtain maximum penalized likelihood estimators when data consist of several i.i.d. windows. We prove consistency using Grenander's method of sieves.     

Estimating expected sojourn times for a markov renewal process

We consider the estimation of the expected sojourn time in a Markov renewal process under the data condition that only the counts of the exits from the states are available for fixed intervals of time. For analytical and illustrative purposes we concentrate on the two-state process case. We present least squares and method of moments estimators and compare their statistical properties both analytically and empirically. We also present modified estimators with improved properties based upon an overlapping interval sampling strategy. The major results indicate that the least squares estimator is biased in general with the bias depending on the size of the sampling interval and the first two moments of the sojourn time distribution function. The bias becomes negligible as the size of the sampling interval increases. Analytical and empirical results indicate that the method of moments estimator is less sensitive to the size of the sampling interval and has slightly better mean squared error properties than the least squares estimator.     

Estimation of the Entropy Rate of a Countable Markov Chain

We consider here ergodic homogeneous Markov chains with countable state spaces. The entropy rate of the chain is an explicit function of its transition and stationary distributions. We construct estimators for this entropy rate and for the entropy of the stationary distribution of the chain, in the parametric and nonparametric cases. We study estimation from one sample with long length and from many independent samples with given length. In the parametric case, the estimators are deduced by plug-in from the maximum likelihood estimator of the parameter. In the nonparametric case, the estimators are deduced by plug-in from the empirical estimators of the transition and stationary distributions. They are proven to have good asymptotic properties.     

Conditional failure occurrence rates for semi-Markov chains*

ABSTRACT

In the present paper, we aim at providing plug-in-type empirical estimators that enable us to quantify the contribution of each operational or/and non-functioning state to the failures of a system described by a semi-Markov model. In the discrete-time and finite state space semi-Markov framework, we study different conditional versions of an important reliability measure for random repairable systems, the failure occurrence rate, which is based on counting processes. The identification of potential failure contributors through the conditional counterparts of the failure occurrence rate is of paramount importance since it could lead to corrective actions that minimize the occurrence of the more important failure modes and therefore improve the reliability of the system. The aforementioned estimators are characterized by appealing asymptotic properties such as strong consistency and asymptotic normality. We further obtain detailed analytical expressions for the covariance matrices of the random vectors describing the conditional failure occurrence rates. As particular cases we present the failure occurrence rates for hidden (semi-) Markov models. We illustrate our results by means of a simulated study. Different applications are presented based on wind, earthquake and vibration data.     

Estimating models based on Markov jump processes given fragmented observation series

We consider the problem of estimating the rate matrix governing a finite-state Markov jump process given a number of fragmented time series. We propose to concatenate the observed series and to employ the emerging non-Markov process for estimation. We describe the bias arising if standard methods for Markov processes are used for the concatenated process, and provide a post-processing method to correct for this bias. This method applies to discrete-time Markov chains and to more general models based on Markov jump processes where the underlying state process is not observed directly. This is demonstrated in detail for a Markov switching model. We provide applications to simulated time series and to financial market data, where estimators resulting from maximum likelihood methods and Markov chain Monte Carlo sampling are improved using the presented correction.     

Estimation with right‐censored observations under a semi‐Markov model

The semi‐Markov process often provides a better framework than the classical Markov process for the analysis of events with multiple states. The purpose of this paper is twofold. First, we show that in the presence of right censoring, when the right end‐point of the support of the censoring time is strictly less than the right end‐point of the support of the semi‐Markov kernel, the transition probability of the semi‐Markov process is nonidentifiable, and the estimators proposed in the literature are inconsistent in general. We derive the set of all attainable values for the transition probability based on the censored data, and we propose a nonparametric inference procedure for the transition probability using this set. Second, the conventional approach to constructing confidence bands is not applicable for the semi‐Markov kernel and the sojourn time distribution. We propose new perturbation resampling methods to construct these confidence bands. Different weights and transformations are explored in the construction. We use simulation to examine our proposals and illustrate them with hospitalization data from a recent cancer survivor study. The Canadian Journal of Statistics 41: 237–256; 2013 © 2013 Statistical Society of Canada     

SOJOURN TIMES IN NON-HOMOGENEOUS QBD PROCESSES WITH PROCESSOR SHARING

We study sojourn times of customers in a processor sharing queue with a service rate that varies over time, depending on the number of customers and on the state of a random environment. An explicit expression is derived for the Laplace–Stieltjes transform of the sojourn time conditional on the state upon arrival and the amount of work brought into the system. Particular attention is paid to the conditional mean sojourn time of a customer as a function of his required amount of work, and we establish the existence of an asymptote as the amount of work tends to infinity. The method of random time change is then extended to include the possibility of a varying service rate. By means of this method, we explain the well-established proportionality between the conditional mean sojourn time and required amount of work in processor sharing queues without random environment. Based on numerical experiments, we propose an approximation for the conditional mean sojourn time. Although first presented for exponentially distributed service requirements, the analysis is shown to extend to phase-type services. The service discipline of discriminatory processor sharing is also shown to fall within the framework.     

Nonparametric estimation of waiting time distributions in a Markov model based on current status data

We construct nonparametric estimators of state waiting time distribution functions in a Markov multistate model using current status data. This is a particularly difficult problem since neither the entry nor the exit times of a given state are directly observed. These estimators are obtained, using the Markov property, from estimators of counting processes of state entry and exit times, as well as, the size of “at risk” sets of state entry and transitions out of that state. Consistency of our estimators is established. Finite-sample behavior of our estimators is studied by simulation, in which we show that our estimators based on current status data compare well with those based on complete data. We also illustrate our method using a pubertal development data set obtained from the NHANES III [1997. NHANES III Reference Manuals and Reports (CD-ROM). Analytic and Reporting Guidelines: The Third National Health and Nutrition Examination Survey (1988–94). National Center for Health Statistics, Centers for Disease Control and Prevention, Hyattsville, MD] study.     

On the functional central limit theorem for first passage time of nonlinear semi-Markov reward processes

Abstract

In this article we examine the functional central limit theorem for the first passage time of reward processes defined over a finite state space semi-Markov process. In order to apply this process for a wider range of real-world applications, the reward functions, considered in this work, are assumed to have general forms instead of the constant rates reported in the other studies. We benefit from the martingale theory and Poisson equations to prove and establish the convergence of the first passage time of reward processes to a zero mean Brownian motion. Necessary conditions to derive the results presented in this article are the existence of variances for sojourn times in each state and second order integrability of reward functions with respect to the distribution of sojourn times. We finally verify the presented methodology through a numerical illustration.     

Quasi- and pseudo-maximum likelihood estimators for discretely observed continuous-time Markov branching processes

This article deals with quasi- and pseudo-likelihood estimation for a class of continuous-time multi-type Markov branching processes observed at discrete points in time. “Conventional” and conditional estimation are discussed for both approaches. We compare their properties and identify situations where they lead to asymptotically equivalent estimators. Both approaches possess robustness properties, and coincide with maximum likelihood estimation in some cases. Quasi-likelihood functions involving only linear combinations of the data may be unable to estimate all model parameters. Remedial measures exist, including the resort either to non-linear functions of the data or to conditioning the moments on appropriate sigma-algebras. The method of pseudo-likelihood may also resolve this issue. We investigate the properties of these approaches in three examples: the pure birth process, the linear birth-and-death process, and a two-type process that generalizes the previous two examples. Simulations studies are conducted to evaluate performance in finite samples.     

Matrix-analytic Models and their Analysis

We survey phase-type distributions and Markovian point processes, aspects of how to use such models in applied probability calculations and how to fit them to observed data. A phase-type distribution is defined as the time to absorption in a finite continuous time Markov process with one absorbing state. This class of distributions is dense and contains many standard examples like all combinations of exponential in series/parallel. A Markovian point process is governed by a finite continuous time Markov process (typically ergodic), such that points are generated at a Poisson intensity depending on the underlying state and at transitions; a main special case is a Markov-modulated Poisson process. In both cases, the analytic formulas typically contain matrix-exponentials, and the matrix formalism carried over when the models are used in applied probability calculations as in problems in renewal theory, random walks and queueing. The statistical analysis is typically based upon the EM algorithm, viewing the whole sample path of the background Markov process as the latent variable.     

Long-Range Dependence of Markov Renewal Processes

This paper examines long‐range dependence (LRD) and asymptotic properties of Markov renewal processes generalizing results of Daley for renewal processes. The Hurst index and discrepancy function, which is the difference between the expected number of arrivals in (0, t] given a point at 0 and the number of arrivals in (0, t] in the time stationary version, are examined in terms of the moment index. The moment index is the supremum of the set of r > 0 such that the rth moment of the first return time to a state is finite, employing the solidarity results of Sgibnev. The results are derived for irreducible, regular Markov renewal processes on countable state spaces. The paper also derives conditions to determine the moment index of the first return times in terms of the Markov renewal kernel distribution functions of the process.     

Reliability evaluation of a Semi-Markov repairable system under alternative environments

ABSTRACT

To accurately describe the performance of repairable systems operating under alternative environments, for example, mild/harsh, working/idling, maximum/minimum level demand etc., a Semi-Markov process with a finite state space and two different Semi-Markov kernels is introduced. The state set of the system which is regarded as acceptable might depend on the environments. Two important reliability indices, the availability and time to the first system failure, are obtained via Markov renewal theory, transform and matrix methods. The results and numerical examples are also provided for two special cases: (1) when sojourn times under alternative environments are constants and (2) when sojourn times under environments have exponential distributions.     

A semi‐Markov model for binary longitudinal responses subject to misclassification

The authors propose a two‐state continuous‐time semi‐Markov model for an unobservable alternating binary process. Another process is observed at discrete time points that may misclassify the true state of the process of interest. To estimate the model's parameters, the authors propose a minimum Pearson chi‐square type estimating approach based on approximated joint probabilities when the true process is in equilibrium. Three consecutive observations are required to have sufficient degrees of freedom to perform estimation. The methodology is demonstrated on parasitic infection data with exponential and gamma sojourn time distributions.     

Estimation for semi-Markov models with partial observations via self-consistency equations

Nonparametric estimators of the survival function S(t) = P(T ≥ t) for a partially observed time variable T have been defined by several methods, in particular, by integral self-consistency equations. The author establishes explicit expressions of the estimators in an additive form and extend this approach to several cases: a left-truncated and right-censored variable and the left-censored or left-truncated sojourn times of a right-censored semi-Markov process. These estimators are always identical to the product-limit estimators if hazard functions may be defined.     

ASYMPTOTIC PROPERTIES FOR MAXIMUM LIKELIHOOD ESTIMATORS FOR RELIABILITY AND FAILURE RATES OF MARKOV CHAINS

ABSTRACT

In this paper, we shall study a homogeneous ergodic, finite state, Markov chain with unknown transition probability matrix. Starting from the well known maximum likelihood estimator of transition probability matrix, we define estimators of reliability and its measurements. Our aim is to show that these estimators are uniformly strongly consistent and converge in distribution to normal random variables. The construction of the confidence intervals for availability, reliability, and failure rates are also given. Finally we shall give a numerical example for illustration and comparing our results with the usual empirical estimator results.     

Estimation using plug-in of the stationary distribution and Shannon entropy of continuous time Markov processes

A natural way to deal with the uncertainty of an ergodic finite state space Markov process is to investigate the entropy of its stationary distribution. When the process is observed, it becomes necessary to estimate this entropy.We estimate both the stationary distribution and its entropy by plug-in of the estimators of the infinitesimal generator. Three situations of observation are discussed: one long trajectory is observed, several independent short trajectories are observed, or the process is observed at discrete times. The good asymptotic behavior of the plug-in estimators is established. We also illustrate the behavior of the estimators through simulation.     

On a multidimensional general bootstrap for empirical estimator of continuous-time semi-Markov kernels with applications

The present paper introduces a general notion and presents results of bootstrapped empirical estimators of the semi-Markov kernels and of the conditional transition distributions for semi-Markov processes with countable state space, constructed by exchangeably weighting the sample. Our proposal provides a unification of bootstrap methods in the semi-Markov setting including, in particular, Efron's bootstrap. Asymptotic properties of these generalised bootstrapped empirical distributions are obtained, under mild conditions by a martingale approach. We also obtain some new results on the weak convergence of the empirical semi-Markov processes. We apply these general results in several statistical problems such as the construction of confidence bands and the goodness-of-fit tests where the limiting distributions are derived under the null hypothesis. Finally, we introduce the quantile estimators and their bootstrapped versions in the semi-Markov framework and we establish their limiting laws by using the functional delta methods. Our theoretical results and numerical examples by simulations demonstrate the merits of the proposed techniques.