Markov Systems in a General State Space

In this article, we introduce and study Markov systems on general spaces (MSGS) as a first step of an entire theory on the subject. Also, all the concepts and basic results needed for this scope are given and analyzed. This could be thought of as an extension of the theory of a non homogeneous Markov system (NHMS) and that of a non homogeneous semi-Markov system on countable spaces, which has realized an interesting growth in the last thirty years. In addition, we study the asymptotic behaviour or ergodicity of Markov systems on general state spaces. The problem of asymptotic behaviour of Markov chains has been central for finite or countable spaces since the foundation of the subject. It has also been basic in the theory of NHMS and NHSMS. Two basic theorems are provided in answering the important problem of the asymptotic distribution of the population of the memberships of a Markov system that lives in the general space (X, ?(X)). Finally, we study the total variability from the invariant measure of the Markov system given that there exists an asymptotic behaviour. We prove a theorem which states that the total variation is finite. This problem is known also as the coupling problem.     

On the Variances and Convariances of the Duration State Sizes of Semi-Markov Systems

In the present article we study the asymptotic behaviour of the sequence of vectors with components expectations, variances, and covariances of the state sizes of a semi-Markov system. In this respect, we transform the semi-Markov system into a Markov system with a different though equivalent state space and relate the sequence of the transition probabilities of the respective Markov system as functions of the parameters of the semi-Markov system. Also, we study the asymptotic behavior of the sequence of vectors with components variances and covariances of the duration state sizes, of the related Markov system, under perturbation of the transition probability matrices. We use the results in the study of asymptotic behavior under perturbation of the sequence of vectors with components variances and covariances of the semi-Markov system.     

Statistics of Shape,Direction and Cylindrical Variables

In statistical shape analysis, the shape of an object is understood to be what remains after the effects of location, scale and rotation are removed. We consider the distributional problem of triangular shape and an associated direction; motivated by a data set of microscopic fossils. We begin by constructing a parallel transport system such that the data transform onto the space 𝒮²×𝒮². A joint shape distribution on 𝒮²×𝒮¹ is proposed based on Jupp & Mardia's bivariate distribution on 𝒮²×𝒮¹. For concentrated data, an approximation to the distribution on 𝒮²×𝒮¹ is given by a distribution on ?¹×𝒮¹, and we explore a distribution on this space by extending Mardia & Sutton's distribution on ?²×𝒮¹. In this distribution, the expected edgel direction varies linearly in the shape coordinates. This is found to be a useful model for the microfossil data.     

Multistate models in health insurance

We illustrate how multistate Markov and semi-Markov models can be used for the actuarial modeling of health insurance policies, focusing on health insurances that are pursued on a similar technical basis to that of life insurance. In the first part, we give an overview of the basic modeling frameworks that are commonly used and explain the calculation of prospective reserves and net premiums. In the second part, we discuss the biometric insurance risk, focusing on the calculation of implicit safety margins. We present new results on implicit margins in the semi-Markov model and on biometric estimation risk in the Markov model, and we explain why there is a need for future research concerning the systematic biometric risk.     

Smoothed nonparametric estimation in window censored semi-Markov processes

Consider a process that jumps among a finite set of states, with random times spent in between. In semi-Markov processes transitions follow a Markov chain and the sojourn distributions depend only on the connecting states. Suppose that the process started far in the past, achieving stationary. We consider non-parametric estimation by modelling the log-hazard of the sojourn times through linear splines; and we obtain maximum penalized likelihood estimators when data consist of several i.i.d. windows. We prove consistency using Grenander's method of sieves.     

Asymptotic theory for the Cox semi-Markov illness-death model

Irreversible illness-death models are used to model disease processes and in cancer studies to model disease recovery. In most applications, a Markov model is assumed for the multistate model. When there are covariates, a Cox (1972, J Roy Stat Soc Ser B 34:187–220) model is used to model the effect of covariates on each transition intensity. Andersen et al. (2000, Stat Med 19:587–599) proposed a Cox semi-Markov model for this problem. In this paper, we study the large sample theory for that model and provide the asymptotic variances of various probabilities of interest. A Monte Carlo study is conducted to investigate the robustness and efficiency of Markov/Semi-Markov estimators. A real data example from the PROVA (1991, Hepatology 14:1016–1024) trial is used to illustrate the theory.     

On a multidimensional general bootstrap for empirical estimator of continuous-time semi-Markov kernels with applications

The present paper introduces a general notion and presents results of bootstrapped empirical estimators of the semi-Markov kernels and of the conditional transition distributions for semi-Markov processes with countable state space, constructed by exchangeably weighting the sample. Our proposal provides a unification of bootstrap methods in the semi-Markov setting including, in particular, Efron's bootstrap. Asymptotic properties of these generalised bootstrapped empirical distributions are obtained, under mild conditions by a martingale approach. We also obtain some new results on the weak convergence of the empirical semi-Markov processes. We apply these general results in several statistical problems such as the construction of confidence bands and the goodness-of-fit tests where the limiting distributions are derived under the null hypothesis. Finally, we introduce the quantile estimators and their bootstrapped versions in the semi-Markov framework and we establish their limiting laws by using the functional delta methods. Our theoretical results and numerical examples by simulations demonstrate the merits of the proposed techniques.     

Bayesian model selection and parameter estimation for possibly asymmetric and non-stationary time series using a reversible jump Markov chain Monte Carlo approach

A Markov chain Monte Carlo (MCMC) approach, called a reversible jump MCMC, is employed in model selection and parameter estimation for possibly non-stationary and non-linear time series data. The non-linear structure is modelled by the asymmetric momentum threshold autoregressive process (MTAR) of Enders & Granger (1998) or by the asymmetric self-exciting threshold autoregressive process (SETAR) of Tong (1990). The non-stationary and non-linear feature is represented by the MTAR (or SETAR) model in which one ( 𝜌 1 ) of the AR coefficients is greater than one, and the other ( 𝜌 2 ) is smaller than one. The other non-stationary and linear, stationary and nonlinear, and stationary and linear features, represented respectively by ( 𝜌 1 = 𝜌 2 = 1 ), ( 𝜌 1 p 𝜌 2 < 1 ) and ( 𝜌 1 = 𝜌 2 < 1 ), are also considered as possible models. The reversible jump MCMC provides estimates of posterior probabilities for these four different models as well as estimates of the AR coefficients 𝜌 1 and 𝜌 2 . The proposed method is illustrated by analysing six series of US interest rates in terms of model selection, parameter estimation, and forecasting.     

The Augmented Semi-Markov System in Continuous Time

This article proposes a continuous time semi-Markov hierarchical manpower planning model that incorporates the need of the employees to attend seminars, so as to enhance their prospects, as well as the organizations' intention to avoid situations concerning unavailability in skilled personnel when needed. At large, we study a hierarchical system where the workforce demand at each time period can be met via internal mobility and two streams of recruitment; one from the outside environment and another from a supplementary auxiliary system. For the suggested model, namely the Continuous Time Augmented Semi-Markov System, we examine initially its dynamic behavior by deriving the equations reflecting the expected number of persons in each grade. In the sequel, we probe its limiting population structure and it is found that under a set of conditions this structure exists and is specified. Finally, we present a real case which demonstrates the practical motivation of the subject under study.     

A Fixed Point Approach to the Classification of Markov Chains with a Tree Structure

In this paper, we study the classification problem of discrete time and continuous time Markov processes with a tree structure. We first show some useful properties associated with the fixed points of a nondecreasing mapping. Mainly we find the conditions for a fixed point to be the minimal fixed point by using fixed point theory and degree theory. We then use these results to identify conditions for Markov chains of M/G/1 type or GI/M/1 type with a tree structure to be positive recurrent, null recurrent, or transient. The results are generalized to Markov chains of matrix M/G/1 type with a tree structure. For all these cases, a relationship between a certain fixed point, the matrix of partial differentiation (Jacobian) associated with the fixed point, and the classification of the Markov chain with a tree structure is established. More specifically, we show that the Perron-Frobenius eigenvalue of the matrix of partial differentiation associated with a certain fixed point provides information for a complete classification of the Markov chains of interest.     

Cumulative ratio information based on general cumulative entropy

In this paper, we suggest an extension of the cumulative residual entropy (CRE) and call it generalized cumulative entropy. The proposed entropy not only retains attributes of the existing uncertainty measures but also possesses the absolute homogeneous property with unbounded support, which the CRE does not have. We demonstrate its mathematical properties including the entropy of order statistics and the principle of maximum general cumulative entropy. We also introduce the cumulative ratio information as a measure of discrepancy between two distributions and examine its application to a goodness-of-fit test of the logistic distribution. Simulation study shows that the test statistics based on the cumulative ratio information have comparable statistical power with competing test statistics.     

On the limit distribution of a second-order semi-Markov chain in state and duration

In this paper, we derive conditions under which second-order semi-Markov chains in state and duration admits a limit distribution.     

Transient analysis of a finite source discrete-time queueing system using homogeneous Markov system with state size capacities (HMS/c)

Abstract

In this article, a finite source discrete-time queueing system is modeled as a discrete-time homogeneous Markov system with finite state size capacities (HMS/c) and transition priorities. This Markov system is comprised of three states. The first state of the HMS/c corresponds to the source and the second one to the state with the servers. The second state has a finite capacity which corresponds to the number of servers. The members of the system which can not enter the second state, due to its finite capacity, enter the third state which represents the system's queue. In order to examine the variability of the state sizes recursive formulae for their factorial and mixed factorial moments are derived in matrix form. As a consequence the probability mass function of each state size can be evaluated. Also the expected time in queue is computed by means of the interval transition probabilities. The theoretical results are illustrated by a numerical example.     

Semi-Markov Disability Insurance Models

In this article, we present a stochastic model for disability insurance contracts. The model is based on a discrete time non homogeneous semi-Markov process (DTNHSMP) to which the backward recurrence time process is introduced. This permits a more exhaustive study of disability evolution and a more efficient approach to the duration problem. The use of semi-Markov reward processes facilitates the possibility of deriving equations of the prospective and retrospective mathematical reserves. The model is applied to a sample of contracts drawn at random from a mutual insurance company.     

Bounding convergence rates for Markov chains: An example of the use of computer algebra

Kolassa and Tanner (J. Am. Stat. Assoc. (1994) 89, 697–702) present the Gibbs-Skovgaard algorithm for approximate conditional inference. Kolassa (Ann Statist. (1999), 27, 129–142) gives conditions under which their Markov chain is known to converge. This paper calculates explicity bounds on convergence rates in terms calculable directly from chain transition operators. These results are useful in cases like those considered by Kolassa (1999).     

Towards optimal scaling of metropolis-coupled Markov chain Monte Carlo

We consider optimal temperature spacings for Metropolis-coupled Markov chain Monte Carlo (MCMCMC) and Simulated Tempering algorithms. We prove that, under certain conditions, it is optimal (in terms of maximising the expected squared jumping distance) to space the temperatures so that the proportion of temperature swaps which are accepted is approximately 0.234. This generalises related work by physicists, and is consistent with previous work about optimal scaling of random-walk Metropolis algorithms.     

Large Deviations for Empirical Estimators of the Stationary Distribution of a Semi-Markov Process with Finite State Space

We prove the large deviation principle for empirical estimators of stationary distributions of semi-Markov processes with finite state space, irreducible embedded Markov chain, and finite mean sojourn time in each state. We consider on/off Gamma sojourn processes as an illustrative example, and, in particular, continuous time Markov chains with two states. In the second case, we compare the rate function in this article with the known rate function concerning another family of empirical estimators of the stationary distribution.