On the natural restrictions in the singular Gauss–Markov model

A Gauss–Markov model is said to be singular if the covariance matrix of the observable random vector in the model is singular. In such a case, there exist some natural restrictions associated with the observable random vector and the unknown parameter vector in the model. In this paper, we derive through the matrix rank method a necessary and sufficient condition for a vector of parametric functions to be estimable, and necessary and sufficient conditions for a linear estimator to be unbiased in the singular Gauss–Markov model. In addition, we give some necessary and sufficient conditions for the ordinary least-square estimator (OLSE) and the best linear unbiased estimator (BLUE) under the model to satisfy the natural restrictions.      

A New Policy Evaluation Algorithm for Markov Decision Processes with Quasi Birth-Death Structure

ABSTRACT

This paper describes a new algorithm for policy evaluation for Markov decision processes (MDP) that possess a quasi birth-death structure. The proposed algorithm is based on matrix analytic methods which use probabilistic concepts associated with restricting the underlying Markov process to certain state subsets. A telecommunications application example shows that the method offers significant computational reduction compared to a standard MDP policy evaluation approach.     

Mode Jumping Proposals in MCMC

Markov chain Monte Carlo algorithms generate samples from a target distribution by simulating a Markov chain. Large flexibility exists in specification of transition matrix of the chain. In practice, however, most algorithms used only allow small changes in the state vector in each iteration. This choice typically causes problems for multi-modal distributions as moves between modes become rare and, in turn, results in slow convergence to the target distribution. In this paper we consider continuous distributions on R ⁿ and specify how optimization for local maxima of the target distribution can be incorporated in the specification of the Markov chain. Thereby, we obtain a chain with frequent jumps between modes. We demonstrate the effectiveness of the approach in three examples. The first considers a simple mixture of bivariate normal distributions, whereas the two last examples consider sampling from posterior distributions based on previously analysed data sets.     

Nonparametric Bayesian analysis for multi-site hidden Markov model

The hidden Markov model (HMM) provides an attractive framework for modeling long-term persistence in a variety of applications including pattern recognition. Unlike typical mixture models, hidden Markov states can represent the heterogeneity in data and it can be extended to a multivariate case using a hierarchical Bayesian approach. This article provides a nonparametric Bayesian modeling approach to the multi-site HMM by considering stick-breaking priors for each row of an infinite state transition matrix. This extension has many advantages over a parametric HMM. For example, it can provide more flexible information for identifying the structure of the HMM than parametric HMM analysis, such as the number of states in HMM. We exploit a simulation example and a real dataset to evaluate the proposed approach.     

Singular Gaussian graphical models: Structure learning

ABSTRACT

The goal of this article is to introduce singular Gaussian graphical models and their conditional independence properties. In fact, we extend the concept of Gaussian Markov Random Field to the case of a multivariate normally distributed vector with a singular covariance matrix. We construct, then, the associated graph’s structure from the covariance matrix’s pseudo-inverse on the basis of a characterization of the pairwise conditional independence. The proposed approach can also be used when the covariance matrix is ill-conditioned, through projecting data on a smaller subspace. In this case, our method ensures numerical stability and consistency of the constructed graph and significantly reduces the inference problem’s complexity. These aspects are illustrated using numerical experiments.     

Central Limit Theorem approximations for the number of runs in Markov-dependent binary sequences

We consider Markov-dependent binary sequences and study various types of success runs (overlapping, non-overlapping, exact, etc.) by examining additive functionals based on state visits and transitions in an appropriate Markov chain. We establish a multivariate Central Limit Theorem for the number of these types of runs and obtain its covariance matrix by means of the recurrent potential matrix of the Markov chain. Explicit expressions for the covariance matrix are given in the Bernoulli and a simple Markov-dependent case by expressing the recurrent potential matrix in terms of the stationary distribution and the mean transition times in the chain. We also obtain a multivariate Central Limit Theorem for the joint number of non-overlapping runs of various sizes and give its covariance matrix in explicit form for Markov dependent trials.     

A stationarity test on Markov chain models based on marginal distribution

A stationarity test on Markov chain models is proposed in this paper. Most of the previous test procedures for the Markov chain models have been done based on the conditional probabilities of a transition matrix. The likelihood ratio and Pearson type chi-square tests have been used for testing stationarity and order of Markov chains. This paper uses the efficient score test, an extension of the test developed by Tsiatis (1980) [18], for testing the stationarity of Markov chain models based on the marginal distribution as obtained by Azzalini (1994) [2]. For testing the suitability of the proposed method, a numerical example of real life data and simulation studies for comparison with an alternative test procedure are given.     

Transient Analysis of the M/M/k/N/N Queue using a Continuous Time Homogeneous Markov System with Finite State Size Capacity

In this article, the M/M/k/N/N queue is modeled as a continuous-time homogeneous Markov system with finite state size capacity (HMS/c_s). In order to examine the behavior of the queue a continuous-time homogeneous Markov system (HMS) constituted of two states is used. The first state of this HMS corresponds to the source and the second one to the state with the servers. The second state has a finite capacity which corresponds to the number of servers. The members of the system which can not enter the second state, due to its finite capacity, enter the buffer state which represents the system's queue. In order to examine the variability of the state sizes formulae for their factorial and mixed factorial moments are derived in matrix form. As a consequence, the pmf of each state size can be evaluated for any t ∈ ?⁺. The theoretical results are illustrated by a numerical example.     

Phase II monitoring of binary profiles in the presence of within-profile autocorrelation based on Markov Model

This paper introduces a Markov model in Phase II profile monitoring with autocorrelated binary response variable. In the proposed approach, a logistic regression model is extended to describe the within-profile autocorrelation. The likelihood function is constructed and then a particle swarm optimization algorithm (PSO) is tuned and utilized to estimate the model parameters. Furthermore, two control charts are extended in which the covariance matrix is derived based on the Fisher information matrix. Simulation studies are conducted to evaluate the detecting capability of the proposed control charts. A numerical example is also given to illustrate the application of the proposed method.     

Estimating models based on Markov jump processes given fragmented observation series

We consider the problem of estimating the rate matrix governing a finite-state Markov jump process given a number of fragmented time series. We propose to concatenate the observed series and to employ the emerging non-Markov process for estimation. We describe the bias arising if standard methods for Markov processes are used for the concatenated process, and provide a post-processing method to correct for this bias. This method applies to discrete-time Markov chains and to more general models based on Markov jump processes where the underlying state process is not observed directly. This is demonstrated in detail for a Markov switching model. We provide applications to simulated time series and to financial market data, where estimators resulting from maximum likelihood methods and Markov chain Monte Carlo sampling are improved using the presented correction.     

Modelling of unexpected shift in SPC

Optimal statistical process control (SPC) requires models of both in-control and out-of-control process states. Whereas a normal distribution is the generally accepted model for the in-control state, there is a doubt as to the existence of reliable models for out-of-control cases. Various process models, available in the literature, for discrete manufacturing systems (parts industry) can be treated as bounded discrete-space Markov chains, completely characterized by the original in-control state and a transition matrix for shifts to an out-of-control state. The present work extends these models by using a continuous-state Markov chain, incorporating non-random corrective actions. These actions are to be realized according to the SPC technique and should substantially affect the model. The developed stochastic model yields a Laplace distribution of a process mean. An alternative approach, based on the Information theory, also results in a Laplace distribution. Real-data tests confirm the applicability of a Laplace distribution for the parts industry and show that the distribution parameter is mainly controlled by the SPC sample size.     

Matrix-analytic Models and their Analysis

We survey phase-type distributions and Markovian point processes, aspects of how to use such models in applied probability calculations and how to fit them to observed data. A phase-type distribution is defined as the time to absorption in a finite continuous time Markov process with one absorbing state. This class of distributions is dense and contains many standard examples like all combinations of exponential in series/parallel. A Markovian point process is governed by a finite continuous time Markov process (typically ergodic), such that points are generated at a Poisson intensity depending on the underlying state and at transitions; a main special case is a Markov-modulated Poisson process. In both cases, the analytic formulas typically contain matrix-exponentials, and the matrix formalism carried over when the models are used in applied probability calculations as in problems in renewal theory, random walks and queueing. The statistical analysis is typically based upon the EM algorithm, viewing the whole sample path of the background Markov process as the latent variable.     

Statistical inference for discretely observed Markov jump processes

Summary.  Likelihood inference for discretely observed Markov jump processes with finite state space is investigated. The existence and uniqueness of the maximum likelihood estimator of the intensity matrix are investigated. This topic is closely related to the imbedding problem for Markov chains. It is demonstrated that the maximum likelihood estimator can be found either by the EM algorithm or by a Markov chain Monte Carlo procedure. When the maximum likelihood estimator does not exist, an estimator can be obtained by using a penalized likelihood function or by the Markov chain Monte Carlo procedure with a suitable prior. The methodology and its implementation are illustrated by examples and simulation studies.     

Bayesian goodness-of-fit test for censored data

We propose a Bayesian computation and inference method for the Pearson-type chi-squared goodness-of-fit test with right-censored survival data. Our test statistic is derived from the classical Pearson chi-squared test using the differences between the observed and expected counts in the partitioned bins. In the Bayesian paradigm, we generate posterior samples of the model parameter using the Markov chain Monte Carlo procedure. By replacing the maximum likelihood estimator in the quadratic form with a random observation from the posterior distribution of the model parameter, we can easily construct a chi-squared test statistic. The degrees of freedom of the test equal the number of bins and thus is independent of the dimensionality of the underlying parameter vector. The test statistic recovers the conventional Pearson-type chi-squared structure. Moreover, the proposed algorithm circumvents the burden of evaluating the Fisher information matrix, its inverse and the rank of the variance–covariance matrix. We examine the proposed model diagnostic method using simulation studies and illustrate it with a real data set from a prostate cancer study.     

Adaptive Kalman Filtering with Multivariate Generalized Laplace System Noise

An adaptive Kalman filter is proposed to estimate the states of a system where the system noise is assumed to be a multivariate generalized Laplace random vector. In the presence of outliers in the system noise, it is shown that improved state estimates can be obtained by using an adaptive factor to estimate the dispersion matrix of the system noise term. For the implementation of the filter, an algorithm which includes both single and multiple adaptive factors is proposed. A Monte-Carlo investigation is also carried out to access the performance of the proposed filters in comparison with other robust filters. The results show that, in the sense of minimum mean squared state error, the proposed filter is superior to other filters when the magnitude of a system change is moderate or large.     

On the estimation of the mean vector of a multivariate normal distribution under symmetry

The problem of estimation of the mean vector of a multivariate normal distribution with unknown covariance matrix, under uncertain prior information (UPI) that the component mean vectors are equal, is considered. The shrinkage preliminary test maximum likelihood estimator (SPTMLE) for the parameter vector is proposed. The risk and covariance matrix of the proposed estimato are derived and parameter range in which SPTMLE dominates the usual preliminary test maximum likelihood estimator (PTMLE) is investigated. It is shown that the proposed estimator provides a wider range than the usual premilinary test estimator in which it dominates the classical estimator. Further, the SPTMLE has more appropriate size for the preliminary test than the PTMLE.     

Transient analysis of a finite source discrete-time queueing system using homogeneous Markov system with state size capacities (HMS/c)

Abstract

In this article, a finite source discrete-time queueing system is modeled as a discrete-time homogeneous Markov system with finite state size capacities (HMS/c) and transition priorities. This Markov system is comprised of three states. The first state of the HMS/c corresponds to the source and the second one to the state with the servers. The second state has a finite capacity which corresponds to the number of servers. The members of the system which can not enter the second state, due to its finite capacity, enter the third state which represents the system's queue. In order to examine the variability of the state sizes recursive formulae for their factorial and mixed factorial moments are derived in matrix form. As a consequence the probability mass function of each state size can be evaluated. Also the expected time in queue is computed by means of the interval transition probabilities. The theoretical results are illustrated by a numerical example.     

On directional Metropolis–Hastings algorithms

New Metropolis–Hastings algorithms using directional updates are introduced in this paper. Each iteration of a directional Metropolis–Hastings algorithm consists of three steps (i) generate a line by sampling an auxiliary variable, (ii) propose a new state along the line, and (iii) accept/reject according to the Metropolis–Hastings acceptance probability. We consider two classes of directional updates. The first uses a point in ⁿ as auxiliary variable, the second an auxiliary direction vector. The proposed algorithms generalize previous directional updating schemes since we allow the distribution of the auxiliary variable to depend on properties of the target at the current state. By letting the proposal distribution along the line depend on the density of the auxiliary variable, we identify proposal mechanisms that give unit acceptance rate. When we use direction vector as auxiliary variable, we get the advantageous effect of large moves in the Markov chain and hence the autocorrelation length of the samples is small. We apply the directional Metropolis–Hastings algorithms to a Gaussian example, a mixture of Gaussian densities, and a Bayesian model for seismic data.     

A New Approach to Volatility Modeling: The Factorial Hidden Markov Volatility Model

A new process—the factorial hidden Markov volatility (FHMV) model—is proposed to model financial returns or realized variances. Its dynamics are driven by a latent volatility process specified as a product of three components: a Markov chain controlling volatility persistence, an independent discrete process capable of generating jumps in the volatility, and a predictable (data-driven) process capturing the leverage effect. An economic interpretation is attached to each one of these components. Moreover, the Markov chain and jump components allow volatility to switch abruptly between thousands of states, and the transition matrix of the model is structured to generate a high degree of volatility persistence. An empirical study on six financial time series shows that the FHMV process compares favorably to state-of-the-art volatility models in terms of in-sample fit and out-of-sample forecasting performance over time horizons ranging from 1 to 100 days. Supplementary materials for this article are available online.     

Testing transition probability matrix of a multi-state model with censored data

In this paper, we develop procedures to test hypotheses concerning transition probability matrices arising from certain nonhomogeneous Markov processes. It is assumed that the data consist of sample paths, some of which are observed until a certain terminal state, and the other paths are censored. Problems of this type arise in the context of multi-state models relevant to Health Related Quality of Life (HRQoL) and Competing Risks. The test statistic is based on the estimator for the associated intensity matrix. We show that the asymptotic null distribution of the proposed statistic is Gaussian, and demonstrate how the procedure can be adopted for HRQoL studies and competing risks model using real data sets. Finally, we establish that the test statistic for the HRQoL has greatest local asymptotic power against a sequence of proportional hazards alternatives converging to the null hypothesis.