Forecasting with non-homogeneous hidden Markov models

We present a Bayesian forecasting methodology of discrete-time finite state-space hidden Markov models with non-constant transition matrix that depends on a set of exogenous covariates. We describe an MCMC reversible jump algorithm for predictive inference, allowing for model uncertainty regarding the set of covariates that affect the transition matrix. We apply our models to interest rates and we show that our general model formulation improves the predictive ability of standard homogeneous hidden Markov models.     

BAYESIAN DETECTION AND ANALYSIS OF CHANGING TRANSITION MATRICES OF STATIONARY MARKOV CHAINS

This paper considers the detection of abrupt changes in the transition matrix of a Markov chain from a Bayesian viewpoint. It derives Bayes factors and posterior probabilities for unknown numbers of change‐points, as well as the positions of the change‐points, assuming non‐informative but proper priors on the parameters and fixed upper bound. The Markov chain Monte Carlo approach proposed by Chib in 1998 for estimating multiple change‐points models is adapted for the Markov chain model. It is especially useful when there are many possible change‐points. The method can be applied in a wide variety of disciplines and is particularly relevant in the social and behavioural sciences, for analysing the effects of events on the attitudes of people.     

A comparison of estimators of a common correlation coefficient

It is often of interest to test the hypothesis that all off-diagonal elements of the correlation matrix of a multivariate normal distribution are equal. If the hypothesis of equal correlation can be accepted, it then may be of interest to estimate the common correlation coefficient. In this paper, four estimators of the common correlation are compared in terms of bias, variance, mean squared error, adequacy of the normal approximation, and ease of calculation. The average sample correlation is seen to be comparable to the other estimators and is recommended here since it is the easiest to calculate. The estimators are compared using simulation.     

Application of a non-homogeneous Markov chain with seasonal transition probabilities to ozone data

In this work, we assume that the sequence recording whether or not an ozone exceedance of an environmental threshold has occurred in a given day is ruled by a non-homogeneous Markov chain of order one. In order to account for the possible presence of cycles in the empirical transition probabilities, a parametric form incorporating seasonal components is considered. Results show that even though some covariates (namely, relative humidity and temperature) are not included explicitly in the model, their influence is captured in the behavior of the transition probabilities. Parameters are estimated using the Bayesian point of view via Markov chain Monte Carlo algorithms. The model is applied to ozone data obtained from the monitoring network of Mexico City, Mexico. An analysis of how the methodology could be used as an aid in the decision-making is also given.     

Central Limit Theorem approximations for the number of runs in Markov-dependent binary sequences

We consider Markov-dependent binary sequences and study various types of success runs (overlapping, non-overlapping, exact, etc.) by examining additive functionals based on state visits and transitions in an appropriate Markov chain. We establish a multivariate Central Limit Theorem for the number of these types of runs and obtain its covariance matrix by means of the recurrent potential matrix of the Markov chain. Explicit expressions for the covariance matrix are given in the Bernoulli and a simple Markov-dependent case by expressing the recurrent potential matrix in terms of the stationary distribution and the mean transition times in the chain. We also obtain a multivariate Central Limit Theorem for the joint number of non-overlapping runs of various sizes and give its covariance matrix in explicit form for Markov dependent trials.     

MCMC and the Fibonacci distribution

A method to create a Markov transition matrix for Markov chain Monte Carlo studies is presented and applied to the Fibonacci probability distribution.     

ASYMPTOTIC PROPERTIES FOR MAXIMUM LIKELIHOOD ESTIMATORS FOR RELIABILITY AND FAILURE RATES OF MARKOV CHAINS

ABSTRACT

In this paper, we shall study a homogeneous ergodic, finite state, Markov chain with unknown transition probability matrix. Starting from the well known maximum likelihood estimator of transition probability matrix, we define estimators of reliability and its measurements. Our aim is to show that these estimators are uniformly strongly consistent and converge in distribution to normal random variables. The construction of the confidence intervals for availability, reliability, and failure rates are also given. Finally we shall give a numerical example for illustration and comparing our results with the usual empirical estimator results.     

Stochastic monotonicity and comparability of Markov chains with block-monotone transition matrices and their applications to queueing systems

ABSTRACT

Motivated by various applications in queueing theory, this article is devoted to the stochastic monotonicity and comparability of Markov chains with block-monotone transition matrices. First, we introduce the notion of block-increasing convex order for probability vectors, and characterize the block-monotone matrices in the sense of the block-increasing order and block-increasing convex order. Second, we characterize the Markov chain with general transition matrix by martingale and provide a stochastic comparison of two block-monotone Markov chains under the two block-monotone orders. Third, the stochastic comparison results for the Markov chains corresponding to the discrete-time GI/G/1 queue with different service distributions under the two block-monotone orders are given, and the lower bound and upper bound of the Markov chain corresponding to the discrete-time GI/G/1 queue in the sense of the block-increasing convex order are found.     

Variance estimation in the central limit theorem for Markov chains

This article concerns the variance estimation in the central limit theorem for finite recurrent Markov chains. The associated variance is calculated in terms of the transition matrix of the Markov chain. We prove the equivalence of different matrix forms representing this variance. The maximum likelihood estimator for this variance is constructed and it is proved that it is strongly consistent and asymptotically normal. The main part of our analysis consists in presenting closed matrix forms for this new variance. Additionally, we prove the asymptotic equivalence between the empirical and the maximum likelihood estimation (MLE) for the stationary distribution.     

Applying a Markov chain for the stock pricing of a novel forecasting model

In this article, a stock-forecasting model is developed to analyze a company's stock price variation related to the Taiwanese company HTC. The main difference to previous articles is that this study uses the data of the HTC in recent ten years to build a Markov transition matrix. Instead of trying to predict the stock price variation through the traditional approach to the HTC stock problem, we integrate two types of Markov chain that are used in different ways. One is a regular Markov chain, and the other is an absorbing Markov chain. Through a regular Markov chain, we can obtain important information such as what happens in the long run or whether the distribution of the states tends to stabilize over time in an efficient way. Next, we used an artificial variable technique to create an absorbing Markov chain. Thus, we used an absorbing Markov chain to provide information about the period between the increases before arriving at the decreasing state of the HTC stock. We provide investors with information on how long the HTC stock will keep increasing before its price begins to fall, which is extremely important information to them.     

A stationarity test on Markov chain models based on marginal distribution

A stationarity test on Markov chain models is proposed in this paper. Most of the previous test procedures for the Markov chain models have been done based on the conditional probabilities of a transition matrix. The likelihood ratio and Pearson type chi-square tests have been used for testing stationarity and order of Markov chains. This paper uses the efficient score test, an extension of the test developed by Tsiatis (1980) [18], for testing the stationarity of Markov chain models based on the marginal distribution as obtained by Azzalini (1994) [2]. For testing the suitability of the proposed method, a numerical example of real life data and simulation studies for comparison with an alternative test procedure are given.     

Development of Specific hypothesis tests for estimated markov chains

We develop and evaluate the validity and power of two specific tests for the transition probabilities in a Markov chain estimated from aggregate frequency data. The two null hypotheses considered are (1) constancy of the diagonal elements of the one-step transition probability matrix and (2) an arbitrarily chosen transition probability’s being equal to a specific value. The formation of tests uses a general framework for statistical inference on estimated Markov processes; we also indicate how this framework can be used to form tests for a variety of other hypotheses. The validity and power performance of the two tests formed in this paper are examined in factorially designed Monte Carlo experiments. The results indicate that the proposed tests lead to type I error probabilities which are close to the desired levels and to high power against even small deviations from the null hypotheses considered.     

Do Leading Indicators Lead Peaks More Than Troughs?

We develop a novel Markov switching vector autoregressive model to investigate the possibility that leading indicators have different lead times at business cycle peaks and at troughs. In this model, coincident and leading indicators share a common Markov state process, but their cycles are nonsynchronous, with the nonsynchronicity varying across regimes. An application shows that on average the Conference Board’s Composite Leading Index leads the Composite Coincident Index by nearly 1 year at peaks but by only 1 quarter at troughs. Allowing for asymmetric lead times yields improved real-time dating and forecasting of business cycle turning points.     

Likelihood Ratio Testing for Hidden Markov Models Under Non-standard Conditions

Abstract.  In practical applications, when testing parametric restrictions for hidden Markov models (HMMs), one frequently encounters non-standard situations such as testing for zero entries in the transition matrix, one-sided tests for the parameters of the transition matrix or for the components of the stationary distribution of the underlying Markov chain, or testing boundary restrictions on the parameters of the state-dependent distributions. In this paper, we briefly discuss how the relevant asymptotic distribution theory for the likelihood ratio test (LRT) when the true parameter is on the boundary extends from the independent and identically distributed situation to HMMs. Then we concentrate on discussing a number of relevant examples. The finite-sample performance of the LRT in such situations is investigated in a simulation study. An application to series of epileptic seizure counts concludes the paper.     

Statistical inference for discretely observed Markov jump processes

Summary.  Likelihood inference for discretely observed Markov jump processes with finite state space is investigated. The existence and uniqueness of the maximum likelihood estimator of the intensity matrix are investigated. This topic is closely related to the imbedding problem for Markov chains. It is demonstrated that the maximum likelihood estimator can be found either by the EM algorithm or by a Markov chain Monte Carlo procedure. When the maximum likelihood estimator does not exist, an estimator can be obtained by using a penalized likelihood function or by the Markov chain Monte Carlo procedure with a suitable prior. The methodology and its implementation are illustrated by examples and simulation studies.     

Regularized covariance matrix estimation under the common principal components model

The common principal components (CPC) model provides a way to model the population covariance matrices of several groups by assuming a common eigenvector structure. When appropriate, this model can provide covariance matrix estimators of which the elements have smaller standard errors than when using either the pooled covariance matrix or the per group unbiased sample covariance matrix estimators. In this article, a regularized CPC estimator under the assumption of a common (or partially common) eigenvector structure in the populations is proposed. After estimation of the common eigenvectors using the Flury–Gautschi (or other) algorithm, the off-diagonal elements of the nearly diagonalized covariance matrices are shrunk towards zero and multiplied with the orthogonal common eigenvector matrix to obtain the regularized CPC covariance matrix estimates. The optimal shrinkage intensity per group can be estimated using cross-validation. The efficiency of these estimators compared to the pooled and unbiased estimators is investigated in a Monte Carlo simulation study, and the regularized CPC estimator is applied to a real dataset to demonstrate the utility of the method.     

Stability of Approximations of Average Run Length of Risk-Adjusted CUSUM Schemes Using the Markov Approach: Comparing Two Methods of Calculating Transition Probabilities

Risk-adjusted CUSUM schemes are designed to monitor the number of adverse outcomes following a medical procedure. An approximation of the average run length (ARL), which is the usual performance measure for a risk-adjusted CUSUM, may be found using its Markov property. We compare two methods of computing transition probability matrices where the risk model classifies patient populations into discrete, finite levels of risk. For the first method, a process of scaling and rounding off concentrates probability in the center of the Markov states, which are non overlapping sub-intervals of the CUSUM decision interval, and, for the second, a smoothing process spreads probability uniformly across the Markov states. Examples of risk-adjusted CUSUM schemes are used to show, if rounding is used to calculate transition probabilities, the values of ARLs estimated using the Markov property vary erratically as the number of Markov states vary and, on occasion, fail to converge for mesh sizes up to 3,000. On the other hand, if smoothing is used, the approximate ARL values remain stable as the number of Markov states vary. The smoothing technique gave good estimates of the ARL where there were less than 1,000 Markov states.     

A New Approach to Volatility Modeling: The Factorial Hidden Markov Volatility Model

A new process—the factorial hidden Markov volatility (FHMV) model—is proposed to model financial returns or realized variances. Its dynamics are driven by a latent volatility process specified as a product of three components: a Markov chain controlling volatility persistence, an independent discrete process capable of generating jumps in the volatility, and a predictable (data-driven) process capturing the leverage effect. An economic interpretation is attached to each one of these components. Moreover, the Markov chain and jump components allow volatility to switch abruptly between thousands of states, and the transition matrix of the model is structured to generate a high degree of volatility persistence. An empirical study on six financial time series shows that the FHMV process compares favorably to state-of-the-art volatility models in terms of in-sample fit and out-of-sample forecasting performance over time horizons ranging from 1 to 100 days. Supplementary materials for this article are available online.     

Semiparametric hidden Markov model with non-parametric regression

The hidden Markov model regression (HMMR) has been popularly used in many fields such as gene expression and activity recognition. However, the traditional HMMR requires the strong linearity assumption for the emission model. In this article, we propose a hidden Markov model with non-parametric regression (HMM-NR), where the mean and variance of emission model are unknown smooth functions. The new semiparametric model might greatly reduce the modeling bias and thus enhance the applicability of the traditional hidden Markov model regression. We propose an estimation procedure for the transition probability matrix and the non-parametric mean and variance functions by combining the ideas of the EM algorithm and the kernel regression. Simulation studies and a real data set application are used to demonstrate the effectiveness of the new estimation procedure.