Dynamic latent trait models with mixed hidden Markov structure for mixed longitudinal outcomes

We propose a general Bayesian joint modeling approach to model mixed longitudinal outcomes from the exponential family for taking into account any differential misclassification that may exist among categorical outcomes. Under this framework, outcomes observed without measurement error are related to latent trait variables through generalized linear mixed effect models. The misclassified outcomes are related to the latent class variables, which represent unobserved real states, using mixed hidden Markov models (MHMMs). In addition to enabling the estimation of parameters in prevalence, transition and misclassification probabilities, MHMMs capture cluster level heterogeneity. A transition modeling structure allows the latent trait and latent class variables to depend on observed predictors at the same time period and also on latent trait and latent class variables at previous time periods for each individual. Simulation studies are conducted to make comparisons with traditional models in order to illustrate the gains from the proposed approach. The new approach is applied to data from the Southern California Children Health Study to jointly model questionnaire-based asthma state and multiple lung function measurements in order to gain better insight about the underlying biological mechanism that governs the inter-relationship between asthma state and lung function development.     

Consistent estimation for discretely observed Markov jump processes with an absorbing state

For a continuous-time Markov process, commonly, only discrete-time observations are available. We analyze multiple observations of a homogeneous Markov jump process with an absorbing state. We establish consistency of the maximum likelihood estimator, as the number of Markov processes increases. To accomplish uniform convergence in the continuous mapping theorem, we use the continuity of the transition probability in the parameters, the compactness of the parameter space and the boundedness of probabilities. We allow for a stochastic time-grid of observation points with different intensities for each observation process. Furthermore, we account for right censoring. The estimate is obtained via the EM algorithm with an E-step given in closed form. In our empirical application of credit rating histories, we fit the model of Weißbach and Mollenhauer (J Korean Stat Soc 40:469–485, 2011) and find marked differences, compared to the continuous-time analysis.     

Empirical Performance and Asset Pricing in Hidden Markov Models

Abstract

To improve the empirical performance of the Black-Scholes model, many alternative models have been proposed to address leptokurtic feature, volatility smile, and volatility clustering effects of the asset return distributions. However, analytical tractability remains a problem for most alternative models. In this article, we study a class of hidden Markov models including Markov switching models and stochastic volatility models, that can incorporate leptokurtic feature, volatility clustering effects, as well as provide analytical solutions to option pricing. We show that these models can generate long memory phenomena when the transition probabilities depend on the time scale. We also provide an explicit analytic formula for the arbitrage-free price of the European options under these models. The issues of statistical estimation and errors in option pricing are also discussed in the Markov switching models.     

Markov Poisson regression models for discrete time series. Part 1: Methodology

This paper proposes and investigates a class of Markov Poisson regression models in which Poisson rate functions of covariates are conditional on unobserved states which follow a finite-state Markov chain. Features of the proposed model, estimation, inference, bootstrap confidence intervals, model selection and other implementation issues are discussed. Monte Carlo studies suggest that the proposed estimation method is accurate and reliable for single- and multiple-subject time series data; the choice of starting probabilities for the Markov process has little eff ect on the parameter estimates; and penalized likelihood criteria are reliable for determining the number of states. Part 2 provides applications of the proposed model.     

Nonparametric estimation of transition probabilities in a non-Markov illness–death model

In this paper we consider nonparametric estimation of transition probabilities for multi-state models. Specifically, we focus on the illness-death or disability model. The main novelty of the proposed estimators is that they do not rely on the Markov assumption, typically assumed to hold in a multi-state model. We investigate the asymptotic properties of the introduced estimators, such as their consistency and their convergence to a normal law. Simulations demonstrate that the new estimators may outperform Aalen–Johansen estimators (the classical nonparametric tool for estimating the transition probabilities) in non-Markov situation. An illustration through real data analysis is included.     

Discrete time, discrete state latent Markov modelling for assessing and predicting household acquisitions of financial products

Summary.  The paper demonstrates application of the latent Markov model for assessing developments by individuals through stages of a process. This approach is applied by using a database on ownership of 12 financial products and various demographic variables. The latent Markov model derives latent classes, representing household product portfolios, and shows the relationship between class membership and household demographics. The analysis provides insight into switching between the latent classes, reflecting developments of individual household product portfolios, and the effects of demographics on such switches. Based on this, we formulate equations to predict future acquisitions of financial products. The model accurately predicts which product a specific household unit acquires next, for most of the products.     

Estimation and tests for a longitudinal regression model based on the Markov chain

In this paper, the dependence of transition probabilities on covariates and a test procedure for covariate dependent Markov models are examined. The nonparametric test for the role of waiting time proposed by Jones and Crowley [M. Jones, J. Crowley, Nonparametric tests of the Markov model for survival data Biometrika 79 (3) (1992) 513–522] has been extended here to transitions and reverse transitions. The limitation of the Jones and Crowley method is that it does not take account of other covariates that might have association with the probabilities of transition. A simple test procedure is proposed that can be employed for testing: (i) the significance of association between covariates and transition probabilities, and (ii) the impact of waiting time on the transition probabilities. The procedure is illustrated using panel data on hospitalization of the elderly population in the USA from the Health and Retirement Survey (HRS).     

A New Model for Multivariate Markov Chains

We propose a new model for multivariate Markov chains of order one or higher on the basis of the mixture transition distribution (MTD) model. We call it the MTD‐Probit. The proposed model presents two attractive features: it is completely free of constraints, thereby facilitating the estimation procedure, and it is more precise at estimating the transition probabilities of a multivariate or higher‐order Markov chain than the standard MTD model.     

Stability of Approximations of Average Run Length of Risk-Adjusted CUSUM Schemes Using the Markov Approach: Comparing Two Methods of Calculating Transition Probabilities

Risk-adjusted CUSUM schemes are designed to monitor the number of adverse outcomes following a medical procedure. An approximation of the average run length (ARL), which is the usual performance measure for a risk-adjusted CUSUM, may be found using its Markov property. We compare two methods of computing transition probability matrices where the risk model classifies patient populations into discrete, finite levels of risk. For the first method, a process of scaling and rounding off concentrates probability in the center of the Markov states, which are non overlapping sub-intervals of the CUSUM decision interval, and, for the second, a smoothing process spreads probability uniformly across the Markov states. Examples of risk-adjusted CUSUM schemes are used to show, if rounding is used to calculate transition probabilities, the values of ARLs estimated using the Markov property vary erratically as the number of Markov states vary and, on occasion, fail to converge for mesh sizes up to 3,000. On the other hand, if smoothing is used, the approximate ARL values remain stable as the number of Markov states vary. The smoothing technique gave good estimates of the ARL where there were less than 1,000 Markov states.     

A competing risks approach for nonparametric estimation of transition probabilities in a non-Markov illness-death model

Competing risks model time to first event and type of first event. An example from hospital epidemiology is the incidence of hospital-acquired infection, which has to account for hospital discharge of non-infected patients as a competing risk. An illness-death model would allow to further study hospital outcomes of infected patients. Such a model typically relies on a Markov assumption. However, it is conceivable that the future course of an infected patient does not only depend on the time since hospital admission and current infection status but also on the time since infection. We demonstrate how a modified competing risks model can be used for nonparametric estimation of transition probabilities when the Markov assumption is violated.     

Maximum likelihood estimation of bivariate circular hidden Markov models from incomplete data

In this paper, we propose a hidden Markov model for the analysis of the time series of bivariate circular observations, by assuming that the data are sampled from bivariate circular densities, whose parameters are driven by the evolution of a latent Markov chain. The model segments the data by accounting for redundancies due to correlations along time and across variables. A computationally feasible expectation maximization (EM) algorithm is provided for the maximum likelihood estimation of the model from incomplete data, by treating the missing values and the states of the latent chain as two different sources of incomplete information. Importance-sampling methods facilitate the computation of bootstrap standard errors of the estimates. The methodology is illustrated on a bivariate time series of wind and wave directions and compared with popular segmentation models for bivariate circular data, which ignore correlations across variables and/or along time.     

Earthquakes occurrences estimation through a parametric semi-Markov approach

The estimation of earthquakes’ occurrences prediction in seismic areas is a challenging problem in seismology and earthquake engineering. Indeed, the prevention and the quantification of possible damage provoked by destructive earthquakes are directly linked to this kind of prevision. In our paper, we adopt a parametric semi-Markov approach. This model assumes that a sequence of earthquakes is seen as a Markov process and besides it permits to take into consideration the more realistic assumption of events’ dependence in space and time. The elapsed time between two consecutive events is modeled as a general Weibull distribution. We determine then the transition probabilities and the so-called crossing states probabilities. We conclude then with a Monte Carlo simulation and the model is validated through a large database containing real data.     

A multiple group item response theory model with centered skew-normal latent trait distributions under a Bayesian framework

Very often, in psychometric research, as in educational assessment, it is necessary to analyze item response from clustered respondents. The multiple group item response theory (IRT) model proposed by Bock and Zimowski [12] provides a useful framework for analyzing such type of data. In this model, the selected groups of respondents are of specific interest such that group-specific population distributions need to be defined. The usual assumption for parameter estimation in this model, which is that the latent traits are random variables following different symmetric normal distributions, has been questioned in many works found in the IRT literature. Furthermore, when this assumption does not hold, misleading inference can result. In this paper, we consider that the latent traits for each group follow different skew-normal distributions, under the centered parameterization. We named it skew multiple group IRT model. This modeling extends the works of Azevedo et al. [4], Bazán et al. [11] and Bock and Zimowski [12] (concerning the latent trait distribution). Our approach ensures that the model is identifiable. We propose and compare, concerning convergence issues, two Monte Carlo Markov Chain (MCMC) algorithms for parameter estimation. A simulation study was performed in order to evaluate parameter recovery for the proposed model and the selected algorithm concerning convergence issues. Results reveal that the proposed algorithm recovers properly all model parameters. Furthermore, we analyzed a real data set which presents asymmetry concerning the latent traits distribution. The results obtained by using our approach confirmed the presence of negative asymmetry for some latent trait distributions.     

A structural model for the markov chain of change in vote intention

Assuming a first-order Markov chain, we propose a structural model for the transition probabilities in vote intention. The proposed model utilizes the ordering among the categories representing vote intentions and carries the flavor of distance models. It also allows a stochastic ordering among distributions reflecting the extent of change. The model is easy to fit and provides a nice interpretation of the data. The model is applied to a panel study of vote intention acquired through six successive interviews before the 1940 Presidential election in Erie County, Ohio.     

State-space models for multivariate longitudinal data of mixed types

We propose a class of state-space models for multivariate longitudinal data where the components of the response vector may have different distributions. The approach is based on the class of Tweedie exponential dispersion models, which accommodates a wide variety of discrete, continuous and mixed data. The latent process is assumed to be a Markov process, and the observations are conditionally independent given the latent process, over time as well as over the components of the response vector. This provides a fully parametric alternative to the quasilikelihood approach of Liang and Zeger. We estimate the regression parameters for time-varying covariates entering either via the observation model or via the latent process, based on an estimating equation derived from the Kalman smoother. We also consider analysis of residuals from both the observation model and the latent process.     

A Bayesian Change-Point Analysis for Software Reliability Models

In most software reliability models which utilize the nonhomogeneous Poisson process (NHPP), the intensity function for the counting process is usually assumed to be continuous and monotone. However, on account of various practical reasons, there may exist some change points in the intensity function and thus the assumption of continuous and monotone intensity function may be unrealistic in many real situations. In this article, the Bayesian change-point approach using beta-mixtures for modeling the intensity function with possible change points is proposed. The hidden Markov model with non constant transition probabilities is applied to the beta-mixture for detecting the change points of the parameters. The estimation and interpretation of the model is illustrated using the Naval Tactical Data System (NTDS) data. The proposed change point model will be also compared with the competing models via marginal likelihood. It can be seen that the proposed model has the highest marginal likelihood and outperforms the competing models.     

Markov zero-inflated Poisson regression models for a time series of counts with excess zeros

This paper discusses a class of Markov zero-inflated Poisson regression models for a time series of counts with the presence of excess zero relative to a Poisson distribution, in which the frequency distribution changes according to an underlying two-state Markov chain. Features of the proposed model, estimation method based on the EM and quasi-Newton algorithms, and other implementation issues are discussed. A Monte Carlo study shows that the estimation method is accurate and reliable as long as the sample size is reasonably large, and the choice of starting probabilities for the Markov process has little impact on the parameter estimates. The methodology is illustrated using daily numbers of phone calls reporting faults for a mainframe computer system.     

Bayesian model learning based on a parallel MCMC strategy

We introduce a novel Markov chain Monte Carlo algorithm for estimation of posterior probabilities over discrete model spaces. Our learning approach is applicable to families of models for which the marginal likelihood can be analytically calculated, either exactly or approximately, given any fixed structure. It is argued that for certain model neighborhood structures, the ordinary reversible Metropolis-Hastings algorithm does not yield an appropriate solution to the estimation problem. Therefore, we develop an alternative, non-reversible algorithm which can avoid the scaling effect of the neighborhood. To efficiently explore a model space, a finite number of interacting parallel stochastic processes is utilized. Our interaction scheme enables exploration of several local neighborhoods of a model space simultaneously, while it prevents the absorption of any particular process to a relatively inferior state. We illustrate the advantages of our method by an application to a classification model. In particular, we use an extensive bacterial database and compare our results with results obtained by different methods for the same data.     

Online change detection of Markov chains with unknown post-change transition probabilities

ABSTRACT

In this paper, we investigate the performance of cumulative sum (CUSUM) stopping rules for the online detection of unknown change point in a time homogeneous Markov chain. Under the condition that the post-change transition probabilities are unknown, we proposed two CUSUM type schemes for the detection. The first scheme is based on the maximum likelihood estimates of the post-change transition probabilities. This scheme is limited by its computation burden, which is mitigated by another scheme based on the reference transition probabilities selected from a prior known region. We give the bounds of the mean delay time and the mean time between false alarms to illustrate the effectiveness of the proposed schemes. The results of the simulation also demonstrate the feasibility of the proposed schemes.     

A Bayesian Model for Markov Chains via Jeffrey's Prior

Abstract

This work deals with the problem of Bayesian estimation of the transition probabilities associated with multistate Markov chain. The model is based on the Jeffreys' noninformative prior. The Bayesian estimator is approximated by means of MCMC techniques. A numerical study by simulation is done in order to compare the Bayesian estimator with the maximum likelihood estimator.