Markov switching model of nonlinear autoregressive with skew-symmetric innovations

We consider data generating structures which can be represented as a Markov switching of nonlinear autoregressive model with considering skew-symmetric innovations such that switching between the states is controlled by a hidden Markov chain. We propose semi-parametric estimators for the nonlinear functions of the proposed model based on a maximum likelihood (ML) approach and study sufficient conditions for geometric ergodicity of the process. Also, an Expectation-Maximization type optimization for obtaining the ML estimators are presented. A simulation study and a real world application are also performed to illustrate and evaluate the proposed methodology.     

Nonparametric estimation in semi-parametric univariate mixture models

We introduce new estimates of the mixing proportions, locations, and variances of the components of a finite univariate mixture model. We assume that the components are symmetric and differ only in the locations. No parametric model assumptions are imposed on the components. Further, when there is additional information available in the form of training samples that contain information concerning the mixing proportion, the new methods are robust to the symmetry assumption.     

Bayesian analysis of generalized elliptical semi-parametric models

In this paper, we study the statistical inference based on the Bayesian approach for regression models with the assumption that independent additive errors follow normal, Student-t, slash, contaminated normal, Laplace or symmetric hyperbolic distribution, where both location and dispersion parameters of the response variable distribution include nonparametric additive components approximated by B-splines. This class of models provides a rich set of symmetric distributions for the model error. Some of these distributions have heavier or lighter tails than the normal as well as different levels of kurtosis. In order to draw samples of the posterior distribution of the interest parameters, we propose an efficient Markov Chain Monte Carlo (MCMC) algorithm, which combines Gibbs sampler and Metropolis–Hastings algorithms. The performance of the proposed MCMC algorithm is assessed through simulation experiments. We apply the proposed methodology to a real data set. The proposed methodology is implemented in the R package BayesGESM using the function gesm().     

Double threshold autoregressive conditionally heteroscedastic model building by genetic algorithms

Asymmetric behaviour in both mean and variance is often observed in real time series. The approach we adopt is based on double threshold autoregressive conditionally heteroscedastic (DTARCH) model with normal innovations. This model allows threshold nonlinearity in mean and volatility to be modelled as a result of the impact of lagged changes in assets and squared shocks, respectively. A methodology for building DTARCH models is proposed based on genetic algorithms (GAs). The most important structural parameters, that is regimes and thresholds, are searched for by GAs, while the remaining structural parameters, that is the delay parameters and models orders, vary in some pre-specified intervals and are determined using exhaustive search and an Asymptotic Information Criterion (AIC) like criterion. For each structural parameters trial set, a DTARCH model is fitted that maximizes the (penalized) likelihood (AIC criterion). For this purpose the iteratively weighted least squares algorithm is used. Then the best model according to the AIC criterion is chosen. Extension to the double threshold generalized ARCH (DTGARCH) model is also considered. The proposed methodology is checked using both simulated and market index data. Our findings show that our GAs-based procedure yields results that comparable to that reported in the literature and concerned with real time series. As far as artificial time series are considered, the proposed procedure seems to be able to fit the data quite well. In particular, a comparison is performed between the present procedure and the method proposed by Tsay [Tsay, R.S., 1989, Testing and modeling threshold autoregressive processes. Journal of the American Statistical Association, Theory and Methods, 84, 231–240.] for estimating the delay parameter. The former almost always yields better results than the latter. However, adopting Tsay's procedure as a preliminary stage for finding the appropriate delay parameter may save computational time specially if the delay parameter may vary in a large interval.     

Variable selection in finite mixture of semi-parametric regression models

Abstract

In this paper we are concerned with variable selection in finite mixture of semiparametric regression models. This task consists of model selection for non parametric component and variable selection for parametric part. Thus, we encountered separate model selections for every non parametric component of each sub model. To overcome this computational burden, we introduced a class of variable selection procedures for finite mixture of semiparametric regression models using penalized approach for variable selection. It is shown that the new method is consistent for variable selection. Simulations show that the performance of proposed method is good, and it consequently improves pervious works in this area and also requires much less computing power than existing methods.     

Semiparametric mixture of additive regression models

In this article, we propose a semiparametric mixture of additive regression models, in which the regression functions are additive and non parametric while the mixing proportions and variances are constant. Compared with the mixture of linear regression models, the proposed methodology is more flexible in modeling the non linear relationship between the response and covariate. A two-step procedure based on the spline-backfitted kernel method is derived for computation. Moreover, we establish the asymptotic normality of the resultant estimators and examine their good performance through a numerical example.     

Mixtures of regression models with incomplete and noisy data

The estimation of the mixtures of regression models is usually based on the normal assumption of components and maximum likelihood estimation of the normal components is sensitive to noise, outliers, or high-leverage points. Missing values are inevitable in many situations and parameter estimates could be biased if the missing values are not handled properly. In this article, we propose the mixtures of regression models for contaminated incomplete heterogeneous data. The proposed models provide robust estimates of regression coefficients varying across latent subgroups even under the presence of missing values. The methodology is illustrated through simulation studies and a real data analysis.     

Robust fitting of hidden Markov regression models under a longitudinal setting

We propose a robust estimation procedure for the analysis of longitudinal data including a hidden process to account for unobserved heterogeneity between subjects in a dynamic fashion. We show how to perform estimation by an expectation–maximization-type algorithm in the hidden Markov regression literature. We show that the proposed robust approaches work comparably to the maximum-likelihood estimator when there are no outliers and the error is normal and outperform it when there are outliers or the error is heavy tailed. A real data application is used to illustrate our proposal. We also provide details on a simple criterion to choose the number of hidden states.     

Harmonic mean approach to unbalanced mixed effects models under heteroscedastic variances

In this paper we consider unbalanced mixed models (Scheffe's model) under heteroscedastic variances. By using the harmonic mean approach, It is shown that the problems appear to be anologous to those problems from balanced mixed models under homoscedastic variance. Thus, by using harmonic mean approach, statistical inferences about fixed effects and variance components are derived by using those from balanced models under homoscedastic variance. Laguerre polynomial expansion is used Lo approximate sampling distributions of relevant statistics.     

Label switching and its solutions for frequentist mixture models

In this article, the label switching problem and the importance of solving it are discussed for frequentist mixture models if a simulation study is used to evaluate the performance of mixture model estimators. Two effective labelling methods are proposed by using true label for each observation. The empirical studies demonstrate that the new proposed methods work well and provide better results than the rule of thumb method of order constraint labelling. In addition, a Monte Carlo study also demonstrates that simple order constraint labelling can sometimes produce severely biased, and possibly meaningless, estimated bias and standard errors.     

Semiparametric models: a generalized self-consistency approach

Summary. In semiparametric models, the dimension d of the maximum likelihood problem is potentially unlimited. Conventional estimation methods generally behave like O ( d ³). A new O ( d ) estimation procedure is proposed for a large class of semiparametric models. Potentially unlimited dimension is handled in a numerically efficient way through a Nelson–Aalen-like estimator. Discussion of the new method is put in the context of recently developed minorization–maximization algorithms based on surrogate objective functions. The procedure for semiparametric models is used to demonstrate three methods to construct a surrogate objective function: using the difference of two concave functions, the EM way and the new quasi-EM (QEM) approach. The QEM approach is based on a generalization of the EM-like construction of the surrogate objective function so it does not depend on the missing data representation of the model. Like the EM algorithm, the QEM method has a dual interpretation, a result of merging the idea of surrogate maximization with the idea of imputation and self-consistency. The new approach is compared with other possible approaches by using simulations and analysis of real data. The proportional odds model is used as an example throughout the paper.     

Model selection for the localized mixture of experts models

In this paper, we propose a penalized likelihood method to simultaneous select covariate, and mixing component and obtain parameter estimation in the localized mixture of experts models. We develop an expectation maximization algorithm to solve the proposed penalized likelihood procedure, and introduce a data-driven procedure to select the tuning parameters. Extensive numerical studies are carried out to compare the finite sample performances of our proposed method and other existing methods. Finally, we apply the proposed methodology to analyze the Boston housing price data set and the baseball salaries data set.     

A continuous-time HMM approach to modeling the magnitude-frequency distribution of earthquakes

The magnitude-frequency distribution (MFD) of earthquake is a fundamental statistic in seismology. The so-called b-value in the MFD is of particular interest in geophysics. A continuous time hidden Markov model (HMM) is proposed for characterizing the variability of b-values. The HMM-based approach to modeling the MFD has some appealing properties over the widely used sliding-window approach. Often, large variability appears in the estimation of b-value due to window size tuning, which may cause difficulties in interpretation of b-value heterogeneities. Continuous-time hidden Markov models (CT-HMMs) are widely applied in various fields. It bears some advantages over its discrete time counterpart in that it can characterize heterogeneities appearing in time series in a finer time scale, particularly for highly irregularly-spaced time series, such as earthquake occurrences. We demonstrate an expectation–maximization algorithm for the estimation of general exponential family CT-HMM. In parallel with discrete-time hidden Markov models, we develop a continuous time version of Viterbi algorithm to retrieve the overall optimal path of the latent Markov chain. The methods are applied to New Zealand deep earthquakes. Before the analysis, we first assess the completeness of catalogue events to assure the analysis is not biased by missing data. The estimation of b-value is stable over the selection of magnitude thresholds, which is ideal for the interpretation of b-value variability.     

A likelihood ratio test of a homoscedastic normal mixture against a heteroscedastic normal mixture

It is generally assumed that the likelihood ratio statistic for testing the null hypothesis that data arise from a homoscedastic normal mixture distribution versus the alternative hypothesis that data arise from a heteroscedastic normal mixture distribution has an asymptotic χ ² reference distribution with degrees of freedom equal to the difference in the number of parameters being estimated under the alternative and null models under some regularity conditions. Simulations show that the χ ² reference distribution will give a reasonable approximation for the likelihood ratio test only when the sample size is 2000 or more and the mixture components are well separated when the restrictions suggested by Hathaway (Ann. Stat. 13:795–800, 1985) are imposed on the component variances to ensure that the likelihood is bounded under the alternative distribution. For small and medium sample sizes, parametric bootstrap tests appear to work well for determining whether data arise from a normal mixture with equal variances or a normal mixture with unequal variances.     

Estimation and diagnostics for heteroscedastic nonlinear regression models based on scale mixtures of skew-normal distributions

An extension of some standard likelihood based procedures to heteroscedastic nonlinear regression models under scale mixtures of skew-normal (SMSN) distributions is developed. This novel class of models provides a useful generalization of the heteroscedastic symmetrical nonlinear regression models (Cysneiros et al., 2010), since the random term distributions cover both symmetric as well as asymmetric and heavy-tailed distributions such as skew-t, skew-slash, skew-contaminated normal, among others. A simple EM-type algorithm for iteratively computing maximum likelihood estimates of the parameters is presented and the observed information matrix is derived analytically. In order to examine the performance of the proposed methods, some simulation studies are presented to show the robust aspect of this flexible class against outlying and influential observations and that the maximum likelihood estimates based on the EM-type algorithm do provide good asymptotic properties. Furthermore, local influence measures and the one-step approximations of the estimates in the case-deletion model are obtained. Finally, an illustration of the methodology is given considering a data set previously analyzed under the homoscedastic skew-t nonlinear regression model.     

Mixture models for matched pairs

Mixture models for matched pairs arise when the pair-specific parameter is assumed to be a random quantity. We explore the use of semiparametric maximum-likelihood methods for a family of mixture models for matched pairs. The geometry of mixture likelihoods provides insight into the properties of these models.     

Mixtures of Linear Regression with Measurement Errors

Existing research on mixtures of regression models are limited to directly observed predictors. The estimation of mixtures of regression for measurement error data imposes challenges for statisticians. For linear regression models with measurement error data, the naive ordinary least squares method, which directly substitutes the observed surrogates for the unobserved error-prone variables, yields an inconsistent estimate for the regression coefficients. The same inconsistency also happens to the naive mixtures of regression estimate, which is based on the traditional maximum likelihood estimator and simply ignores the measurement error. To solve this inconsistency, we propose to use the deconvolution method to estimate the mixture likelihood of the observed surrogates. Then our proposed estimate is found by maximizing the estimated mixture likelihood. In addition, a generalized EM algorithm is also developed to find the estimate. The simulation results demonstrate that the proposed estimation procedures work well and perform much better than the naive estimates.     

The independent factor analysis approach to latent variable modelling

Independent factor analysis (IFA) has recently been proposed in the signal processing literature as a way to model a set of observed variables through linear combinations of latent independent variables and a noise term. A peculiarity of the method is that it defines a probability density function for the latent variables by mixtures of Gaussians. The aim of this paper is to cast the method into a more rigorous statistical framework and to propose some developments. In the first part, we present the IFA model in its population version, address identifiability issues and draw some parallels between the IFA model and the ordinary factor analysis (FA) one. Then we show that the IFA model may be reinterpreted as an independent component analysis-based rotation of an ordinary FA solution. We also give evidence that the IFA model represents a special case of mixture of factor analysers. In the second part, we address inferential issues, also deriving the standard errors for the model parameter estimates and providing model selection criteria. Finally, we present some empirical results on real data sets.     

Non parametric mixture of strictly monotone regression models

In this article, we propose the non parametric mixture of strictly monotone regression models. For implementation, a two-step procedure is derived. We further establish the asymptotic normality of the resultant estimator and demonstrate its good performance through numerical examples.     

Switching nonparametric regression models

We propose a methodology to analyse data arising from a curve that, over its domain, switches among J states. We consider a sequence of response variables, where each response y depends on a covariate x according to an unobserved state z. The states form a stochastic process and their possible values are j=1,?…?, J. If z equals j the expected response of y is one of J unknown smooth functions evaluated at x. We call this model a switching nonparametric regression model. We develop an Expectation–Maximisation algorithm to estimate the parameters of the latent state process and the functions corresponding to the J states. We also obtain standard errors for the parameter estimates of the state process. We conduct simulation studies to analyse the frequentist properties of our estimates. We also apply the proposed methodology to the well-known motorcycle dataset treating the data as coming from more than one simulated accident run with unobserved run labels.