Profile Hellinger distance estimation

The successful application of the Hellinger distance approach to fully parametric models is well known. The corresponding optimal estimators, known as minimum Hellinger distance (MHD) estimators, are efficient and have excellent robustness properties [Beran R. Minimum Hellinger distance estimators for parametric models. Ann Statist. 1977;5:445–463]. This combination of efficiency and robustness makes MHD estimators appealing in practice. However, their application to semiparametric statistical models, which have a nuisance parameter (typically of infinite dimension), has not been fully studied. In this paper, we investigate a methodology to extend the MHD approach to general semiparametric models. We introduce the profile Hellinger distance and use it to construct a minimum profile Hellinger distance estimator of the finite-dimensional parameter of interest. This approach is analogous in some sense to the profile likelihood approach. We investigate the asymptotic properties such as the asymptotic normality, efficiency, and adaptivity of the proposed estimator. We also investigate its robustness properties. We present its small-sample properties using a Monte Carlo study.     

On minimum Hellinger distance estimation

Efficiency and robustness are two fundamental concepts in parametric estimation problems. It was long thought that there was an inherent contradiction between the aims of achieving robustness and efficiency; that is, a robust estimator could not be efficient and vice versa. It is now known that the minimum Hellinger distance approached introduced by Beran [R. Beran, Annals of Statistics 1977;5:445–463] is one way of reconciling the conflicting concepts of efficiency and robustness. For parametric models, it has been shown that minimum Hellinger estimators achieve efficiency at the model density and simultaneously have excellent robustness properties. In this article, we examine the application of this approach in two semiparametric models. In particular, we consider a two‐component mixture model and a two‐sample semiparametric model. In each case, we investigate minimum Hellinger distance estimators of finite‐dimensional Euclidean parameters of particular interest and study their basic asymptotic properties. Small sample properties of the proposed estimators are examined using a Monte Carlo study. The results can be extended to semiparametric models of general form as well. The Canadian Journal of Statistics 37: 514–533; 2009 © 2009 Statistical Society of Canada     

Recursive estimation of bilinear time series models

In this paper we are concerned with the recursive estimation of bilinear models. Some methods from linear time invariant systems are adapted to suit bilinear time series models. The time-varying Kalman filter and associated parameter estimation algorithm is carried on the bilinear time series models. The methods are illustrated with examples.     

Robust and efficient estimation of GARCH models based on Hellinger distance

It is well known that financial data frequently contain outlying observations. Almost all methods and techniques used to estimate GARCH models are likelihood-based and thus generally non-robust against outliers. Minimum distance method, as an important tool for statistical inferences and a competitive alternative for achieving robustness, has surprisingly not been well explored for GARCH models. In this paper, we proposed a minimum Hellinger distance estimator (MHDE) and a minimum profile Hellinger distance estimator (MPHDE), depending on whether the innovation distribution is specified or not, for estimating the parameters in GARCH models. The construction and investigation of the two estimators are quite involved due to the non-i.i.d. nature of data. We proved that the MHDE is a consistent estimator and derived its bias in explicit expression. For both of the proposed estimators, we demonstrated their finite-sample performance through simulation studies and compared with the well-established methods including MLE, Gaussian Quasi-MLE, Non-Gaussian Quasi-MLE and Least Absolute Deviation estimator. Our numerical results showed that MHDE and MPHDE have much better performance than MLE-based methods when data are contaminated while simultaneously they are very competitive when data is clean, which testified to the robustness and efficiency of the two proposed MHD-type estimations.     

Moment method estimation of first-order continuous-time bilinear processes

In the present paper, we propose an estimation method of the first order continuous-time bilinear (COBL) process based on Euler-Maruyama discretization of the Itô solution asociated with the stochastic differerential equation (SDE) defining the process, and we suggest a standard moment method (MM) estimates of the unknown parameters involving in COBL process. So, some relationships linking the parameters and the theoretical moments of the process and its quadratic version are given. These relationships we allow to construct two algorithms to estimate the parameters based on MM. Using the fact that the incremented processes are strongly mixing with exponential rate whenever certain conditions are fulfilled, we show that the resulting estimators are strongly consistent and asymptotically normal. The theory can be applied to the COGARCH(1, 1), Gaussian Ornstein-Uhlenbeck (OU) models and among other specifications. Finite sample properties are also considered throught Monte-Carlo experimencts. In end, this algorithm is then used to model the exchanges rate of the Algerian Dinar against the US-dollar and against the single European currency.     

Improved model selection criteria for SETAR time series models

The purpose of this paper is threefold. First, we obtain the asymptotic properties of the modified model selection criteria proposed by Hurvich et al. (1990. Improved estimators of Kullback-Leibler information for autoregressive model selection in small samples. Biometrika 77, 709–719) for autoregressive models. Second, we provide some highlights on the better performance of this modified criteria. Third, we extend the modification introduced by these authors to model selection criteria commonly used in the class of self-exciting threshold autoregressive (SETAR) time series models. We show the improvements of the modified criteria in their finite sample performance. In particular, for small and medium sample size the frequency of selecting the true model improves for the consistent criteria and the root mean square error (RMSE) of prediction improves for the efficient criteria. These results are illustrated via simulation with SETAR models in which we assume that the threshold and the parameters are unknown.     

Minimum hellinger distance estimation for normal models

A robust estimator introduced by Beran (1977a, 1977b), which is based on the minimum Hellinger distance between a projection model density and a nonparametric sample density, is studied empirically. An extensive simulation provides an estimate of the small sample distribution and supplies empirical evidence of the estimator performance for a normal location-scale model. While the performance of the minimum Hellinger distance estimator is seen to be competitive with the maximum likelihood estimator at the true model, its robustness to deviations from normality is shown to be competitive in this setting with that obtained from the M-estimator and the Cramér-von Mises minimum distance estimator. Beran also introduced a goodness-of-fit statisticH ₂, based on the minimized Hellinger distance between a member of a parametric family of densities and a nonparametric density estimate. We investigate the statistic H (the square root of H ₂) as a test for normality when both location and scale are unspecified. Empirically derived critical values are given which do not require extensive tables. The power of the statistic H compares favorably with four other widely used tests for normality.     

On Using Hellinger Distance in Checking the Validity of Dynamic Approximations

The purpose of this article is to provide validation for the approximate algebraic propagation algorithms to accommodate non-Gaussian dynamic processes. These algorithms have been developed to carry out Bayesian analysis based on conjugate forms and presented with detailed examples of response distributions such as Poisson and Lognormal. The validity of the approximation algorithms can be checked by introducing a metric (Hellinger divergence measure) over the distribution of the states (parameters) and use it to judge the approximation. Theoretical bounds for the efficacy of such procedure are discussed.     

Estimation of a multiple-threshold AR(p) model

The present paper deals with the multiple-threshold p-order autoregressive model which has been introduced by Tong and Lim [H. Tong, K.S. Lim, Threshold autoregression, limit cycles and cyclical data, J. R. Stat. Soc. Ser. B 42 (1980) 245–292] in nonlinear system modelling. Under some conditions on the coefficients of the model which ensure the stationarity, the existence of moments and the strong mixing property of this process and under other mild assumptions, we establish the asymptotic properties (consistency and asymptotic normality) of the minimum Hellinger distance estimates of the autoregressive coefficients of the model.     

Bayesian analysis of bilinear time series models : a gibbs sampling approach

Nonlinear time series analysis plays an important role in recent econometric literature, especially the bilinear model. In this paper, we cast the bilinear time series model in a Bayesian framework and make inference by using the Gibbs sampler, a Monte Carlo method. The methodology proposed is illustrated by using generated examples, two real data sets, as well as a simulation study. The results show that the Gibbs sampler provides a very encouraging option in analyzing bilinear time series.     

Bayesian inference for the proportion of true null hypotheses using minimum Hellinger distance

It is important that the proportion of true null hypotheses be estimated accurately in a multiple hypothesis context. Current estimation methods, however, are not suitable for high-dimensional data such as microarray data. First, they do not consider the (strong) dependence between hypotheses (or genes), thereby resulting in inaccurate estimation. Second, the unknown distribution of false null hypotheses cannot be estimated properly by these methods. Third, the estimation is affected strongly by outliers. In this paper, we find out the optimal procedure for estimating the proportion of true null hypotheses under a (strong) dependence based on the Dirichlet process prior. In addition, by using the minimum Hellinger distance, the estimation should be robust to any model misspecification as well as to any outliers while maintaining efficiency. The results are confirmed by a simulation study, and the newly developed methodology is illustrated by a real microarray data.     

Second-order least squares estimation of censored regression models

This paper proposes the second-order least squares estimation, which is an extension of the ordinary least squares method, for censored regression models where the error term has a general parametric distribution (not necessarily normal). The strong consistency and asymptotic normality of the estimator are derived under fairly general regularity conditions. We also propose a computationally simpler estimator which is consistent and asymptotically normal under the same regularity conditions. Finite sample behavior of the proposed estimators under both correctly and misspecified models are investigated through Monte Carlo simulations. The simulation results show that the proposed estimator using optimal weighting matrix performs very similar to the maximum likelihood estimator, and the estimator with the identity weight is more robust against the misspecification.     

Asymptotic Properties of the Maximum Quasi-Likelihood Estimator in Quasi-Likelihood Nonlinear Models

Quasi-likelihood nonlinear models (QLNM) are a further extension of generalized linear models by only specifying the expectation and variance functions of the response variable. In this article, some mild regularity conditions are proposed. These regularity conditions, respectively, assure the existence, strong consistency, and the asymptotic normality of the maximum quasi-likelihood estimator (MQLE) in QLNM.     

On nonlinear models for time series

The paper is a review of nonlinear processes used in time series analysis and presents some new original results about stationary distribution of a nonlinear autoregres-sive process of the first order. The following models are considered: nonlinear autoregessive processes, threshold AR processes, threshold MA processes, bilinear models, auto-regressive models with random parameters including double stochastic models, exponential AR models, generalized threshold models and smooth transition autoregressive models, Some tests for linearity of processes are also presented.     

Some asymptotic distribution results in time-sequential estimation of the mean exponential survival time

The asymptotic distribution of the stopping time N in a time-sequential procedure for the estimation of the mean exponential survival time given by Gardiner, Susarla, and van Ryzin (1986) is obtained. The same techniques used to obtain this asymptotic distribution of N are used to obtain the asymptotic distribution of the statistic representing the time-on-test expended per unit item in the study.     

Bootstrap entropy test for general location-scale time series models with heteroscedasticity

This study considers a goodness-of-fit test for location-scale time series models with heteroscedasticity, including a broad class of generalized autoregressive conditional heteroscedastic-type models. In financial time series analysis, the correct identification of model innovations is crucial for further inferences in diverse applications such as risk management analysis. To implement a goodness-of-fit test, we employ the residual-based entropy test generated from the residual empirical process. Since this test often shows size distortions and is affected by parameter estimation, its bootstrap version is considered. It is shown that the bootstrap entropy test is weakly consistent, and thereby its usage is justified. A simulation study and data analysis are conducted by way of an illustration.