An empirical likelihood ratio based goodness-of-fit test for Inverse Gaussian distributions

The Inverse Gaussian (IG) distribution is commonly introduced to model and examine right skewed data having positive support. When applying the IG model, it is critical to develop efficient goodness-of-fit tests. In this article, we propose a new test statistic for examining the IG goodness-of-fit based on approximating parametric likelihood ratios. The parametric likelihood ratio methodology is well-known to provide powerful likelihood ratio tests. In the nonparametric context, the classical empirical likelihood (EL) ratio method is often applied in order to efficiently approximate properties of parametric likelihoods, using an approach based on substituting empirical distribution functions for their population counterparts. The optimal parametric likelihood ratio approach is however based on density functions. We develop and analyze the EL ratio approach based on densities in order to test the IG model fit. We show that the proposed test is an improvement over the entropy-based goodness-of-fit test for IG presented by Mudholkar and Tian (2002). Theoretical support is obtained by proving consistency of the new test and an asymptotic proposition regarding the null distribution of the proposed test statistic. Monte Carlo simulations confirm the powerful properties of the proposed method. Real data examples demonstrate the applicability of the density-based EL ratio goodness-of-fit test for an IG assumption in practice.     

A density based empirical likelihood approach for testing bivariate normality

Sample entropy based tests, methods of sieves and Grenander estimation type procedures are known to be very efficient tools for assessing normality of underlying data distributions, in one-dimensional nonparametric settings. Recently, it has been shown that the density based empirical likelihood (EL) concept extends and standardizes these methods, presenting a powerful approach for approximating optimal parametric likelihood ratio test statistics, in a distribution-free manner. In this paper, we discuss difficulties related to constructing density based EL ratio techniques for testing bivariate normality and propose a solution regarding this problem. Toward this end, a novel bivariate sample entropy expression is derived and shown to satisfy the known concept related to bivariate histogram density estimations. Monte Carlo results show that the new density based EL ratio tests for bivariate normality behave very well for finite sample sizes. To exemplify the excellent applicability of the proposed approach, we demonstrate a real data example.     

Statistic inference for a single-index ARCH-M model

For a single-index autoregressive conditional heteroscedastic (ARCH-M) model, estimators of the parametric and non parametric components are proposed by the profile likelihood method. The research results had shown that all the estimators have consistency and the parametric estimators have asymptotic normality. We extend this line of research by deriving the asymptotic normality of the non parametric estimator. Based on the asymptotic properties, we propose Wald statistic and generalized likelihood ratio statistic to investigate the testing problems for ARCH effect and goodness of fit, respectively. A simulation study is conducted to evaluate the finite-sample performance of the proposed estimation methodology and testing procedure.     

Interpreting statistical evidence by using imperfect models: robust adjusted likelihood functions

Summary. The strength of statistical evidence is measured by the likelihood ratio. Two key performance properties of this measure are the probability of observing strong misleading evidence and the probability of observing weak evidence. For the likelihood function associated with a parametric statistical model, these probabilities have a simple large sample structure when the model is correct. Here we examine how that structure changes when the model fails. This leads to criteria for determining whether a given likelihood function is robust (continuing to perform satisfactorily when the model fails), and to a simple technique for adjusting both likelihoods and profile likelihoods to make them robust. We prove that the expected information in the robust adjusted likelihood cannot exceed the expected information in the likelihood function from a true model. We note that the robust adjusted likelihood is asymptotically fully efficient when the working model is correct, and we show that in some important examples this efficiency is retained even when the working model fails. In such cases the Bayes posterior probability distribution based on the adjusted likelihood is robust, remaining correct asymptotically even when the model for the observable random variable does not include the true distribution. Finally we note a link to standard frequentist methodology—in large samples the adjusted likelihood functions provide robust likelihood-based confidence intervals.     

Profile Hellinger distance estimation

The successful application of the Hellinger distance approach to fully parametric models is well known. The corresponding optimal estimators, known as minimum Hellinger distance (MHD) estimators, are efficient and have excellent robustness properties [Beran R. Minimum Hellinger distance estimators for parametric models. Ann Statist. 1977;5:445–463]. This combination of efficiency and robustness makes MHD estimators appealing in practice. However, their application to semiparametric statistical models, which have a nuisance parameter (typically of infinite dimension), has not been fully studied. In this paper, we investigate a methodology to extend the MHD approach to general semiparametric models. We introduce the profile Hellinger distance and use it to construct a minimum profile Hellinger distance estimator of the finite-dimensional parameter of interest. This approach is analogous in some sense to the profile likelihood approach. We investigate the asymptotic properties such as the asymptotic normality, efficiency, and adaptivity of the proposed estimator. We also investigate its robustness properties. We present its small-sample properties using a Monte Carlo study.     

Two-sample high-dimensional empirical likelihood

In this paper, we apply empirical likelihood for two-sample problems with growing high dimensionality. Our results are demonstrated for constructing confidence regions for the difference of the means of two p-dimensional samples and the difference in value between coefficients of two p-dimensional sample linear model. We show that empirical likelihood based estimator has the efficient property. That is, as p → ∞ for high-dimensional data, the limit distribution of the EL ratio statistic for the difference of the means of two samples and the difference in value between coefficients of two-sample linear model is asymptotic normal distribution. Furthermore, empirical likelihood (EL) gives efficient estimator for regression coefficients in linear models, and can be as efficient as a parametric approach. The performance of the proposed method is illustrated via numerical simulations.     

On empirical likelihood for semiparametric two-sample density ratio models

We consider estimation and test problems for some semiparametric two-sample density ratio models. The profile empirical likelihood (EL) poses an irregularity problem under the null hypothesis that the laws of the two samples are equal. We show that a dual form of the profile EL is well defined even under the null hypothesis. A statistical test, based on the dual form of the EL ratio statistic (ELRS), is then proposed. We give an interpretation for the dual form of the ELRS through φ$\phi$-divergences and duality techniques. The asymptotic properties of the test statistic are presented both under the null and the alternative hypotheses, and approximation of the power function of the test is deduced.     

Sieve Empirical Likelihood and Extensions of the Generalized Least Squares

The empirical likelihood cannot be used directly sometimes when an infinite dimensional parameter of interest is involved. To overcome this difficulty, the sieve empirical likelihoods are introduced in this paper. Based on the sieve empirical likelihoods, a unified procedure is developed for estimation of constrained parametric or non-parametric regression models with unspecified error distributions. It shows some interesting connections with certain extensions of the generalized least squares approach. A general asymptotic theory is provided. In the parametric regression setting it is shown that under certain regularity conditions the proposed estimators are asymptotically efficient even if the restriction functions are discontinuous. In the non-parametric regression setting the convergence rate of the maximum estimator based on the sieve empirical likelihood is given. In both settings, it is shown that the estimator is adaptive for the inhomogeneity of conditional error distributions with respect to predictor, especially for heteroscedasticity.     

Adjusted empirical likelihood for value at risk and expected shortfall

Value at risk (VaR) and expected shortfall (ES) are widely used risk measures of the risk of loss on a specific portfolio of financial assets. Adjusted empirical likelihood (AEL) is an important non parametric likelihood method which is developed from empirical likelihood (EL). It can overcome the limitation of convex hull problems in EL. In this paper, we use AEL method to estimate confidence region for VaR and ES. Theoretically, we find that AEL has the same large sample statistical properties as EL, and guarantees solution to the estimating equations in EL. In addition, simulation results indicate that the coverage probabilities of the new confidence regions are higher than that of the original EL with the same level. These results show that the AEL estimation for VaR and ES deserves to recommend for the real applications.     

Bayes inference for the modulated power law process

The Modulated Power Law process has been recently proposed as a suitable model for describing the failure pattern of repairable systems when both renewal-type behaviour and time trend are present. Unfortunately, the maximum likelihood method provides neither accurate confidence intervals on the model parameters for small or moderate sample sizes nor predictive intervals on future observations.

This paper proposes a Bayes approach, based on both non-informative and vague prior, as an alternative to the classical method. Point and interval estimation of the parameters, as well as point and interval prediction of future failure times, are given. Monte Carlo simulation studies show that the Bayes estimation and prediction possess good statistical properties in a frequentist context and, thus, are a valid alternative to the maximum likelihood approach.

Numerical examples illustrate the estimation and prediction procedures.     

Testing symmetry based on empirical likelihood

In this paper, we propose a general kth correlation coefficient between the density function and distribution function of a continuous variable as a measure of symmetry and asymmetry. We first propose a root-n moment-based estimator of the kth correlation coefficient and present its asymptotic results. Next, we consider statistical inference of the kth correlation coefficient by using the empirical likelihood (EL) method. The EL statistic is shown to be asymptotically a standard chi-squared distribution. Last, we propose a residual-based estimator of the kth correlation coefficient for a parametric regression model to test whether the density function of the true model error is symmetric or not. We present the asymptotic results of the residual-based kth correlation coefficient estimator and also construct its EL-based confidence intervals. Simulation studies are conducted to examine the performance of the proposed estimators, and we also use our proposed estimators to analyze the air quality dataset.     

Statistical Inference in Partially Linear Varying-Coefficient Models with Missing Responses at Random

This article considers statistical inference for partially linear varying-coefficient models when the responses are missing at random. We propose a profile least-squares estimator for the parametric component with complete-case data and show that the resulting estimator is asymptotically normal. To avoid to estimate the asymptotic covariance in establishing confidence region of the parametric component with the normal-approximation method, we define an empirical likelihood based statistic and show that its limiting distribution is chi-squared distribution. Then, the confidence regions of the parametric component with asymptotically correct coverage probabilities can be constructed by the result. To check the validity of the linear constraints on the parametric component, we construct a modified generalized likelihood ratio test statistic and demonstrate that it follows asymptotically chi-squared distribution under the null hypothesis. Then, we extend the generalized likelihood ratio technique to the context of missing data. Finally, some simulations are conducted to illustrate the proposed methods.     

Empirical likelihood based variable selection

Information criteria form an important class of model/variable selection methods in statistical analysis. Parametric likelihood is a crucial part of these methods. In some applications such as the generalized linear models, the models are only specified by a set of estimating functions. To overcome the non-availability of well defined likelihood function, the information criteria under empirical likelihood are introduced. Under this setup, we successfully solve the existence problem of the profile empirical likelihood due to the over constraint in variable selection problems. The asymptotic properties of the new method are investigated. The new method is shown to be consistent at selecting the variables under mild conditions. Simulation studies find that the proposed method has comparable performance to the parametric information criteria when a suitable parametric model is available, and is superior when the parametric model assumption is violated. A real data set is also used to illustrate the usefulness of the new method.     

Dimension-reduced empirical likelihood inference for response mean with data missing at random

To make efficient inference for mean of a response variable when the data are missing at random and the dimension of covariate is not low, we construct three bias-corrected empirical likelihood (EL) methods in conjunction with dimension-reduced kernel estimation of propensity or/and conditional mean response function. Consistency and asymptotic normality of the maximum dimension-reduced EL estimators are established. We further study the asymptotic properties of the resulting dimension-reduced EL ratio functions and the corresponding EL confidence intervals for the response mean are constructed. The finite-sample performance of the proposed estimators is studied through simulation, and an application to HIV-CD4 data set is also presented.     

Generalised quasi‐likelihood inference in a semi‐parametric binary dynamic mixed logit model

There exists a recent study where dynamic mixed‐effects regression models for count data have been extended to a semi‐parametric context. However, when one deals with other discrete data such as binary responses, the results based on count data models are not directly applicable. In this paper, we therefore begin with existing binary dynamic mixed models and generalise them to the semi‐parametric context. For inference, we use a new semi‐parametric conditional quasi‐likelihood (SCQL) approach for the estimation of the non‐parametric function involved in the semi‐parametric model, and a semi‐parametric generalised quasi‐likelihood (SGQL) approach for the estimation of the main regression, dynamic dependence and random effects variance parameters. A semi‐parametric maximum likelihood (SML) approach is also used as a comparison to the SGQL approach. The properties of the estimators are examined both asymptotically and empirically. More specifically, the consistency of the estimators is established and finite sample performances of the estimators are examined through an intensive simulation study.     

Empirical phi-divergence test statistics for testing simple and composite null hypotheses

The main purpose of this paper is to introduce first a new family of empirical test statistics for testing a simple null hypothesis when the vector of parameters of interest is defined through a specific set of unbiased estimating functions. This family of test statistics is based on a distance between two probability vectors, with the first probability vector obtained by maximizing the empirical likelihood (EL) on the vector of parameters, and the second vector defined from the fixed vector of parameters under the simple null hypothesis. The distance considered for this purpose is the phi-divergence measure. The asymptotic distribution is then derived for this family of test statistics. The proposed methodology is illustrated through the well-known data of Newcomb's measurements on the passage time for light. A simulation study is carried out to compare its performance with that of the EL ratio test when confidence intervals are constructed based on the respective statistics for small sample sizes. The results suggest that the ‘empirical modified likelihood ratio test statistic’ provides a competitive alternative to the EL ratio test statistic, and is also more robust than the EL ratio test statistic in the presence of contamination in the data. Finally, we propose empirical phi-divergence test statistics for testing a composite null hypothesis and present some asymptotic as well as simulation results for evaluating the performance of these test procedures.     

ASYMPTOTIC DISTRIBUTIONS OF SEMIPARAMETRIC MAXIMUM LIKELIHOOD ESTIMATORS WITH ESTIMATING EQUATIONS FOR GROUP-CENSORED DATA

Semiparametric maximum likelihood estimation with estimating equations (SMLE) is more flexible than traditional methods; it has fewer restrictions on distributions and regression models. The required information about distribution and regression structures is incorporated in estimating equations of the SMLE to improve the estimation quality of non‐parametric methods. The likelihood of SMLE for censored data involves complicated implicit functions without closed‐form expressions, and the first derivatives of the log‐profile‐likelihood cannot be expressed as summations of independent and identically distributed random variables; it is challenging to derive asymptotic properties of the SMLE for censored data. For group‐censored data, the paper shows that all the implicit functions are well defined and obtains the asymptotic distributions of the SMLE for model parameters and lifetime distributions. With several examples the paper compares the SMLE, the regular non‐parametric likelihood estimation method and the parametric MLEs in terms of their asymptotic efficiencies, and illustrates application of SMLE. Various asymptotic distributions of the likelihood ratio statistics are derived for testing the adequacy of estimating equations and a partial set of parameters equal to some known values.     

Average Most Powerful Tests for a Segmented Regression

To improve the goodness of fit between a regression model and observations, the model can be complicated; however, that can reduce the statistical power when the complication does not lead significantly to an improved model. In the context of two-phase (segmented) logistic regressions, the model evaluation needs to include testing for simple (one-phase) versus two-phase logistic regression models. In this article, we propose and examine a class of likelihood ratio type tests for detecting a change in logistic regression parameters that splits the model into two-phases. We show that the proposed tests, based on Shiryayev–Roberts type statistics, are on average the most powerful. The article argues in favor of a new approach for fixing Type I errors of tests when the parameters of null hypotheses are unknown. Although the suggested approach is partly based on Bayes–Factor-type testing procedures, the classical significance levels of the proposed tests are under control. We demonstrate applications of the average most powerful tests to an epidemiologic study entitled “Time to pregnancy and multiple births.”     

Cressie–Read Power-Divergence Statistics for Non-Gaussian Vector Stationary Processes

Abstract.  For a class of vector-valued non-Gaussian stationary processes, we develop the Cressie–Read power-divergence (CR) statistic approach which has been proposed for the i.i.d. case. The CR statistic includes empirical likelihood as a special case. Therefore, by adopting this CR statistic approach, the theory of estimation and testing based on empirical likelihood is greatly extended. We use an extended Whittle likelihood as score function and derive the asymptotic distribution of the CR statistic. We apply this result to estimation of autocorrelation and the AR coefficient, and get narrower confidence intervals than those obtained by existing methods. We also consider the power properties of the test based on asymptotic theory. Under a sequence of contiguous local alternatives, we derive the asymptotic distribution of the CR statistic. The problem of testing autocorrelation is discussed and we introduce some interesting properties of the local power.     

The characterization of Monte Carlo errors for the quantification of the value of forensic evidence

Recent developments in forensic science have lead to a proliferation of methods for quantifying the probative value of evidence by constructing a Bayes Factor that allows a decision-maker to select between the prosecution and defense models. Unfortunately, the analytical form of a Bayes Factor is often computationally intractable. A typical approach in statistics uses Monte Carlo integration to numerically approximate the marginal likelihoods composing the Bayes Factor. This article focuses on developing a generally applicable method for characterizing the numerical error associated with Monte Carlo integration techniques used in constructing the Bayes Factor. The derivation of an asymptotic Monte Carlo standard error (MCSE) for the Bayes Factor will be presented and its applicability to quantifying the value of evidence will be explored using a simulation-based example involving a benchmark data set. The simulation will also explore the effect of prior choice on the Bayes Factor approximations and corresponding MCSEs.