Invariant tests based on M-estimators,estimating functions,and the generalized method of moments

We study the invariance properties of various test criteria which have been proposed for hypothesis testing in the context of incompletely specified models, such as models which are formulated in terms of estimating functions (Godambe, 1960) or moment conditions and are estimated by generalized method of moments (GMM) procedures (Hansen, 1982), and models estimated by pseudo-likelihood (Gouriéroux, Monfort, and Trognon, 1984b,c) and M-estimation methods. The invariance properties considered include invariance to (possibly nonlinear) hypothesis reformulations and reparameterizations. The test statistics examined include Wald-type, LR-type, LM-type, score-type, and C(α)?type criteria. Extending the approach used in Dagenais and Dufour (1991), we show first that all these test statistics except the Wald-type ones are invariant to equivalent hypothesis reformulations (under usual regularity conditions), but all five of them are not generally invariant to model reparameterizations, including measurement unit changes in nonlinear models. In other words, testing two equivalent hypotheses in the context of equivalent models may lead to completely different inferences. For example, this may occur after an apparently innocuous rescaling of some model variables. Then, in view of avoiding such undesirable properties, we study restrictions that can be imposed on the objective functions used for pseudo-likelihood (or M-estimation) as well as the structure of the test criteria used with estimating functions and generalized method of moments (GMM) procedures to obtain invariant tests. In particular, we show that using linear exponential pseudo-likelihood functions allows one to obtain invariant score-type and C(α)?type test criteria, while in the context of estimating function (or GMM) procedures it is possible to modify a LR-type statistic proposed by Newey and West (1987) to obtain a test statistic that is invariant to general reparameterizations. The invariance associated with linear exponential pseudo-likelihood functions is interpreted as a strong argument for using such pseudo-likelihood functions in empirical work.     

Asymptotics of Selective Inference

In this paper, we seek to establish asymptotic results for selective inference procedures removing the assumption of Gaussianity. The class of selection procedures we consider are determined by affine inequalities, which we refer to as affine selection procedures. Examples of affine selection procedures include selective inference along the solution path of the least absolute shrinkage and selection operator (LASSO), as well as selective inference after fitting the least absolute shrinkage and selection operator at a fixed value of the regularization parameter. We also consider some tests in penalized generalized linear models. Our result proves asymptotic convergence in the high‐dimensional setting where n<p, and n can be of a logarithmic factor of the dimension p for some procedures.     

ON PREDICTION OF TOTAL VALUE IN INCOMPLETELY SPECIFIED DOMAINS

This paper presents the problem of prediction of a domain total value based on the general linear model. In many methods presented in the survey sampling literature (e.g. Cassel, Särndal & Wretman, 1977 [Foundations of inference in survey sampling, New York: John Wiley & Sons]; Valliant, Dorfman & Royall, 2000 [Finite population sampling and inference. A prediction approach. New York: John Wiley & Sons]; Rao, 2003 [Small area estimation. New York; John Wiley & Sons]) a common assumption is that for each element of a population the domain to which it belongs is known. This assumption is especially important in the situation when a superpopulation model with auxiliary variables is considered. In this paper a method is proposed for prediction of the domain total when it is not known whether a unit belongs to a given domain or not, or when the information is available only for sampled elements of the population.     

A new heteroskedasticity-consistent covariance matrix estimator for the linear regression model

The assumption that all random errors in the linear regression model share the same variance (homoskedasticity) is often violated in practice. The ordinary least squares estimator of the vector of regression parameters remains unbiased, consistent and asymptotically normal under unequal error variances. Many practitioners then choose to base their inferences on such an estimator. The usual practice is to couple it with an asymptotically valid estimation of its covariance matrix, and then carry out hypothesis tests that are valid under heteroskedasticity of unknown form. We use numerical integration methods to compute the exact null distributions of some quasi-t test statistics, and propose a new covariance matrix estimator. The numerical results favor testing inference based on the estimator we propose.     

Tests for homogeneity of risk differences in stratified design with correlated bilateral data

ABSTRACT

Correlated bilateral data arise from stratified studies involving paired body organs in a subject. When it is desirable to conduct inference on the scale of risk difference, one needs first to assess the assumption of homogeneity in risk differences across strata. For testing homogeneity of risk differences, we herein propose eight methods derived respectively from weighted-least-squares (WLS), the Mantel-Haenszel (MH) estimator, the WLS method in combination with inverse hyperbolic tangent transformation, and the test statistics based on their log-transformation, the modified Score test statistic and Likelihood ratio test statistic. Simulation results showed that four of the tests perform well in general, with the tests based on the WLS method and inverse hyperbolic tangent transformation always performing satisfactorily even under small sample size designs. The methods are illustrated with a dataset.     

Asymptotic properties of a goodness-of-fit test based on maximum correlations

We study the efficiency properties of the goodness-of-fit test based on the Q _n statistic introduced in Fortiana and Grané [Goodness-of-fit tests based on maximum correlations and their orthogonal decompositions, J. R. Stat. Soc. B 65 (2003), pp. 115–126] using the concepts of Bahadur asymptotic relative efficiency and Bahadur asymptotic optimality. We compare the test based on this statistic with those based on the Kolmogorov–Smirnov, the Cramér-von Mises criterion and the Anderson–Darling statistics. We also describe the distribution families for which the test based on Q _n is locally asymptotically optimal in the Bahadur sense and, as an application, we use this test to detect the presence of hidden periodicities in a stationary time series.     

Fisher information in weighted distributions

Suppose that a density f_θ (x) belongs to an exponential family, but that inference about θ must be based on data that are obtained from a density that is proportional to W(x)f_θ(x). The authors study the Fisher information about θ in observations obtained from such weighted distributions and give conditions under which this information is greater than under the original density. These conditions involve the hazard- and reversed-hazard-rate functions.     

Non-nested hypothesis testing inference for GAMLSS models

Two or more regression models are said to be non-nested if neither can be obtained from the remaining models when parametric restrictions are imposed. Tests for choosing between linear non-nested regression models are found in literature, such as J and MJ tests. In this paper we propose variants of these two tests for the GAMLSS (Generalized Additive Models for Location, Scale and Shape) class of models. We report Monte Carlo evidence on finite sample behaviour of the proposed tests. Bootstrap-based testing inference is also considered. Overall, bootstrap MJ test had the best performance. An empirical application is presented and discussed.     

A note on L1 consistent estimation

Let (??, ??) be a space with a σ-field, M = {P_s; s o} a family of probability measures on A, Θ arbitrary, X₁,…,X_n independently and identically distributed P random variables. Metrize Θ with the L₁ distance between measures, and assume identifiability. Minimum-distance estimators are constructed that relate rates of convergence with Vapnik-Cervonenkis exponents when M is “regular”. An alternative construction of estimates is offered via Kolmogorov's chain argument.     

M-Estimates of regression when the scale is unknown and the error distribution is possibly asymmetric: A minimax result

Huber (1964) found the minimax-variance M-estimate of location under the assumption that the scale parameter is known; Li and Zamar (1991) extended this result to the case when the scale is unknown. We consider the robust estimation of the regression coefficients (β₁,…,β_p) when the scale and the intercept parameters are unknown. The minimax-variance estimates of (β₁,…,β_p) with respect to the trace of their asymptotic covariance matrix are derived. The maximum is taken over ?-contamination neighbourhoods of a central regression model with Gaussian errors (asymmetric contamination is allowed), and the minimum is taken over a large class of generalized M-estimates of regression of the Mallow type. The optimal choice of estimates for the nuisance parameters (scale and intercept) is also considered.     

Heteroscedasticity diagnostics in two-phase linear regression models

In two-phase linear regression models, it is a standard assumption that the random errors of two phases have constant variances. However, this assumption is not necessarily appropriate. This paper is devoted to the tests for variance heterogeneity in these models. We initially discuss the simultaneous test for variance heterogeneity of two phases. When the simultaneous test shows that significant heteroscedasticity occurs in the whole model, we construct two individual tests to investigate whether or not both phases or one of them have/has significant heteroscedasticity. Several score statistics and their adjustments based on Cox and Reid [D. R. Cox and N. Reid, Parameter orthogonality and approximate conditional inference. J. Roy. Statist. Soc. Ser. B 49 (1987), pp. 1–39] are obtained and illustrated with Australian onion data. The simulated powers of test statistics are investigated through Monte Carlo methods.     

Propriétés statistiques des copules de valeurs extrêmes bidimensionnelles

Let (X, Y) be a bivariate random vector whose distribution function H(x, y) belongs to the class of bivariate extreme-value distributions. If F₁ and F₂ are the marginals of X and Y, then H(x, y) = C{F₁(x),F₂(y)}, where C is a bivariate extreme-value dependence function. This paper gives the joint distribution of the random variables Z = {log F₁(X)}/{log F₁(X)F₂(Y)} and W = C{F₁{(X),F₂(Y)}. Using this distribution, an algorithm to generate random variables having bivariate extreme-value distribution is présentés. Furthermore, it is shown that for any bivariate extreme-value dependence function C, the distribution of the random variable W = C{F₁(X),F₂(Y)} belongs to a monoparametric family of distributions. This property is used to derive goodness-of-fit statistics to determine whether a copula belongs to an extreme-value family.     

Minimum hellinger distance estimation for normal models

A robust estimator introduced by Beran (1977a, 1977b), which is based on the minimum Hellinger distance between a projection model density and a nonparametric sample density, is studied empirically. An extensive simulation provides an estimate of the small sample distribution and supplies empirical evidence of the estimator performance for a normal location-scale model. While the performance of the minimum Hellinger distance estimator is seen to be competitive with the maximum likelihood estimator at the true model, its robustness to deviations from normality is shown to be competitive in this setting with that obtained from the M-estimator and the Cramér-von Mises minimum distance estimator. Beran also introduced a goodness-of-fit statisticH ₂, based on the minimized Hellinger distance between a member of a parametric family of densities and a nonparametric density estimate. We investigate the statistic H (the square root of H ₂) as a test for normality when both location and scale are unspecified. Empirically derived critical values are given which do not require extensive tables. The power of the statistic H compares favorably with four other widely used tests for normality.     

Approximating M/G/1 Waiting Time Tail Probabilities

Abstract

We propose a new approximation formula for the waiting time tail probability of the M/G/1 queue with FIFO discipline and unlimited waiting space. The aim is to address the difficulty of obtaining good estimates when the tail probability has non-exponential asymptotics. We show that the waiting time tail probability can be expressed in terms of the waiting time tail probability of a notional M/G/1 queue with truncated service time distribution plus the tail probability of an extreme order statistic. The Cramér–Lundberg approximation is applied to approximate the tail probability of the notional queue. In essence, our technique extends the applicability of the Cramér–Lundberg approximation to cases where the standard Lundberg condition does not hold. We propose a simple moment-based technique for estimating the parameters of the approximation; numerical results demonstrate that our approximation can yield very good estimates over the whole range of the argument.     

Sequential Estimation of Expectations in the Presence of Trend2

A sequence of independent random variables {Z_n:n≥ 1} with unknown probability distributions is considered and the problem of estimating their expectations {M_n+1: n≥ 1} is examined. The estimation of M_n+1 is based on a finite set {z_k:1≤k≤n}, each z_k being an observed value of Z_k, 1 ≤k≤n, and also based on the assumption that {M_n:n≥ 1} follows an unknown trend of a specified form.     

Linear mode regression with covariate measurement error

We consider estimating the mode of a response given an error‐prone covariate. It is shown that ignoring measurement error typically leads to inconsistent inference for the conditional mode of the response given the true covariate, as well as misleading inference for regression coefficients in the conditional mode model. To account for measurement error, we first employ the Monte Carlo corrected score method (Novick & Stefanski, 2002) to obtain an unbiased score function based on which the regression coefficients can be estimated consistently. To relax the normality assumption on measurement error this method requires, we propose another method where deconvoluting kernels are used to construct an objective function that is maximized to obtain consistent estimators of the regression coefficients. Besides rigorous investigation on asymptotic properties of the new estimators, we study their finite sample performance via extensive simulation experiments, and find that the proposed methods substantially outperform a naive inference method that ignores measurement error. The Canadian Journal of Statistics 47: 262–280; 2019 © 2019 Statistical Society of Canada     

Modified inference function for margins for the bivariate clayton copula-based SUN Tobit Model

This paper extends the analysis of the bivariate Seemingly Unrelated Regression (SUN) Tobit model by modeling its nonlinear dependence structure through the Clayton copula. The ability to capture/model the lower tail dependence of the SUN Tobit model where some data are censored (generally, left-censored at zero) is an useful feature of the Clayton copula. We propose a modified version of the (classical) Inference Function for Margins (IFS) method by Joe and XP [H. Joe and J.J. XP, The estimation method of inference functions for margins for multivariate models, Tech. Rep. 166, Department of Statistics, University of British Columbia, 1996], which we refer to as Modified Inference Function for Margins (MIFF) method, to obtain the (point) estimates of the marginal and Clayton copula parameters. More specifically, we employ the (frequenting) data augmentation technique at the second stage of the IFS method (the first stage of the MIFF method is equivalent to the first stage of the IFS method) to generate the censored observations and then estimate the Clayton copula parameter. This process (data augmentation and copula parameter estimation) is repeated until convergence. Such modification at the second stage of the usual estimation method is justified in order to obtain continuous marginal distributions, which ensures the uniqueness of the resulting Clayton copula, as stated by Solar's [A. Solar, Fonctions de répartition à n dimensions et leurs marges, Publ. de l'Institut de Statistique de l'Université de Paris 8 (1959), pp. 229–231] theorem; and also to provide an unbiased estimate of the association parameter (the IFS method provides a biased estimate of the Clayton copula parameter in the presence of censored observations in both margins). Since the usual asymptotic approach, that is the computation of the asymptotic covariance matrix of the parameter estimates, is troublesome in this case, we also propose the use of resampling procedures (bootstrap methods, such as standard normal and percentile, by Efron and Tibshirani [B. Efron and R.J. Tibshirani, An Introduction to the Bootstrap, Chapman & Hall, New York, 1993] to obtain confidence intervals for the model parameters.     

Bartlett corrections in beta regression models

We consider the issue of performing accurate small-sample testing inference in beta regression models, which are useful for modeling continuous variates that assume values in (0,1), such as rates and proportions. We derive the Bartlett correction to the likelihood ratio test statistic and also consider a bootstrap Bartlett correction. Using Monte Carlo simulations we compare the finite sample performances of the two corrected tests to that of the standard likelihood ratio test and also to its variant that employs Skovgaard's adjustment; the latter is already available in the literature. The numerical evidence favors the corrected tests we propose. We also present an empirical application.     

Quadratic subspaces and construction of Bayes invariant quadratic estimators of variance components in mixed linear models

Gnot et al. (J Statist Plann Inference 30(1):223–236, 1992) have presented the formulae for computing Bayes invariant quadratic estimators of variance components in normal mixed linear models of the form where the matrices V _i , 1 ≤ i ≤ k − 1, are symmetric and nonnegative definite and V _k is an identity matrix. These formulae involve a basis of a quadratic subspace containing MV ₁ M,...,MV _k-1 M,M, where M is an orthogonal projector on the null space of X′. In the paper we discuss methods of construction of such a basis. We survey Malley’s algorithms for finding the smallest quadratic subspace including a given set of symmetric matrices of the same order and propose some modifications of these algorithms. We also consider a class of matrices sharing some of the symmetries common to MV ₁ M,...,MV _k-1 M,M. We show that the matrices from this class constitute a quadratic subspace and describe its explicit basis, which can be directly used for computing Bayes invariant quadratic estimators of variance components. This basis can be also used for improving the efficiency of Malley’s algorithms when applied to finding a basis of the smallest quadratic subspace containing the matrices MV ₁ M,...,MV _k-1 M,M. Finally, we present the results of a numerical experiment which confirm the potential usefulness of the proposed methods. Dedicated to the memory of Professor Stanisław Gnot.     

Likelihood ratio inference for mean residual life

One of the basic parameters in survival analysis is the mean residual life M ₀. For right censored observation, the usual empirical likelihood based log-likelihood ratio leads to a scaled c₁²{\chi_1^2} limit distribution and estimating the scaled parameter leads to lower coverage of the corresponding confidence interval. To solve the problem, we present a log-likelihood ratio l(M ₀) by methods of Murphy and van der Vaart (Ann Stat 1471–1509, 1997). The limit distribution of l(M ₀) is the standard c₁²{\chi_1^2} distribution. Based on the limit distribution of l(M ₀), the corresponding confidence interval of M ₀ is constructed. Since the proof of the limit distribution does not offer a computational method for the maximization of the log-likelihood ratio, an EM algorithm is proposed. Simulation studies support the theoretical result.