Nonparametric likelihood and doubly robust estimating equations for marginal and nested structural models

This article considers Robins's marginal and nested structural models in the cross‐sectional setting and develops likelihood and regression estimators. First, a nonparametric likelihood method is proposed by retaining a finite subset of all inherent and modelling constraints on the joint distributions of potential outcomes and covariates under a correctly specified propensity score model. A profile likelihood is derived by maximizing the nonparametric likelihood over these joint distributions subject to the retained constraints. The maximum likelihood estimator is intrinsically efficient based on the retained constraints and weakly locally efficient. Second, two regression estimators, named hat and tilde, are derived as first‐order approximations to the likelihood estimator under the propensity score model. The tilde regression estimator is intrinsically and weakly locally efficient and doubly robust. The methods are illustrated by data analysis for an observational study on right heart catheterization. The Canadian Journal of Statistics 38: 609–632; 2010 © 2010 Statistical Society of Canada     

Doubly robust empirical likelihood inference in covariate-missing data problems

Missing covariate data occurs often in regression analysis. We study methods for estimating the regression coefficients in an assumed conditional mean function when some covariates are completely observed but other covariates are missing for some subjects. We adopt the semiparametric perspective of Robins et al. [Estimation of regression coefficients when some regressors are not always observed. J Amer Statist Assoc. 1994;89:846–866] on regression analyses with missing covariates, in which they pioneered the use of two working models, the working propensity score model and the working conditional score model. A recent approach to missing covariate data analysis is the empirical likelihood method of Qin et al. [Empirical likelihood in missing data problems. J Amer Statist Assoc. 2009;104:1492–1503], which effectively combines unbiased estimating equations. In this paper, we consider an alternative likelihood approach based on the full likelihood of the observed data. This full likelihood-based method enables us to generate estimators for the vector of the regression coefficients that are (a) asymptotically equivalent to those of Qin et al. [Empirical likelihood in missing data problems. J Amer Statist Assoc. 2009;104:1492–1503] when the working propensity score model is correctly specified, and (b) doubly robust, like the augmented inverse probability weighting (AIPW) estimators of Robins et al. [Estimation of regression coefficients when some regressors are not always observed. J Am Statist Assoc. 1994;89:846–866]. Thus, the proposed full likelihood-based estimators improve on the efficiency of the AIPW estimators when the working propensity score model is correct but the working conditional score model is possibly incorrect, and also improve on the empirical likelihood estimators of Qin, Zhang and Leung [Empirical likelihood in missing data problems. J Amer Statist Assoc. 2009;104:1492–1503] when the reverse is true, that is, the working conditional score model is correct but the working propensity score model is possibly incorrect. In addition, we consider a regression method for estimation of the regression coefficients when the working conditional score model is correctly specified; the asymptotic variance of the resulting estimator is no greater than the semiparametric variance bound characterized by the theory of Robins et al. [Estimation of regression coefficients when some regressors are not always observed. J Amer Statist Assoc. 1994;89:846–866]. Finally, we compare the finite-sample performance of various estimators in a simulation study.     

Finite sample properties of an HPT estimator when each individual regression coefficient is estimated in a misspecified linear regression model

ABSTRACT

In this paper, assuming that there exist omitted variables in the specified model, we analytically derive the exact formula for the mean squared error (MSE) of a heterogeneous pre-test (HPT) estimator whose components are the ordinary least squares (OLS) and feasible ridge regression (FRR) estimators. Since we cannot examine the MSE performance analytically, we execute numerical evaluations to investigate small sample properties of the HPT estimator, and compare the MSE performance of the HPT estimator with those of the FRR estimator and the usual OLS estimator. Our numerical results show that (1) the HPT estimator is more efficient when the model misspecification is severe; (2) the HPT estimator with the optimal critical value obtained under the correctly specified model can be safely used even when there exist omitted variables in the specified model.     

Robust Linear Calibration

We regard the simple linear calibration problem where only the response y of the regression line y = β₀ + β₁ t is observed with errors. The experimental conditions t are observed without error. For the errors of the observations y we assume that there may be some gross errors providing outlying observations. This situation can be modeled by a conditionally contaminated regression model. In this model the classical calibration estimator based on the least squares estimator has an unbounded asymptotic bias. Therefore we introduce calibration estimators based on robust one-step-M-estimators which have a bounded asymptotic bias. For this class of estimators we discuss two problems: The optimal estimators and their corresponding optimal designs. We derive the locally optimal solutions and show that the maximin efficient designs for non-robust estimation and robust estimation coincide.     

Estimation of conditional cumulative incidence functions under generalized semiparametric regression models with missing covariates,with application to analysis of biomarker correlates in vaccine trials

This article presents generalized semiparametric regression models for conditional cumulative incidence functions with competing risks data when covariates are missing by sampling design or happenstance. A doubly robust augmented inverse probability weighted (AIPW) complete-case approach to estimation and inference is investigated. This approach modifies IPW complete-case estimating equations by exploiting the key features in the relationship between the missing covariates and the phase-one data to improve efficiency. An iterative numerical procedure is derived to solve the nonlinear estimating equations. The asymptotic properties of the proposed estimators are established. A simulation study examining the finite-sample performances of the proposed estimators shows that the AIPW estimators are more efficient than the IPW estimators. The developed method is applied to the RV144 HIV-1 vaccine efficacy trial to investigate vaccine-induced IgG binding antibodies to HIV-1 as correlates of acquisition of HIV-1 infection while taking account of whether the HIV-1 sequences are near or far from the HIV-1 sequences represented in the vaccine construct.     

Robust weighted one-way ANOVA: Improved approximation and efficiency

A robust test for the one-way ANOVA model under heteroscedasticity is developed in this paper. The data are assumed to be symmetrically distributed, apart from some outliers, although the assumption of normality may be violated. The test statistic to be used is a weighted sum of squares similar to the Welch [1951. On the comparison of several mean values: an alternative approach. Biometrika 38, 330-336.] test statistic, but any of a variety of robust measures of location and scale for the populations of interest may be used instead of the usual mean and standard deviation. Under the commonly occurring condition that the robust measures of location and scale are asymptotically normal, we derive approximations to the distribution of the test statistic under the null hypothesis and to its distribution under alternative hypotheses. An expression for relative efficiency is derived, thus allowing comparison of the efficiency of the test as a function of the choice of the location and scale estimators used in the test statistic. As an illustration of the theory presented here, we apply it to three commonly used robust location–scale estimator pairs: the trimmed mean with the Winsorized standard deviation; the Huber Proposal 2 estimator pair; and the Hampel robust location estimator with the median absolute deviation.     

A comparison of doubly robust estimators of the mean with missing data

We consider data with a continuous outcome that is missing at random and a fully observed set of covariates. We compare by simulation a variety of doubly-robust (DR) estimators for estimating the mean of the outcome. An estimator is DR if it is consistent when either the regression model for the mean function or the propensity to respond is correctly specified. Performance of different methods is compared in terms of root mean squared error of the estimates and width and coverage of confidence intervals or posterior credibility intervals in repeated samples. Overall, the DR methods tended to yield better inference than the incorrect model when either the propensity or mean model is correctly specified, but were less successful for small sample sizes, where the asymptotic DR property is less consequential. Two methods tended to outperform the other DR methods: penalized spline of propensity prediction [Little RJA, An H. Robust likelihood-based analysis of multivariate data with missing values. Statist Sinica. 2004;14:949–968] and the robust method proposed in [Cao W, Tsiatis AA, Davidian M. Improving efficiency and robustness of the doubly robust estimator for a population mean with incomplete data. Biometrika. 2009;96:723–734].     

Exclusion restrictions in instrumental variables equations

In this paper we consider two-stage estimators of parameters of a structural equation in a model with recursive exclusion restrictions on the instrumental variables equations. The estimations considered are simple OLS and GLS estimators after substitution of estimates of the systematic part of the IV equations for the endogenous variables. It is known in the literature that neither imposing the restrictions in the first stage nor ignoring them will in general be more efficient than the alternative. We introduce a class of mixed instrumental variables estimators (MIV) with these possibilities as special cases which yields an estimator which is not only more efficient than the two stage estimators considered in the literature but as efficient as an efficient system estimator like 3SLS.     

PMSE dominance of the positive-part shrinkage estimator in a regression model with proxy variables

Consider a linear regression model with some relevant regressors are unobservable. In such a situation, we estimate the model by using the proxy variables as regressors or by simply omitting the relevant regressors. In this paper, we derive the explicit formula of predictive mean squared error (PMSE) of a general family of shrinkage estimators of regression coefficients. It is shown analytically that the positive-part shrinkage estimator dominates the ordinary shrinkage estimator even when proxy variables are used in place of the unobserved variables. Also, as an example, our result is applied to the double k-class estimator proposed by Ullah and Ullah (Double k-class estimators of coefficients in linear regression. Econometrica. 1978;46:705–722). Our numerical results show that the positive-part double k-class estimator with proxy variables has preferable PMSE performance.     

Exclusion restrictions in instrumental variables equations

In this paper we consider two-stage estimators of parameters of a structural equation in a model with recursive exclusion restrictions on the instrumental variables equations. The estimations considered are simple OLS and GLS estimators after substitution of estimates of the systematic part of the IV equations for the endogenous variables. It is known in the literature that neither imposing the restrictions in the first stage nor ignoring them will in general be more efficient than the alternative. We introduce a class of mixed instrumental variables estimators (MIV) with these possibilities as special cases which yields an estimator which is not only more efficient than the two stage estimators considered in the literature but as efficient as an efficient system estimator like 3SLS.     

Using generalized doubly robust estimator to estimate average treatment effects of multiple treatments in observational studies

The generalized doubly robust estimator is proposed for estimating the average treatment effect (ATE) of multiple treatments based on the generalized propensity score (GPS). In medical researches where observational studies are conducted, estimations of ATEs are usually biased since the covariate distributions could be unbalanced among treatments. To overcome this problem, Imbens [The role of the propensity score in estimating dose-response functions, Biometrika 87 (2000), pp. 706–710] and Feng et al. [Generalized propensity score for estimating the average treatment effect of multiple treatments, Stat. Med. (2011), in press. Available at: http://onlinelibrary.wiley.com/doi/10.1002/sim.4168/abstract] proposed weighted estimators that are extensions of a ratio estimator based on GPS to estimate ATEs with multiple treatments. However, the ratio estimator always produces a larger empirical sample variance than the doubly robust estimator, which estimates an ATE between two treatments based on the estimated propensity score (PS). We conduct a simulation study to compare the performance of our proposed estimator with Imbens’ and Feng et al.’s estimators, and simulation results show that our proposed estimator outperforms their estimators in terms of bias, empirical sample variance and mean-squared error of the estimated ATEs.     

Estimation of the scale parameter after a pre-test for homogeneity in a mis-specified regression model

The risk properties of estimators of the scale parameter after a pre-test for homogeneity of the error variances in the two sample linear regression model has received quite an amount of attention in the literature. This literature typically assumes normal disturbances and a properly specified model. In this paper we consider that both equations may be mis-specified by the omission of relevant regressors and that the error distributions may belong to a wider class than the normal distribution. We derive and analyse the exact risk (under quadratic loss) of the pre-test estimator of the scale parameter for the first sub-sample.     

M-estimators for isotonic regression

In this paper we propose a family of robust estimates for isotonic regression: isotonic M-estimators. We show that their asymptotic distribution is, up to an scalar factor, the same as that of Brunk's classical isotonic estimator. We also derive the influence function and the breakdown point of these estimates. Finally we perform a Monte Carlo study that shows that the proposed family includes estimators that are simultaneously highly efficient under Gaussian errors and highly robust when the error distribution has heavy tails.     

Semiparametric ARCH Models: An Estimating Function Approach

We introduce the method of estimating functions to study the class of autoregressive conditional heteroscedasticity (ARCH) models. We derive the optimal estimating functions by combining linear and quadratic estimating functions. The resultant estimators are more efficient than the quasi-maximum likelihood estimator. If the assumption of conditional normality is imposed, the estimator obtained by using the theory of estimating functions is identical to that obtained by using the maximum likelihood method in finite samples. The relative efficiencies of the estimating function (EF) approach in comparison with the quasi-maximum likelihood estimator are developed. We illustrate the EF approach using a univariate GARCH(1,1) model with conditional normal, Student-t, and gamma distributions. The efficiency benefits of the EF approach relative to the quasi-maximum likelihood approach are substantial for the gamma distribution with large skewness. Simulation analysis shows that the finite-sample properties of the estimators from the EF approach are attractive. EF estimators tend to display less bias and root mean squared error than the quasi-maximum likelihood estimator. The efficiency gains are substantial for highly nonnormal distributions. An example demonstrates that implementation of the method is straightforward.     

Minimum distance estimation in linear regression with strong mixing errors

Abstract

Minimum distance estimation on the linear regression model with independent errors is known to yield an efficient and robust estimator. We extend the method to the model with strong mixing errors and obtain an estimator of the vector of the regression parameters. The goal of this article is to demonstrate the proposed estimator still retains efficiency and robustness. To that end, this article investigates asymptotic distributional properties of the proposed estimator and compares it with other estimators. The efficiency and the robustness of the proposed estimator are empirically shown, and its superiority over the other estimators is established.     

A sufficient condition for the MSE dominance of the positive-part shrinkage estimator when each individual regression coefficient is estimated in a misspecified linear regression model

In this paper, assuming that there exist omitted explanatory variables in the specified model, we derive the exact formula for the mean squared error (MSE) of a general family of shrinkage estimators for each individual regression coefficient. It is shown analytically that when our concern is to estimate each individual regression coefficient, the positive-part shrinkage estimators have smaller MSE than the original shrinkage estimators under some conditions even when the relevant regressors are omitted. Also, by numerical evaluations, we showed the effects of our theorem for several specific cases. It is shown that the positive-part shrinkage estimators have smaller MSE than the original shrinkage estimators for wide region of parameter space even when there exist omitted variables in the specified model.     

Inference for proportional hazard model with propensity score

Since the publication of the seminal paper by Cox (1972), proportional hazard model has become very popular in regression analysis for right censored data. In observational studies, treatment assignment may depend on observed covariates. If these confounding variables are not accounted for properly, the inference based on the Cox proportional hazard model may perform poorly. As shown in Rosenbaum and Rubin (1983), under the strongly ignorable treatment assignment assumption, conditioning on the propensity score yields valid causal effect estimates. Therefore we incorporate the propensity score into the Cox model for causal inference with survival data. We derive the asymptotic property of the maximum partial likelihood estimator when the model is correctly specified. Simulation results show that our method performs quite well for observational data. The approach is applied to a real dataset on the time of readmission of trauma patients. We also derive the asymptotic property of the maximum partial likelihood estimator with a robust variance estimator, when the model is incorrectly specified.     

Doubly robust point and variance estimation in the presence of imputed survey data

Imputation is often used in surveys to treat item nonresponse. It is well known that treating the imputed values as observed values may lead to substantial underestimation of the variance of the point estimators. To overcome the problem, a number of variance estimation methods have been proposed in the literature, including resampling methods such as the jackknife and the bootstrap. In this paper, we consider the problem of doubly robust inference in the presence of imputed survey data. In the doubly robust literature, point estimation has been the main focus. In this paper, using the reverse framework for variance estimation, we derive doubly robust linearization variance estimators in the case of deterministic and random regression imputation within imputation classes. Also, we study the properties of several jackknife variance estimators under both negligible and nonnegligible sampling fractions. A limited simulation study investigates the performance of various variance estimators in terms of relative bias and relative stability. Finally, the asymptotic normality of imputed estimators is established for stratified multistage designs under both deterministic and random regression imputation. The Canadian Journal of Statistics 40: 259–281; 2012 © 2012 Statistical Society of Canada     

Nonparametric quasi-likelihood for right censored data

Quasi-likelihood was extended to right censored data to handle heteroscedasticity in the frame of the accelerated failure time (AFT) model. However, the assumption of known variance function in the quasi-likelihood for right censored data is usually unrealistic. In this paper, we propose a nonparametric quasi-likelihood by replacing the specified variance function with a nonparametric variance function estimator. This nonparametric variance function estimator is obtained by smoothing a function of squared residuals via local polynomial regression. The rate of convergence of the nonparametric variance function estimator and the asymptotic limiting distributions of the regression coefficient estimators are derived. It is demonstrated in simulations that for finite samples the proposed nonparametric quasi-likelihood method performs well. The new method is illustrated with one real dataset.     

Robust quasi-randomization-based estimation with ensemble learning for missing data

Missing data analysis requires assumptions about an outcome model or a response probability model to adjust for potential bias due to nonresponse. Doubly robust (DR) estimators are consistent if at least one of the models is correctly specified. Multiply robust (MR) estimators extend DR estimators by allowing for multiple models for both the outcome and/or response probability models and are consistent if at least one of the multiple models is correctly specified. We propose a robust quasi-randomization-based model approach to bring more protection against model misspecification than the existing DR and MR estimators, where any multiple semiparametric, nonparametric or machine learning models can be used for the outcome variable. The proposed estimator achieves unbiasedness by using a subsampling Rao–Blackwell method, given cell-homogenous response, regardless of any working models for the outcome. An unbiased variance estimation formula is proposed, which does not use any replicate jackknife or bootstrap methods. A simulation study shows that our proposed method outperforms the existing multiply robust estimators.