A simulation study of least squares and ridge estimaors for normal and nonnormal autocorrelated disturbances

The purpose of this paper is to examine the small sample properties of various ridge estimators along with least squares, in some special settings.Specifically, we consider a first order autoregressive structuure for normal and nonnormal disturbances, and report on a Monte Carlo study the small sample behavior of these estimators according to the criteria of bias and dispersion.The results suggest that under all the examined settings and for all the criteria used the HKB estimator exhibited a superior performance compared to the other estimators, while the LS and LW estimators gave consistently poor results.Also if the error term is only moderately autocorrelated the performance of the ridge estimators that do not account for autocorrelation outperform their counterparts as well as least squares that account for autocorrelation.     

Nonparametric curve estimation with missing data: A general empirical process approach

A general nonparametric imputation procedure, based on kernel regression, is proposed to estimate points as well as set- and function-indexed parameters when the data are missing at random (MAR). The proposed method works by imputing a specific function of a missing value (and not the missing value itself), where the form of this specific function is dictated by the parameter of interest. Both single and multiple imputations are considered. The associated empirical processes provide the right tool to study the uniform convergence properties of the resulting estimators. Our estimators include, as special cases, the imputation estimator of the mean, the estimator of the distribution function proposed by Cheng and Chu [1996. Kernel estimation of distribution functions and quantiles with missing data. Statist. Sinica 6, 63–78], imputation estimators of a marginal density, and imputation estimators of regression functions.     

On Bias Reduction in Robust Inference for Generalized Linear Models

Abstract.  It is well known that one or more outlying points in the data may adversely affect the consistency of the quasi-likelihood or the likelihood estimators for the regression effects. Similar to the quasi-likelihood approach, the existing outliers-resistant Mallow's type quasi-likelihood (MQL) estimation approach may also produce biased regression estimators. As a remedy, by using a fully standardized score function in the MQL estimating equation, in this paper, we demonstrate that the fully standardized MQL estimators are almost unbiased ensuring its higher consistency performance. Both count and binary responses subject to one or more outliers are used in the study. The small sample as well as asymptotic results for the competitive estimators are discussed.     

Immaculating the inconsistent estimator of slope parameter in measurement error model with replicated data

ABSTRACT

The measurement error model with replicated data on study as well as explanatory variables is considered. The measurement error variance associated with the explanatory variable is estimated using the complete data and the grouped data which is used for the construction of the consistent estimators of regression coefficient. These estimators are further used in constructing an almost unbiased estimator of regression coefficient. The large sample properties of these estimators are derived without assuming any distributional form of the measurement errors and the random error component under the setup of an ultrastructural model.     

Nonparametric estimation for survival data with censoring indicators missing at random

In this paper, we consider the problem of hazard rate estimation in the presence of covariates, for survival data with censoring indicators missing at random. We propose in the context usually denoted by MAR (missing at random, in opposition to MCAR, missing completely at random, which requires an additional independence assumption), nonparametric adaptive strategies based on model selection methods for estimators admitting finite dimensional developments in functional orthonormal bases. Theoretical risk bounds are provided, they prove that the estimators behave well in term of mean square integrated error (MISE). Simulation experiments illustrate the statistical procedure.     

Doubly robust empirical likelihood inference in covariate-missing data problems

Missing covariate data occurs often in regression analysis. We study methods for estimating the regression coefficients in an assumed conditional mean function when some covariates are completely observed but other covariates are missing for some subjects. We adopt the semiparametric perspective of Robins et al. [Estimation of regression coefficients when some regressors are not always observed. J Amer Statist Assoc. 1994;89:846–866] on regression analyses with missing covariates, in which they pioneered the use of two working models, the working propensity score model and the working conditional score model. A recent approach to missing covariate data analysis is the empirical likelihood method of Qin et al. [Empirical likelihood in missing data problems. J Amer Statist Assoc. 2009;104:1492–1503], which effectively combines unbiased estimating equations. In this paper, we consider an alternative likelihood approach based on the full likelihood of the observed data. This full likelihood-based method enables us to generate estimators for the vector of the regression coefficients that are (a) asymptotically equivalent to those of Qin et al. [Empirical likelihood in missing data problems. J Amer Statist Assoc. 2009;104:1492–1503] when the working propensity score model is correctly specified, and (b) doubly robust, like the augmented inverse probability weighting (AIPW) estimators of Robins et al. [Estimation of regression coefficients when some regressors are not always observed. J Am Statist Assoc. 1994;89:846–866]. Thus, the proposed full likelihood-based estimators improve on the efficiency of the AIPW estimators when the working propensity score model is correct but the working conditional score model is possibly incorrect, and also improve on the empirical likelihood estimators of Qin, Zhang and Leung [Empirical likelihood in missing data problems. J Amer Statist Assoc. 2009;104:1492–1503] when the reverse is true, that is, the working conditional score model is correct but the working propensity score model is possibly incorrect. In addition, we consider a regression method for estimation of the regression coefficients when the working conditional score model is correctly specified; the asymptotic variance of the resulting estimator is no greater than the semiparametric variance bound characterized by the theory of Robins et al. [Estimation of regression coefficients when some regressors are not always observed. J Amer Statist Assoc. 1994;89:846–866]. Finally, we compare the finite-sample performance of various estimators in a simulation study.     

Aalen's linear model for doubly censored data

Double censoring often occurs in registry studies when left censoring is present in addition to right censoring. In this work, we examine estimation of Aalen's nonparametric regression coefficients based on doubly censored data. We propose two estimation techniques. The first type of estimators, including ordinary least squared (OLS) estimator and weighted least squared (WLS) estimators, are obtained using martingale arguments. The second type of estimator, the maximum likelihood estimator (MLE), is obtained via expectation-maximization (EM) algorithms that treat the survival times of left censored observations as missing. Asymptotic properties, including the uniform consistency and weak convergence, are established for the MLE. Simulation results demonstrate that the MLE is more efficient than the OLS and WLS estimators.     

Added variable plots for linear regression with censored data

In linear regression the structure of the hat matrix plays an important part in regression diagnostics. In this note we investigate the properties of the hat matrix for regression with censored responses in the presence of one or more explanatory variables observed without censoring. The censored points in the scatterplot are renovated to positions had they been observed without censoring in a renovation process based on Buckley-James censored regression estimators. This allows natural links to be established with the structure of ordinary least squares estimators. In particular, we show that the renovated hat matrix may be partitioned in a manner which assists in deciding whether further explanatory variables should be added to the linear model. The added variable plot for regression with censored data is developed as a diagnostic tool for this decision process.     

Analysis of longitudinal data with irregular, outcome-dependent follow-up

Summary.  A frequent problem in longitudinal studies is that subjects may miss scheduled visits or be assessed at self-selected points in time. As a result, observed outcome data may be highly unbalanced and the availability of the data may be directly related to the outcome measure and/or some auxiliary factors that are associated with the outcome. If the follow-up visit and outcome processes are correlated, then marginal regression analyses will produce biased estimates. Building on the work of Robins, Rotnitzky and Zhao, we propose a class of inverse intensity-of-visit process-weighted estimators in marginal regression models for longitudinal responses that may be observed in continuous time. This allows us to handle arbitrary patterns of missing data as embedded in a subject's visit process. We derive the large sample distribution for our inverse visit-intensity-weighted estimators and investigate their finite sample behaviour by simulation. Our approach is illustrated with a data set from a health services research study in which homeless people with mental illness were randomized to three different treatments and measures of homelessness (as percentage days homeless in the past 3 months) and other auxiliary factors were recorded at follow-up times that are not fixed by design.     

Application of an imputation method for variance estimation under pseudo-likelihood when missing data are NMAR

When data are outcome-dependent non response, pseudo-likelihood yields consistent regression coefficients without specifying the missing data mechanism. However, it is onerous to derive parameter estimators including their standard errors from the regression coefficients under pseudo-likelihood (PL). The present study applies an imputation method to compute the asymptotic standard errors of parameter estimators. The proposed method is simpler than Delta method and it showed similar effect size of the standard errors to bootstrapping in simulation and application studies.     

Estimation of regression models with equi-correlated responses when some observations on the response variable are missing

The present article deals with the problem of estimation of parameters in a linear regression model when some data on response variable is missing and the responses are equi-correlated. The ordinary least squares and optimal homogeneous predictors are employed to find the imputed values of missing observations. Their efficiency properties are analyzed using the small disturbances asymptotic theory. The estimation of regression coefficients using these imputed values is also considered and a comparison of estimators is presented.     

A new modified Jackknifed estimator for the Poisson regression model

The Poisson regression is very popular in applied researches when analyzing the count data. However, multicollinearity problem arises for the Poisson regression model when the independent variables are highly intercorrelated. Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators and some methods for estimating the ridge parameter k in the Poisson regression have been proposed. It has been found that some estimators are better than the commonly used maximum-likelihood (ML) estimator and some other RR estimators. In this study, the modified Jackknifed Poisson ridge regression (MJPR) estimator is proposed to remedy the multicollinearity. A simulation study and a real data example are provided to evaluate the performance of estimators. Both mean-squared error and the percentage relative error are considered as the performance criteria. The simulation study and the real data example results show that the proposed MJPR method outperforms the Poisson ridge regression, Jackknifed Poisson ridge regression and the ML in all of the different situations evaluated in this paper.     

Choosing shrinkage estimators for regression problems

A Bayesian formulation of the canonical form of the standard regression model is used to compare various Stein-type estimators and the ridge estimator of regression coefficients, A particular (“constant prior”) Stein-type estimator having the same pattern of shrinkage as the ridge estimator is recommended for use.     

Simulated deficiencies of robust estimators of a proportionality constant

In linear regression, robust methods are at the beginning of their use in practice. In the small sample case, such robust methods provide a necessary measure of protection against deviations from the assumed error distribution. This paper studies through simulation the deficiencies of bioptimal estimators and compares them with more common methods like Huber's estimator or Tukey's estimator. Polyoptimal estimators are convex combinations of Pitman estimators and are optimally robust for a confrontation containing several shapes. The word confrontation is due to J.W. Tukey. It expresses the situation when compromising two or several error distributions. The paper uses the confrontation containing the Gaussian distribution along with a symmetric heavy-tailed distribution having a tail of order 0(t^-2) as t→ ±∞.     

Weighted exponential distribution: properties and different methods of estimation

In this article, we investigate various properties and methods of estimation of the Weighted Exponential distribution. Although, our main focus is on estimation (from both frequentist and Bayesian point of view) yet, the stochastic ordering, the Bonferroni and the Lorenz curves, various entropies and order statistics are derived first time for the said distribution. Different types of loss functions are considered for Bayesian estimation. Furthermore, the Bayes estimators and their respective posterior risks are computed and compared using Gibbs sampling. The different reliability characteristics including hazard function, stress and strength analysis, and mean residual life function are also derived. Monte Carlo simulations are performed to compare the performances of the proposed methods of estimation and two real data sets have been analysed for illustrative purposes.     

A resampling method by perturbing the estimating functions for quantile regression with missing data

In this article, we propose a resampling method based on perturbing the estimating functions to compute the asymptotic variances of quantile regression estimators under missing at random condition. We prove that the conditional distributions of the resampling estimators are asymptotically equivalent to the distributions of quantile regression estimators. Our method can deal with complex situations, where the response and part of covariates are missing. Numerical results based on simulated and real data are provided under several designs.     

A robust pairwise likelihood method for incomplete longitudinal binary data arising in clusters

Clustered longitudinal data feature cross‐sectional associations within clusters, serial dependence within subjects, and associations between responses at different time points from different subjects within the same cluster. Generalized estimating equations are often used for inference with data of this sort since they do not require full specification of the response model. When data are incomplete, however, they require data to be missing completely at random unless inverse probability weights are introduced based on a model for the missing data process. The authors propose a robust approach for incomplete clustered longitudinal data using composite likelihood. Specifically, pairwise likelihood methods are described for conducting robust estimation with minimal model assumptions made. The authors also show that the resulting estimates remain valid for a wide variety of missing data problems including missing at random mechanisms and so in such cases there is no need to model the missing data process. In addition to describing the asymptotic properties of the resulting estimators, it is shown that the method performs well empirically through simulation studies for complete and incomplete data. Pairwise likelihood estimators are also compared with estimators obtained from inverse probability weighted alternating logistic regression. An application to data from the Waterloo Smoking Prevention Project is provided for illustration. The Canadian Journal of Statistics 39: 34–51; 2011 © 2010 Statistical Society of Canada     

Handling missing data by deleting completely observed records

When data are missing, analyzing records that are completely observed may cause bias or inefficiency. Existing approaches in handling missing data include likelihood, imputation and inverse probability weighting. In this paper, we propose three estimators inspired by deleting some completely observed data in the regression setting. First, we generate artificial observation indicators that are independent of outcome given the observed data and draw inferences conditioning on the artificial observation indicators. Second, we propose a closely related weighting method. The proposed weighting method has more stable weights than those of the inverse probability weighting method (Zhao, L., Lipsitz, S., 1992. Designs and analysis of two-stage studies. Statistics in Medicine 11, 769–782). Third, we improve the efficiency of the proposed weighting estimator by subtracting the projection of the estimating function onto the nuisance tangent space. When data are missing completely at random, we show that the proposed estimators have asymptotic variances smaller than or equal to the variance of the estimator obtained from using completely observed records only. Asymptotic relative efficiency computation and simulation studies indicate that the proposed weighting estimators are more efficient than the inverse probability weighting estimators under wide range of practical situations especially when the missingness proportion is large.     

Shrinkage estimation in lognormal regression model for censored data

We introduce in this paper, the shrinkage estimation method in the lognormal regression model for censored data involving many predictors, some of which may not have any influence on the response of interest. We develop the asymptotic properties of the shrinkage estimators (SEs) using the notion of asymptotic distributional biases and risks. We show that if the shrinkage dimension exceeds two, the asymptotic risk of the SEs is strictly less than the corresponding classical estimators. Furthermore, we study the penalty (LASSO and adaptive LASSO) estimation methods and compare their relative performance with the SEs. A simulation study for various combinations of the inactive predictors and censoring percentages shows that the SEs perform better than the penalty estimators in certain parts of the parameter space, especially when there are many inactive predictors in the model. It also shows that the shrinkage and penalty estimators outperform the classical estimators. A real-life data example using Worcester heart attack study is used to illustrate the performance of the suggested estimators.     

Estimation of the additive hazards model with interval-censored data and missing covariates

The additive hazards model is one of the most commonly used regression models in the analysis of failure time data and many methods have been developed for its inference in various situations. However, no established estimation procedure exists when there are covariates with missing values and the observed responses are interval-censored; both types of complications arise in various settings including demographic, epidemiological, financial, medical and sociological studies. To address this deficiency, we propose several inverse probability weight-based and reweighting-based estimation procedures for the situation where covariate values are missing at random. The resulting estimators of regression model parameters are shown to be consistent and asymptotically normal. The numerical results that we report from a simulation study suggest that the proposed methods work well in practical situations. An application to a childhood cancer survival study is provided. The Canadian Journal of Statistics 48: 499–517; 2020 © 2020 Statistical Society of Canada