Regularization with Maximum Entropy and Quantum Electrodynamics: The Merg(E) Estimators

It is well-known that under fairly conditions linear regression becomes a powerful statistical tool. In practice, however, some of these conditions are usually not satisfied and regression models become ill-posed, implying that the application of traditional estimation methods may lead to non-unique or highly unstable solutions. Addressing this issue, in this paper a new class of maximum entropy estimators suitable for dealing with ill-posed models, namely for the estimation of regression models with small samples sizes affected by collinearity and outliers, is introduced. The performance of the new estimators is illustrated through several simulation studies.     

Controlling for confounding via propensity score methods can result in biased estimation of the conditional AUC: A simulation study

In the medical literature, there has been an increased interest in evaluating association between exposure and outcomes using nonrandomized observational studies. However, because assignments to exposure are not random in observational studies, comparisons of outcomes between exposed and nonexposed subjects must account for the effect of confounders. Propensity score methods have been widely used to control for confounding, when estimating exposure effect. Previous studies have shown that conditioning on the propensity score results in biased estimation of conditional odds ratio and hazard ratio. However, research is lacking on the performance of propensity score methods for covariate adjustment when estimating the area under the ROC curve (AUC). In this paper, AUC is proposed as measure of effect when outcomes are continuous. The AUC is interpreted as the probability that a randomly selected nonexposed subject has a better response than a randomly selected exposed subject. A series of simulations has been conducted to examine the performance of propensity score methods when association between exposure and outcomes is quantified by AUC; this includes determining the optimal choice of variables for the propensity score models. Additionally, the propensity score approach is compared with that of the conventional regression approach to adjust for covariates with the AUC. The choice of the best estimator depends on bias, relative bias, and root mean squared error. Finally, an example looking at the relationship of depression/anxiety and pain intensity in people with sickle cell disease is used to illustrate the estimation of the adjusted AUC using the proposed approaches.     

Model averaging procedure for varying-coefficient partially linear models with missing responses

This paper is concerned with model averaging procedure for varying-coefficient partially linear models with missing responses. The profile least-squares estimation process and inverse probability weighted method are employed to estimate regression coefficients of the partially restricted models, in which the propensity score is estimated by the covariate balancing propensity score method. The estimators of the linear parameters are shown to be asymptotically normal. Then we develop the focused information criterion, formulate the frequentist model averaging estimators and construct the corresponding confidence intervals. Some simulation studies are conducted to examine the finite sample performance of the proposed methods. We find that the covariate balancing propensity score improves the performance of the inverse probability weighted estimator. We also demonstrate the superiority of the proposed model averaging estimators over those of existing strategies in terms of mean squared error and coverage probability. Finally, our approach is further applied to a real data example.     

TAR模型加权秩估计及其性质讨论

秩估计是上个世纪60年代逐渐兴起的一种非参数方法,由于它具有稳健性等特征,从而得到较为广泛的应用。本文主要讨论了TAR模型随机加权秩估计及其性质问题,证明了基于一般计分函数的线性秩统计量关于回归参数的渐近一致线性性。本文讨论的建立在计分规则基础上的秩估计方法,虽然以TAR模型为对象,但其基本原理同样可以应用到其他非线性模型的参数估计中。     

CBPS-based estimation for linear models with responses missing at random

In this article, based on the covariate balancing propensity score (CBPS), estimators for the regression coefficients and the population mean are obtained, when the responses of linear models are missing at random. It is proved that the proposed estimators are asymptotically normal. In simulation studies and real example, the proposed estimators show improved performance relative to usual augmented inverse probability weighted estimators.     

Bias From Censored Regressors

We study the bias that arises from using censored regressors in estimation of linear models. We present results on bias in ordinary least aquares (OLS) regression estimators with exogenous censoring and in instrumental variable (IV) estimators when the censored regressor is endogenous. Bound censoring such as top-coding results in expansion bias, or effects that are too large. Independent censoring results in bias that varies with the estimation method—attenuation bias in OLS estimators and expansion bias in IV estimators. Severe biases can result when there are several regressors and when a 0–1 variable is used in place of a continuous regressor.     

Doubly robust empirical likelihood inference in covariate-missing data problems

Missing covariate data occurs often in regression analysis. We study methods for estimating the regression coefficients in an assumed conditional mean function when some covariates are completely observed but other covariates are missing for some subjects. We adopt the semiparametric perspective of Robins et al. [Estimation of regression coefficients when some regressors are not always observed. J Amer Statist Assoc. 1994;89:846–866] on regression analyses with missing covariates, in which they pioneered the use of two working models, the working propensity score model and the working conditional score model. A recent approach to missing covariate data analysis is the empirical likelihood method of Qin et al. [Empirical likelihood in missing data problems. J Amer Statist Assoc. 2009;104:1492–1503], which effectively combines unbiased estimating equations. In this paper, we consider an alternative likelihood approach based on the full likelihood of the observed data. This full likelihood-based method enables us to generate estimators for the vector of the regression coefficients that are (a) asymptotically equivalent to those of Qin et al. [Empirical likelihood in missing data problems. J Amer Statist Assoc. 2009;104:1492–1503] when the working propensity score model is correctly specified, and (b) doubly robust, like the augmented inverse probability weighting (AIPW) estimators of Robins et al. [Estimation of regression coefficients when some regressors are not always observed. J Am Statist Assoc. 1994;89:846–866]. Thus, the proposed full likelihood-based estimators improve on the efficiency of the AIPW estimators when the working propensity score model is correct but the working conditional score model is possibly incorrect, and also improve on the empirical likelihood estimators of Qin, Zhang and Leung [Empirical likelihood in missing data problems. J Amer Statist Assoc. 2009;104:1492–1503] when the reverse is true, that is, the working conditional score model is correct but the working propensity score model is possibly incorrect. In addition, we consider a regression method for estimation of the regression coefficients when the working conditional score model is correctly specified; the asymptotic variance of the resulting estimator is no greater than the semiparametric variance bound characterized by the theory of Robins et al. [Estimation of regression coefficients when some regressors are not always observed. J Amer Statist Assoc. 1994;89:846–866]. Finally, we compare the finite-sample performance of various estimators in a simulation study.     

Efficient Estimation and Robust Inference of Linear Regression Models in the Presence of Heteroscedastic Errors and High Leverage Points

It is common for linear regression models that the error variances are not the same for all observations and there are some high leverage data points. In such situations, the available literature advocates the use of heteroscedasticity consistent covariance matrix estimators (HCCME) for the testing of regression coefficients. Primarily, such estimators are based on the residuals derived from the ordinary least squares (OLS) estimator that itself can be seriously inefficient in the presence of heteroscedasticity. To get efficient estimation, many efficient estimators, namely the adaptive estimators are available but their performance has not been evaluated yet when the problem of heteroscedasticity is accompanied with the presence of high leverage data. In this article, the presence of high leverage data is taken into account to evaluate the performance of the adaptive estimator in terms of efficiency. Furthermore, our numerical work also evaluates the performance of the robust standard errors based on this efficient estimator in terms of interval estimation and null rejection rate (NRR).     

A Jackknife Method for Estimation of Variance Components

This paper concerns a method of estimation of variance components in a random effect linear model. It is mainly a resampling method and relies on the Jackknife principle. The derived estimators are presented as least squares estimators in an appropriate linear model, and one of them appears as a MINQUE (Minimum Norm Quadratic Unbiased Estimation) estimator. Our resampling method is illustrated by an example given by C. R. Rao [7] and some optimal properties of our estimator are derived for this example. In the last part, this method is used to derive an estimation of variance components in a random effect linear model when one of the components is assumed to be known.     

Estimation of average treatment effects based on parametric propensity score model

In this paper, the estimation of average treatment effects is examined given that the propensity score is of a parametric form with some unknown parameters. Under the assumption that the treatment is ignorable given some observed characteristics, the MLEs for those unknown parameters in the probability assignment model have been achieved firstly and then three estimators have been defined by the inverse probability weighted, regression and imputation methods, respectively. All the estimators are shown asymptotically normal and more importantly, the substantial efficiency gains of the first two estimates have been obtained theoretically compared with the existing estimators in Hahn (1998) and Hirano et al. (2003), i.e., the inverse weighted probability estimator and the regression estimator have smaller asymptotic variances. Our simulation analysis verifies the theoretical results in terms of biases, SEs and MSEs.     

A robust estimation method for the linear regression model parameters with correlated error terms and outliers

Independence of error terms in a linear regression model, often not established. So a linear regression model with correlated error terms appears in many applications. According to the earlier studies, this kind of error terms, basically can affect the robustness of the linear regression model analysis. It is also shown that the robustness of the parameters estimators of a linear regression model can stay using the M-estimator. But considering that, it acquires this feature as the result of establishment of its efficiency. Whereas, it has been shown that the minimum Matusita distance estimators, has both features robustness and efficiency at the same time. On the other hand, because the Cochrane and Orcutt adjusted least squares estimators are not affected by the dependence of the error terms, so they are efficient estimators. Here we are using of a non-parametric kernel density estimation method, to give a new method of obtaining the minimum Matusita distance estimators for the linear regression model with correlated error terms in the presence of outliers. Also, simulation and real data study both are done for the introduced estimation method. In each case, the proposed method represents lower biases and mean squared errors than the other two methods.KEYWORDS: Robust estimation method, minimum Matusita distance estimation method, non-parametric kernel density estimation method, correlated error terms, outliers     

Estimation on semi-functional linear errors-in-variables models

Abstract

Semi-functional linear regression models are important in practice. In this paper, their estimation is discussed when function-valued and real-valued random variables are all measured with additive error. By means of functional principal component analysis and kernel smoothing techniques, the estimators of the slope function and the non parametric component are obtained. To account for errors in variables, deconvolution is involved in the construction of a new class of kernel estimators. The convergence rates of the estimators of the unknown slope function and non parametric component are established under suitable norm and conditions. Simulation studies are conducted to illustrate the finite sample performance of our method.     

Optimal bounded influence regression and scale m-estimators in the context of experimental desing

The problem of simultaneous robust estimation of regression and scale parameters in the linear regression model is studied in the context of experimental design. Optimal M-estimates are given for a modified optimization problem of minimizing the asymptotic variances under bounded influence functions. This is done by reducing the multidimensional regression problem to the problem of estimating one-dimensional location and scale. For the location-scale case two subfamilies of optimal score functions are described in detail along with comparisons of the asymptotic variances and gross-error-sensitivities of the corresponding M-estimators. It turns out that, even for small gross-error-sensitivities, one of the subfamilies provides variances which are close to those of the nonrobust maximum likelihood estimators.     

Statistical inference in the partial linear models with the inverse gaussian kernel

Symmetric kernel smoothing is commonly used in estimating the nonparametric component in the partial linear regression models. In this article, we propose a new estimation method for the partial linear regression models using the inverse Gaussian kernel when the explanatory variable of the nonparametric component is non-negatively supported. As an asymmetric kernel function, the inverse Gaussian kernel is also supported on the non-negative half line. The asymptotic properties, including the asymptotic normality, uniform almost sure convergence, and the iterated logarithm laws, of the proposed estimators are thoroughly discussed for both homoscedastic and heteroscedastic cases. The simulation study is conducted to evaluate the finite sample performance of the proposed estimators.     

Single-index modelling of conditional probabilities in two-way contingency tables

When analysing a contingency table, it is often worth relating the probabilities that a given individual falls into different cells from a set of predictors. These conditional probabilities are usually estimated using appropriate regression techniques. In particular, in this paper, a semiparametric model is developed. Essentially, it is only assumed that the effect of the vector of covariates on the probabilities can entirely be captured by a single index, which is a linear combination of the initial covariates. The estimation is then twofold: the coefficients of the linear combination and the functions linking this index to the related conditional probabilities have to be estimated. Inspired by the estimation procedures already proposed in the literature for single-index regression models, four estimators of the index coefficients are proposed and compared, from a theoretical point-of-view, but also practically, with the aid of simulations. Estimation of the link functions is also addressed.     

基于半参数方法的模型辅助抽样估计研究

在总结现有模型辅助估计方法的基础上,本文通过构造一种半参数超总体模型,同时结合广义差分估计思想提出一种新型的模型辅助估计量。该估计量比传统的非参数和半参数回归估计利用更少、更易得到的辅助信息,即只需利用和广义回归估计相同的辅助信息,但一般会比广义回归估计拥有更高的估计精度。理论证明了该估计量是渐近设计无偏和设计一致的,其渐近设计均方误差为广义差分估计量的方差。模拟结果显示：其至少与广义回归估计一样好;对于线性程度越低的超总体模型,其估计精度比广义回归估计有越明显的提高;就本文模拟而言,光滑参数在0.04～0.12间适当取值时其会取到相对较好的估计效果。     

Variance Reduction in Smoothing Splines

Abstract.  We develop a variance reduction method for smoothing splines. For a given point of estimation, we define a variance-reduced spline estimate as a linear combination of classical spline estimates at three nearby points. We first develop a variance reduction method for spline estimators in univariate regression models. We then develop an analogous variance reduction method for spline estimators in clustered/longitudinal models. Simulation studies are performed which demonstrate the efficacy of our variance reduction methods in finite sample settings. Finally, a real data analysis with the motorcycle data set is performed. Here we consider variance estimation and generate 95% pointwise confidence intervals for the unknown regression function.     

Weighted quantile regression and testing for varying-coefficient models with randomly truncated data

This paper develops a varying-coefficient approach to the estimation and testing of regression quantiles under randomly truncated data. In order to handle the truncated data, the random weights are introduced and the weighted quantile regression (WQR) estimators for nonparametric functions are proposed. To achieve nice efficiency properties, we further develop a weighted composite quantile regression (WCQR) estimation method for nonparametric functions in varying-coefficient models. The asymptotic properties both for the proposed WQR and WCQR estimators are established. In addition, we propose a novel bootstrap-based test procedure to test whether the nonparametric functions in varying-coefficient quantile models can be specified by some function forms. The performance of the proposed estimators and test procedure are investigated through simulation studies and a real data example.     

Asymptotic approximations to the gain of the 2shi over stein estimators in linear regression models when the disturbances are small

The paper considers a new family of explicit or fully operational two-stage Stein or hierarchial information (2SHI) estimators for linear regression models, and provides an expression for the difference between the risks of these estimators and the usual Stein-rule estimator when the variance of the disturbance is small. The condition under which the 2SHI estimators have smaller average MSE than the Stein-rule estimator is also given.     

Penalized calibration in survey sampling: Design-based estimation assisted by mixed models

Calibration techniques in survey sampling, such as generalized regression estimation (GREG), were formalized in the 1990s to produce efficient estimators of linear combinations of study variables, such as totals or means. They implicitly lie on the assumption of a linear regression model between the variable of interest and some auxiliary variables in order to yield estimates with lower variance if the model is true and remaining approximately design-unbiased even if the model does not hold. We propose a new class of model-assisted estimators obtained by releasing a few calibration constraints and replacing them with a penalty term. This penalization is added to the distance criterion to minimize. By introducing the concept of penalized calibration, combining usual calibration and this ‘relaxed’ calibration, we are able to adjust the weight given to the available auxiliary information. We obtain a more flexible estimation procedure giving better estimates particularly when the auxiliary information is overly abundant or not fully appropriate to be completely used. Such an approach can also be seen as a design-based alternative to the estimation procedures based on the more general class of mixed models, presenting new prospects in some scopes of application such as inference on small domains.