Jackknife Estimation for Smooth Functions of the Parametric Component in Partially Linear Regression Models

Abstract

It is known that due to the existence of the nonparametric component, the usual estimators for the parametric component or its function in partially linear regression models are biased. Sometimes this bias is severe. To reduce the bias, we propose two jackknife estimators and compare them with the naive estimator. All three estimators are shown to be asymptotically equivalent and asymptotically normally distributed under some regularity conditions. However, through simulation we demonstrate that the jackknife estimators perform better than the naive estimator in terms of bias when the sample size is small to moderate. To make our results more useful, we also construct consistent estimators of the asymptotic variance, which are robust against heterogeneity of the error variances.     

Efficient Estimation in Heteroscedastic Partially Linear Varying Coefficient Models

This article considers statistical inference for the heteroscedastic partially linear varying coefficient models. We construct an efficient estimator for the parametric component by applying the weighted profile least-squares approach, and show that it is semiparametrically efficient in the sense that the inverse of the asymptotic variance of the estimator reaches the semiparametric efficiency bound. Simulation studies are conducted to illustrate the performance of the proposed method.     

Estimating the error variance in nonparametric regression by a covariate-matched u-statistic

For nonparametric regression models with fixed and random design, two classes of estimators for the error variance have been introduced: second sample moments based on residuals from a nonparametric fit, and difference-based estimators. The former are asymptotically optimal but require estimating the regression function; the latter are simple but have larger asymptotic variance. For nonparametric regression models with random covariates, we introduce a class of estimators for the error variance that are related to difference-based estimators: covariate-matched U-statistics. We give conditions on the random weights involved that lead to asymptotically optimal estimators of the error variance. Our explicit construction of the weights uses a kernel estimator for the covariate density.     

Estimation of regression parameters in generalized linear models for cluster correlated data with measurement error

Liang and Zeger (1986) introduced a class of estimating equations that gives consistent estimates of regression parameters and of their asymptotic variances in the class of generalized linear models for cluster correlated data. When the independent variables or covariates in such models are subject to measurement errors, the parameter estimates obtained from these estimating equations are no longer consistent. To correct for the effect of measurement errors, an estimator with smaller asymptotic bias is constructed along the lines of Stefanski (1985), assuming that the measurement error variance is either known or estimable. The asymptotic distribution of the bias-corrected estimator and a consistent estimator of its asymptotic variance are also given. The special case of a binary logistic regression model is studied in detail. For this case, methods based on conditional scores and quasilikelihood are also extended to cluster correlated data. Results of a small simulation study on the performance of the proposed estimators and associated tests of hypotheses are reported.     

Asymptotic relative efficiency of wald tests in measurement error models

In this paper, asymptotic relative efficiency (ARE) of Wald tests for the Tweedie class of models with log-linear mean, is considered when the aux¬iliary variable is measured with error. Wald test statistics based on the naive maximum likelihood estimator and on a consistent estimator which is obtained by using Nakarnura's (1990) corrected score function approach are defined. As shown analytically, the Wald statistics based on the naive and corrected score function estimators are asymptotically equivalents in terms of ARE. On the other hand, the asymptotic relative efficiency of the naive and corrected Wald statistic with respect to the Wald statistic based on the true covariate equals to the square of the correlation between the unobserved and the observed co-variate. A small scale numerical Monte Carlo study and an example illustrate the small sample size situation.     

Efficient Rank Regression with Wavelet Estimated Scores

We provide an estimate of the score function for rank regression using compactly supported wavelets. This estimate is then used to find a rank-based asymptotically efficient estimator for the slope parameter of a linear model. We also provide a consistent estimator of the asymptotic variance of the rank estimator. For related mixed models, the asymptotic relative efficiency is also discussed     

Adjusting regression attenuation in the Cox proportional hazards model

In the measurement error Cox proportional hazards model, the naive maximum partial likelihood estimator (MPLE) is asymptotically biased. In this paper, we give the formula of the asymptotic bias for the additive measurement error Cox model. By adjusting for this error, we derive an adjusted MPLE that is less biased. The bias can be further reduced by adjusting for the estimator second and even third time. This estimator has the advantage of being easy to apply. The performance of the proposed estimator is evaluated through a simulation study.     

Local regression for vector responses

We explore a class of vector smoothers based on local polynomial regression for fitting nonparametric regression models which have a vector response. The asymptotic bias and variance for the class of estimators are derived for two different ways of representing the variance matrices within both a seemingly unrelated regression and a vector measurement error framework. We show that the asymptotic behaviour of the estimators is different in these four cases. In addition, the placement of the kernel weights in weighted least squares estimators is very important in the seeming unrelated regressions problem (to ensure that the estimator is asymptotically unbiased) but not in the vector measurement error model. It is shown that the component estimators are asymptotically uncorrelated in the seemingly unrelated regressions model but asymptotically correlated in the vector measurement error model. These new and interesting results extend our understanding of the problem of smoothing dependent data.     

Outlier detection using difference-based variance estimators in multiple regression

In this article, we propose an outlier detection approach in a multiple regression model using the properties of a difference-based variance estimator. This type of a difference-based variance estimator was originally used to estimate error variance in a non parametric regression model without estimating a non parametric function. This article first employed a difference-based error variance estimator to study the outlier detection problem in a multiple regression model. Our approach uses the leave-one-out type method based on difference-based error variance. The existing outlier detection approaches using the leave-one-out approach are highly affected by other outliers, while ours is not because our approach does not use the regression coefficient estimator. We compared our approach with several existing methods using a simulation study, suggesting the outperformance of our approach. The advantages of our approach are demonstrated using a real data application. Our approach can be extended to the non parametric regression model for outlier detection.     

Bias corrected estimation for generalized probit regression with covariate measurement error and censored responses

In this paper, we propose a bias corrected estimate of the regression coefficient for the generalized probit regression model when the covariates are subject to measurement error and the responses are subject to interval censoring. The main improvement of our method is that it reduces most of the bias that the naive estimates have. The great advantage of our method is that it is baseline and censoring distribution free, in a sense that the investigator does not need to calculate the baseline or the censoring distribution to obtain the estimator of the regression coefficient, an important property of Cox regression model. A sandwich estimator for the variance is also proposed. Our procedure can be generalized to general measurement error distribution as long as the first four moments of the measurement error are known. The results of extensive simulations show that our approach is very effective in eliminating the bias when the measurement error is not too large relative to the error term of the regression model.     

Estimated Generalized Least Squares for Random Coefficient Regression Models

ABSTRACT. This paper considers a general class of random coefficient regression (RCR) models to represent pooled cross-sectional and time series data. A new method is given to estimate the covariance matrix of the error component in these RCR models. Also, the asymptotic and small sample properties of the estimated generalized least squares estimator of the regression coefficient vector are established. Procedures for testing a linear restriction on the mean vector of the random coefficients are derived. Finally, a test for non-randomness in the RCR model is devised, and the asymptotic distribution of the test statistic is obtained.     

On the use of the Stein variance estimator in the double <Emphasis Type="Italic">k</Emphasis>-class estimator when each individual regression coefficient is estimated

In this paper we consider the double k-class estimator which incorporates the Stein variance estimator. This estimator is called the SVKK estimator. We derive the explicit formula for the mean squared error (MSE) of the SVKK estimator for each individual regression coefficient. It is shown analytically that the MSE performance of the Stein-rule estimator for each individual regression coefficient can be improved by utilizing the Stein variance estimator. Also, MSE’s of several estimators included in a family of the SVKK estimators are compared by numerical evaluations.     

Statistical inference for restricted partially linear varying coefficient errors-in-variables models

As a useful extension of partially linear models and varying coefficient models, the partially linear varying coefficient model is useful in statistical modelling. This paper considers statistical inference for the semiparametric model when the covariates in the linear part are measured with additive error and some additional linear restrictions on the parametric component are available. We propose a restricted modified profile least-squares estimator for the parametric component, and prove the asymptotic normality of the proposed estimator. To test hypotheses on the parametric component, we propose a test statistic based on the difference between the corrected residual sums of squares under the null and alterative hypotheses, and show that its limiting distribution is a weighted sum of independent chi-square distributions. We also develop an adjusted test statistic, which has an asymptotically standard chi-squared distribution. Some simulation studies are conducted to illustrate our approaches.     

Estimation of the error distribution in a varying coefficient regression model

This paper deals with the estimation of the error distribution function in a varying coefficient regression model. We propose two estimators and study their asymptotic properties by obtaining uniform stochastic expansions. The first estimator is a residual-based empirical distribution function. We study this estimator when the varying coefficients are estimated by under-smoothed local quadratic smoothers. Our second estimator which exploits the fact that the error distribution has mean zero is a weighted residual-based empirical distribution whose weights are chosen to achieve the mean zero property using empirical likelihood methods. The second estimator improves on the first estimator. Bootstrap confidence bands based on the two estimators are also discussed.     

On the Estimation of the Probability Density by Trigonometric Series

We present in this article an estimator based on a new orthogonal trigonometric series. We give its statistical properties (bias, variance, mean square error, and mean integrated square error) and the asymptotic properties (convergence of variance, convergence of the mean square error, convergence of the mean integrated square error, uniform convergence in probability, and the rate of convergence of the mean integrated square error). The comparison by simulation on a test density between the estimator obtained from a new trigonometric series with Fejer estimator also based on orthogonal trigonometric series, shows that our estimator is more performant in the sense of the mean integrated square error.     

Variance estimation in the analysis of microarray data

Summary.  Microarrays are one of the most widely used high throughput technologies. One of the main problems in the area is that conventional estimates of the variances that are required in the t -statistic and other statistics are unreliable owing to the small number of replications. Various methods have been proposed in the literature to overcome this lack of degrees of freedom problem. In this context, it is commonly observed that the variance increases proportionally with the intensity level, which has led many researchers to assume that the variance is a function of the mean. Here we concentrate on estimation of the variance as a function of an unknown mean in two models: the constant coefficient of variation model and the quadratic variance–mean model. Because the means are unknown and estimated with few degrees of freedom, naive methods that use the sample mean in place of the true mean are generally biased because of the errors-in-variables phenomenon. We propose three methods for overcoming this bias. The first two are variations on the theme of the so-called heteroscedastic simulation–extrapolation estimator, modified to estimate the variance function consistently. The third class of estimators is entirely different, being based on semiparametric information calculations. Simulations show the power of our methods and their lack of bias compared with the naive method that ignores the measurement error. The methodology is illustrated by using microarray data from leukaemia patients.     

Asymptotics for penalised splines in generalised additive models

This paper discusses asymptotic theory for penalised spline estimators in generalised additive models. The purpose of this paper is to establish the asymptotic bias and variance as well as the asymptotic normality of the ridge-corrected penalised spline estimator. Furthermore, the asymptotics for the penalised quasi-likelihood fit in mixed models are also discussed.     

Large sample behavior of the Bernstein copula estimator

Bernstein polynomial estimators have been used as smooth estimators for density functions and distribution functions. The idea of using them for copula estimation has been given in Sancetta and Satchell (2004). In the present paper we study the asymptotic properties of this estimator: almost sure consistency rates and asymptotic normality. We also obtain explicit expressions for the asymptotic bias and asymptotic variance and show the improvement of the asymptotic mean squared error compared to that of the classical empirical copula estimator. A small simulation study illustrates this superior behavior in small samples.     

A modified estimator of error variance in a partly linear model

We consider estimation of the variance of the error in a partly linear model Our resulting estimator has smaller asymptotic variance than the estimators given in literature.