Bandwidth selection for local linear regression smoothers

Summary. The paper presents a general strategy for selecting the bandwidth of nonparametric regression estimators and specializes it to local linear regression smoothers. The procedure requires the sample to be divided into a training sample and a testing sample. Using the training sample we first compute a family of regression smoothers indexed by their bandwidths. Next we select the bandwidth by minimizing the empirical quadratic prediction error on the testing sample. The resulting bandwidth satisfies a finite sample oracle inequality which holds for all bounded regression functions. This permits asymptotically optimal estimation for nearly any regression function. The practical performance of the method is illustrated by a simulation study which shows good finite sample behaviour of our method compared with other bandwidth selection procedures.     

Convergence rates for uniform confidence intervals based on local polynomial regression estimators

We investigate the convergence rates of uniform bias-corrected confidence intervals for a smooth curve using local polynomial regression for both the interior and boundary region. We discuss the cases when the degree of the polynomial is odd and even. The uniform confidence intervals are based on the volume-of-tube formula modified for biased estimators. We empirically show that the proposed uniform confidence intervals attain, at least approximately, nominal coverage. Finally, we investigate the performance of the volume-of-tube based confidence intervals for independent non-Gaussian errors.     

Local linear kernel estimation for discontinuous nonparametric regression functions

In this paper, we consider using a local linear (LL) smoothing method to estimate a class of discontinuous regression functions. We establish the asymptotic normality of the integrated square error (ISE) of a LL-type estimator and show that the ISE has an asymptotic rate of convergence as good as for smooth functions, and the asymptotic rate of convergence of the ISE of the LL estimator is better than that of the Nadaraya-Watson (NW) and the Gasser-Miiller (GM) estimators.     

Weighted local linear composite quantile estimation for the case of general error distributions

It is known that for nonparametric regression, local linear composite quantile regression (local linear CQR) is a more competitive technique than classical local linear regression since it can significantly improve estimation efficiency under a class of non-normal and symmetric error distributions. However, this method only applies to symmetric errors because, without symmetric condition, the estimation bias is non-negligible and therefore the resulting estimator is inconsistent. In this paper, we propose a weighted local linear CQR method for general error conditions. This method applies to both symmetric and asymmetric random errors. Because of the use of weights, the estimation bias is eliminated asymptotically and the asymptotic normality is established. Furthermore, by minimizing asymptotic variance, the optimal weights are computed and consequently the optimal estimate (the most efficient estimate) is obtained. By comparing relative efficiency theoretically or numerically, we can ensure that the new estimation outperforms the local linear CQR estimation. Finite sample behaviors conducted by simulation studies further illustrate the theoretical findings.     

More efficient local polynomial regression with random-effects panel data models

We propose a modification on the local polynomial estimation procedure to account for the “within-subject” correlation presented in panel data. The proposed procedure is rather simple to compute and has a closed-form expression. We study the asymptotic bias and variance of the proposed procedure and show that it outperforms the working independence estimator uniformly up to the first order. Simulation study shows that the gains in efficiency with the proposed method in the presence of “within-subject” correlation can be significant in small samples. For illustration purposes, the procedure is applied to explore the impact of market concentration on airfare.     

On variable bandwidth selection in local polynomial regression

The performances of data-driven bandwidth selection procedures in local polynomial regression are investigated by using asymptotic methods and simulation. The bandwidth selection procedures considered are based on minimizing 'prelimit' approximations to the (conditional) mean-squared error (MSE) when the MSE is considered as a function of the bandwidth h . We first consider approximations to the MSE that are based on Taylor expansions around h=0 of the bias part of the MSE. These approximations lead to estimators of the MSE that are accurate only for small bandwidths h . We also consider a bias estimator which instead of using small h approximations to bias naïvely estimates bias as the difference of two local polynomial estimators of different order and we show that this estimator performs well only for moderate to large h . We next define a hybrid bias estimator which equals the Taylor-expansion-based estimator for small h and the difference estimator for moderate to large h . We find that the MSE estimator based on this hybrid bias estimator leads to a bandwidth selection procedure with good asymptotic and, for our Monte Carlo examples, finite sample properties.     

Bootstrap inference in local polynomial regression of time series

In this paper we consider the inferential aspect of the nonparametric estimation of a conditional function , where X _t,m represents the vector containing the m conditioning lagged values of the series. Here is an arbitrary measurable function. The local polynomial estimator of order p is used for the estimation of the function g, and of its partial derivatives up to a total order p. We consider α-mixing processes, and we propose the use of a particular resampling method, the local polynomial bootstrap, for the approximation of the sampling distribution of the estimator. After analyzing the consistency of the proposed method, we present a simulation study which gives evidence of its finite sample behaviour.     

Best minimum bias linear estimators in gauss-markoff model

In this note we present a criterion for linear estimation which is similar to MV-MB-LE of Rao (1978) in Gauss-Markoff model (Y, XB, α²G). We call this criterion MMS-MB-LE (Minimum Mean Square Error-Minimum Bias-Linear Estimation)> Representations of solutions to such estimators similar to those of Rao (1978) are provided.     

Dimension reduction via local rank regression

Outer product of gradients (OPG) achieves dimension reduction via estimating the gradients of the regression function. In this paper, we propose two novel OPG estimators via local rank regression: the rank OPG estimator and the Walsh-average OPG estimator. Both proposals guard against a wide range of error distributions, and are safe alternatives to existing OPG estimators based on local linear regression or local L1 regression. The effectiveness of the new proposals are demonstrated via extensive numerical studies.     

Target estimation for the logistic regression model

The method of target estimation developed by Cabrera and Fernholz [(1999). Target estimation for bias and mean square error reduction. The Annals of Statistics, 27(3), 1080–1104.] to reduce bias and variance is applied to logistic regression models of several parameters. The expectation functions of the maximum likelihood estimators for the coefficients in the logistic regression models of one and two parameters are analyzed and simulations are given to show a reduction in both bias and variability after targeting the maximum likelihood estimators. In addition to bias and variance reduction, it is found that targeting can also correct the skewness of the original statistic. An example based on real data is given to show the advantage of using target estimators for obtaining better confidence intervals of the corresponding parameters. The notion of the target median is also presented with some applications to the logistic models.     

On a generalized stein estimator of regression coefficients

This paper studies a generalized Stein estimator of regression coefficients. The small disturbance approximations for the bias and mean square error matrix of the estimator are derived and a necessary and sufficient condition is obtained for the estimator to dominate the ordinary least squares estimator under the mean square error criterion.     

Bias Adjustment Minimizing the Asymptotic Mean Square Error

A method of bias adjustment which minimizes the asymptotic mean square error is presented for an estimator typically given by maximum likelihood. Generally, this adjustment includes unknown population values. However, in some examples, the adjustment can be done without population values. In the case of a logit, a reasonable fixed value for the adjustment is found, which gives the asymptotic mean square error smaller than those of the asymptotically unbiased estimator and the maximum likelihood estimator. The weighted-score method, which yields directly the estimator with the minimized asymptotic mean square error, is also given.     

High-order data sharpening with dependent errors for regression bias reduction

Abstract

In this paper, we show that Y can be introduced into data sharpening to produce non-parametric regression estimators that enjoy high orders of bias reduction. Compared with those in existing literature, the proposed data-sharpening estimator has advantages including simplicity of the estimators, good performance of expectation and variance, and mild assumptions. We generalize this estimator to dependent errors. Finally, we conduct a limited simulation to illustrate that the proposed estimator performs better than existing ones.     

Multivariate bandwidth selection for local linear regression

The existence and properties of optimal bandwidths for multivariate local linear regression are established, using either a scalar bandwidth for all regressors or a diagonal bandwidth vector that has a different bandwidth for each regressor. Both involve functionals of the derivatives of the unknown multivariate regression function. Estimating these functionals is difficult primarily because they contain multivariate derivatives. In this paper, an estimator of the multivariate second derivative is obtained via local cubic regression with most cross-terms left out. This estimator has the optimal rate of convergence but is simpler and uses much less computing time than the full local estimator. Using this as a pilot estimator, we obtain plug-in formulae for the optimal bandwidth, both scalar and diagonal, for multivariate local linear regression. As a simpler alternative, we also provide rule-of-thumb bandwidth selectors. All these bandwidths have satisfactory performance in our simulation study.     

Reducing errors-in-variables bias in linear regression using compact genetic algorithms

A new technique is devised to mitigate the errors-in-variables bias in linear regression. The procedure mimics a 2-stage least squares procedure where an auxiliary regression which generates a better behaved predictor variable is derived. The generated variable is then used as a substitute for the error-prone variable in the first-stage model. The performance of the algorithm is tested by simulation and regression analyses. Simulations suggest the algorithm efficiently captures the additive error term used to contaminate the artificial variables. Regressions provide further credit to the simulations as they clearly show that the compact genetic algorithm-based estimate of the true but unobserved regressor yields considerably better results. These conclusions are robust across different sample sizes and different variance structures imposed on both the measurement error and regression disturbances.     

Bias-corrected confidence bands in nonparametric regression

Bias-corrected confidence bands for general nonparametric regression models are considered. We use local polynomial fitting to construct the confidence bands and combine the cross-validation method and the plug-in method to select the bandwidths. Related asymptotic results are obtained. Our simulations show that confidence bands constructed by local polynomial fitting have much better coverage than those constructed by using the Nadaraya–Watson estimator. The results are also applicable to nonparametric autoregressive time series models.     

Mean square error behavior for prediction in linear regression models

For the problem of individual prediction in linear regression models, that is, estimation of a linear combination of regression coefficients, mean square error behavior of a general class of adaptive predictors is examined.     

On difference-based variance estimation in nonparametric regression when the covariate is high dimensional

Summary.  We consider the problem of estimating the noise variance in homoscedastic nonparametric regression models. For low dimensional covariates t  ∈  R ^d ,  d =1, 2, difference-based estimators have been investigated in a series of papers. For a given length of such an estimator, difference schemes which minimize the asymptotic mean-squared error can be computed for d =1 and d =2. However, from numerical studies it is known that for finite sample sizes the performance of these estimators may be deficient owing to a large finite sample bias. We provide theoretical support for these findings. In particular, we show that with increasing dimension d this becomes more drastic. If d 4, these estimators even fail to be consistent. A different class of estimators is discussed which allow better control of the bias and remain consistent when d 4. These estimators are compared numerically with kernel-type estimators (which are asymptotically efficient), and some guidance is given about when their use becomes necessary.