On variable bandwidth selection in local polynomial regression

The performances of data-driven bandwidth selection procedures in local polynomial regression are investigated by using asymptotic methods and simulation. The bandwidth selection procedures considered are based on minimizing 'prelimit' approximations to the (conditional) mean-squared error (MSE) when the MSE is considered as a function of the bandwidth h . We first consider approximations to the MSE that are based on Taylor expansions around h=0 of the bias part of the MSE. These approximations lead to estimators of the MSE that are accurate only for small bandwidths h . We also consider a bias estimator which instead of using small h approximations to bias naïvely estimates bias as the difference of two local polynomial estimators of different order and we show that this estimator performs well only for moderate to large h . We next define a hybrid bias estimator which equals the Taylor-expansion-based estimator for small h and the difference estimator for moderate to large h . We find that the MSE estimator based on this hybrid bias estimator leads to a bandwidth selection procedure with good asymptotic and, for our Monte Carlo examples, finite sample properties.     

Properties of the mixed regression estimator when disturbances are not necessarily normal

Small-disturbance approximations for the bias vector and mean squared error matrix of the mixed regression estimator for the coefficients in a linear regression model are derived and efficiency with respect to least squares estimator is examined.     

A Remedy for Kernel Estimation Under Random Design

Two common kernel-based methods for non-parametric regression estimation suffer from well-known drawbacks when the design is random. The Gasser-Müller estimator is inadmissible due to its high variance while the Nadaraya-Watson estimator has zero asymptotic efficiency because of poor bias behavior. Under asymptotic consideration, the local linear estimator avoids these two drawbacks of kernel estimators and achieves minimax optimality. However, when based on compact support kernels its finite sample behavior is disappointing because sudden kinks may show up in the estimate.

This paper proposes a modification of the kernel estimator, called the binned convolution estimator leading to a fast O(n) method. Provided the design density is continously differentiable and the conditional fourth moments exist the binned convolution estimator has asymptotic properties identical with those of the local linear estimator.     

Rao and Wu's re-scaling bootstrap modified to achieve extended coverages

Horvitz and Thompson's (HT) [1952. A generalization of sampling without replacement from a finite universe. J. Amer. Statist. Assoc. 47, 663–685] well-known unbiased estimator for a finite population total admits an unbiased estimator for its variance as given by [Yates and Grundy, 1953. Selection without replacement from within strata with probability proportional to size. J. Roy. Statist. Soc. B 15, 253–261], provided the parent sampling design involves a constant number of distinct units in every sample to be chosen. If the design, in addition, ensures uniform non-negativity of this variance estimator, Rao and Wu [1988. Resampling inference with complex survey data. J. Amer. Statist. Assoc. 83, 231–241] have given their re-scaling bootstrap technique to construct confidence interval and to estimate mean square error for non-linear functions of finite population totals of several real variables. Horvitz and Thompson's estimators (HTE) are used to estimate the finite population totals. Since they need to equate the bootstrap variance of the bootstrap estimator to the Yates and Grundy's estimator (YGE) for the variance of the HTE in case of a single variable, i.e., in the linear case the YG variance estimator is required to be positive for the sample usually drawn.     

Wild Bootstrapping in Finite Populations with Auxiliary Information

Consider a finite population u , which can be viewed as a realization of a super-population model. A simple ratio model (linear regression, without intercept) with heteroscedastic errors is supposed to have generated u . A random sample is drawn without replacement from u . In this set-up a two-stage wild bootstrap resampling scheme as well as several other useful forms of bootstrapping in finite populations will be considered. Some asymptotic results for various bootstrap approximations for normalized and Studentized versions of the well-known ratio and regression estimator are given. Bootstrap based confidence interval s for the population total and for the regression parameter of the underlying ratio model are also discussed     

On the simple linear regression model with correlated measurement errors

The simple linear regression model with measurement error has been subject to much research. In this work we will focus on this model when the error in the explanatory variable is correlated with the error in the regression equation. Specifically, we are interested in the comparison between the ordinary errors-in-variables estimator of the regression coefficient β$\beta$ and the estimator that takes account of the correlation between the errors. Based on large sample approximations, we compare the estimators and find that the estimator that takes account of the correlation should be preferred in most situations. We also compare the estimators in small sample situations. This is done by stochastic simulation. The results show that the estimators behave quite similarly in most of the simulated situations, but that the ordinary errors-in-variables estimator performs considerably worse than the estimator that takes account of the correlation for certain parameter combinations. In addition, we look briefly into the bias introduced by ignoring correlated errors when computing sample correlations, and in predictions.     

Conditional inference in finite population sampling under a calibration model

Several estimators, including the classical and the regression estimators of finite population mean, are compared, both theoretically and empirically, under a calibration model, where the dependent variable(y), and not the independent variable(x), can be observed for all units of the finite population. It is shown asymptotically that when conditioned on x, the bias of the classical estimator may be much smaller than that of the regression estimators; whereas when conditioned on y, the regression estimator may have much smaller conditional bias than the classical estimator. Since all the y's(not x's) can be observed, it seems appropriate to make comparison under the conditional distribution of each estimator with y fixed. In this case, the regression estimator has smaller variance, smaller conditional bias, and the conditional coverage probability closer to its nominal level     

An Improved Generalized Difference-Cum-Ratio-Type Estimator for the Population Variance in Two-Phase Sampling Using Two Auxiliary Variables

In this paper, an improved generalized difference-cum-ratio-type estimator for the finite population variance under two-phase sampling design is proposed. The expressions for bias and mean square error (MSE) are derived to first order of approximation. The proposed estimator is more efficient than the usual sample variance estimator, traditional ratio estimator, traditional regression estimator, chain ratio type and chain ratio-product-type estimators, and Jhajj and Walia (2011) estimator. Four datasets are also used to illustrate the performances of different estimators.     

On the restricted almost unbiased estimators in linear regression

In this paper, the restricted almost unbiased ridge regression estimator and restricted almost unbiased Liu estimator are introduced for the vector of parameters in a multiple linear regression model with linear restrictions. The bias, variance matrices and mean square error (MSE) of the proposed estimators are derived and compared. It is shown that the proposed estimators will have smaller quadratic bias but larger variance than the corresponding competitors in literatures. However, they will respectively outperform the latter according to the MSE criterion under certain conditions. Finally, a simulation study and a numerical example are given to illustrate some of the theoretical results.     

On bagging and nonlinear estimation

We propose an elementary model for the way in which stochastic perturbations of a statistical objective function, such as a negative log-likelihood, produce excessive nonlinear variation of the resulting estimator. Theory for the model is transparently simple, and is used to provide new insight into the main factors that affect performance of bagging. In particular, it is shown that if the perturbations are sufficiently symmetric then bagging will not significantly increase bias; and if the perturbations also offer opportunities for cancellation then bagging will reduce variance. For the first property it is sufficient that the third derivative of a perturbation vanish locally, and for the second, that second and fourth derivatives have opposite signs. Functions that satisfy these conditions resemble sinusoids. Therefore, our results imply that bagging will reduce the nonlinear variation, as measured by either variance or mean-squared error, produced in an estimator by sinusoid-like, stochastic perturbations of the objective function. Analysis of our simple model also suggests relationships between the results obtained using different with-replacement and without-replacement bagging schemes. We simulate regression trees in settings that are far more complex than those explicitly addressed by the model, and find that these relationships are generally borne out.     

Probability density estimation with data missing at random when covariables are present

This paper addresses the problem of the probability density estimation in the presence of covariates when data are missing at random (MAR). The inverse probability weighted method is used to define a nonparametric and a semiparametric weighted probability density estimators. A regression calibration technique is also used to define an imputed estimator. It is shown that all the estimators are asymptotically normal with the same asymptotic variance as that of the inverse probability weighted estimator with known selection probability function and weights. Also, we establish the mean squared error (MSE) bounds and obtain the MSE convergence rates. A simulation is carried out to assess the proposed estimators in terms of the bias and standard error.     

One-step M-estimators in the linear model,with dependent errors

We consider the problem of robust M-estimation of a vector of regression parameters, when the errors are dependent. We assume a weakly stationary, but otherwise quite general dependence structure. Our model allows for the representation of the correlations of any time series of finite length. We first construct initial estimates of the regression, scale, and autocorrelation parameters. The initial autocorrelation estimates are used to transform the model to one of approximate independence. In this transformed model, final one-step M-estimates are calculated. Under appropriate assumptions, the regression estimates so obtained are asymptotically normal, with a variance-covariance structure identical to that in the case in which the autocorrelations are known a priori. The results of a simulation study are given. Two versions of our estimator are compared with the L₁ -estimator and several Huber-type M-estimators. In terms of bias and mean squared error, the estimators are generally very close. In terms of the coverage probabilities of confidence intervals, our estimators appear to be quite superior to both the L₁-estimator and the other estimators. The simulations also indicate that the approach to normality is quite fast.     

Inferences from biased samples with a memory effect

Biased sampling occurs often in observational studies. With one biased sample, the problem of nonparametrically estimating both a target density function and a selection bias function is unidentifiable. This paper studies the nonparametric estimation problem when there are two biased samples that have some overlapping observations (i.e. recaptures) from a finite population. Since an intelligent subject sampled previously may experience a memory effect if sampled again, two general 2-stage models that incorporate both a selection bias and a possible memory effect are proposed. Nonparametric estimators of the target density, selection bias, and memory functions, as well as the population size are developed. Asymptotic properties of these estimators are studied and confidence bands for the selection function and memory function are provided. Our procedures are compared with those ignoring the memory effect or the selection bias in finite sample situations. A nonparametric model selection procedure is also given for choosing a model from the two 2-stage models and a mixture of these two models. Our procedures work well with or without a memory effect, and with or without a selection bias. The paper concludes with an application to a real survey data set.     

Estimation and inference of combining quantile and least-square regressions with missing data

In this paper, we consider how to incorporate quantile information to improve estimator efficiency for regression model with missing covariates. We combine the quantile information with least-squares normal equations and construct an unbiased estimating equations (EEs). The lack of smoothness of the objective EEs is overcome by replacing them with smooth approximations. The maximum smoothed empirical likelihood (MSEL) estimators are established based on inverse probability weighted (IPW) smoothed EEs and their asymptotic properties are studied under some regular conditions. Moreover, we develop two novel testing procedures for the underlying model. The finite-sample performance of the proposed methodology is examined by simulation studies. A real example is used to illustrate our methods.     

Guided Censored Regression

Parametrically guided non‐parametric regression is an appealing method that can reduce the bias of a non‐parametric regression function estimator without increasing the variance. In this paper, we adapt this method to the censored data case using an unbiased transformation of the data and a local linear fit. The asymptotic properties of the proposed estimator are established, and its performance is evaluated via finite sample simulations.     

First‐order bias correction for fractionally integrated time series

Most of the long memory estimators for stationary fractionally integrated time series models are known to experience non‐negligible bias in small and finite samples. Simple moment estimators are also vulnerable to such bias, but can easily be corrected. In this article, the authors propose bias reduction methods for a lag‐one sample autocorrelation‐based moment estimator. In order to reduce the bias of the moment estimator, the authors explicitly obtain the exact bias of lag‐one sample autocorrelation up to the order n⁻¹. An example where the exact first‐order bias can be noticeably more accurate than its asymptotic counterpart, even for large samples, is presented. The authors show via a simulation study that the proposed methods are promising and effective in reducing the bias of the moment estimator with minimal variance inflation. The proposed methods are applied to the northern hemisphere data. The Canadian Journal of Statistics 37: 476–493; 2009 © 2009 Statistical Society of Canada     

On regression method for estimating a population proportion

In this paper the regression method of estimation has been discussed for the estimation of proportion of the units possessing the given characteristic. An unbiased regression type estimator has also been discussed. An unbiased estimator is given for estimating the common MSE/variance of the estimators discussed. The results have also been extended to deal with the populations of a peculiar nature.     

Bernstein conditional density estimation with application to conditional distribution and regression functions

In this paper we propose a smooth nonparametric estimation for the conditional probability density function based on a Bernstein polynomial representation. Our estimator can be written as a finite mixture of beta densities with data-driven weights. Using the Bernstein estimator of the conditional density function, we derive new estimators for the distribution function and conditional mean. We establish the asymptotic properties of the proposed estimators, by proving their asymptotic normality and by providing their asymptotic bias and variance. Simulation results suggest that the proposed estimators can outperform the Nadaraya–Watson estimator and, in some specific setups, the local linear kernel estimators. Finally, we use our estimators for modeling the income in Italy, conditional on year from 1951 to 1998, and have another look at the well known Old Faithful Geyser data.     

On a generalized stein estimator of regression coefficients

This paper studies a generalized Stein estimator of regression coefficients. The small disturbance approximations for the bias and mean square error matrix of the estimator are derived and a necessary and sufficient condition is obtained for the estimator to dominate the ordinary least squares estimator under the mean square error criterion.     

Generalized empirical likelihood inference in partial linear regression model for longitudinal data

In this paper, empirical likelihood inference for longitudinal data within the framework of partial linear regression models are investigated. The proposed procedures take into consideration the correlation within groups without involving direct estimation of nuisance parameters in the correlation matrix. The empirical likelihood method is used to estimate the regression coefficients and the baseline function, and to construct confidence intervals. A nonparametric version of Wilk's theorem for the limiting distribution of the empirical likelihood ratio is derived. Compared with methods based on normal approximations, the empirical likelihood does not require consistent estimators for the asymptotic variance and bias. The finite sample behaviour of the proposed method is evaluated with simulation and illustrated with an AIDS clinical trial data set.