Variance estimation in nonparametric regression with jump discontinuities

Variance estimation is an important topic in nonparametric regression. In this paper, we propose a pairwise regression method for estimating the residual variance. Specifically, we regress the squared difference between observations on the squared distance between design points, and then estimate the residual variance as the intercept. Unlike most existing difference-based estimators that require a smooth regression function, our method applies to regression models with jump discontinuities. Our method also applies to the situations where the design points are unequally spaced. Finally, we conduct extensive simulation studies to evaluate the finite-sample performance of the proposed method and compare it with some existing competitors.     

Blind nonparametric regression

One must sometimes follow the evolution of several individuals which cannot be distinguished. The author proposes a graphical estimator of individual evolution that can be used in such cases. She shows that this estimator is consistent and asymptotically normal.     

The exact density of nonparametric regression estimators: fixed design case

This paper studies the exact density of a general nonparametric regression estimator when the errors are non-normal. The fixed design case is considered. The density function is derived by an application of the technique of Davis (1976)     

A Semi-parametric Regression Model with Errors in Variables

Abstract.  In this paper, we consider a partial linear regression model with measurement errors in possibly all the variables. We use a method of moments and deconvolution to construct a new class of parametric estimators together with a non-parametric kernel estimator. Strong convergence, optimal rate of weak convergence and asymptotic normality of the estimators are investigated.     

Plug-in bandwidth choice for estimation of nonparametric part in partial linear regression models with strong mixing errors

Consider a regression model where the regression function is the sum of a linear and a nonparametric component. Assuming that the errors of the model follow a stationary strong mixing process with mean zero, the problem of bandwidth selection for a kernel estimator of the nonparametric component is addressed here. We obtain an asymptotic expression for an optimal band-width and we propose to use a plug-in methodology in order to estimate this bandwidth through preliminary estimates of the unknown quantities. Asymptotic optimality for the plug-in bandwidth is established.     

Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion

Many different methods have been proposed to construct nonparametric estimates of a smooth regression function, including local polynomial, (convolution) kernel and smoothing spline estimators. Each of these estimators uses a smoothing parameter to control the amount of smoothing performed on a given data set. In this paper an improved version of a criterion based on the Akaike information criterion (AIC), termed AIC_C, is derived and examined as a way to choose the smoothing parameter. Unlike plug-in methods, AIC_C can be used to choose smoothing parameters for any linear smoother, including local quadratic and smoothing spline estimators. The use of AIC_C avoids the large variability and tendency to undersmooth (compared with the actual minimizer of average squared error) seen when other 'classical' approaches (such as generalized cross-validation (GCV) or the AIC) are used to choose the smoothing parameter. Monte Carlo simulations demonstrate that the AIC_C-based smoothing parameter is competitive with a plug-in method (assuming that one exists) when the plug-in method works well but also performs well when the plug-in approach fails or is unavailable.     

Testing discontinuities in nonparametric regression

In nonparametric regression, it is often needed to detect whether there are jump discontinuities in the mean function. In this paper, we revisit the difference-based method in [13] and propose to further improve it. To achieve the goal, we first reveal that their method is less efficient due to the inappropriate choice of the response variable in their linear regression model. We then propose a new regression model for estimating the residual variance and the total amount of discontinuities simultaneously. In both theory and simulation, we show that the proposed variance estimator has a smaller mean-squared error compared to the existing estimator, whereas the estimation efficiency for the total amount of discontinuities remains unchanged. Finally, we construct a new test procedure for detection of discontinuities using the proposed method; and via simulation studies, we demonstrate that our new test procedure outperforms the existing one in most settings.     

Nonparametric kernel estimation of the impact of tax policy on the demand for private health insurance in Australia

This paper is motivated by our attempt to answer a policy question: how is private health insurance take‐up in Australia affected by the income threshold at which the Medicare Levy Surcharge (MLS) kicks in? We propose a new difference deconvolution kernel estimator for the location and size of regression discontinuities. We also propose a bootstrapping procedure for estimating the confidence interval for the estimated discontinuity. Performance of the estimator is evaluated by Monte Carlo simulations before it is applied to estimating the effect of the income threshold of MLS on the take‐up of private health insurance in Australia, using contaminated data.     

Change points in nonparametric regresion functions

Estimators of location and size of jumps or discontinuities in a regression function and/or its derivatives are proposed. The estimators are based on the analysis of residuals obtained from the locally weighted least squares regression. The proposed estimators adapt to both fixed and random designs. The asymptotic properties of the estimators are investigated. The method is illustrated through simulation studies.     

Switching nonparametric regression models

We propose a methodology to analyse data arising from a curve that, over its domain, switches among J states. We consider a sequence of response variables, where each response y depends on a covariate x according to an unobserved state z. The states form a stochastic process and their possible values are j=1,?…?, J. If z equals j the expected response of y is one of J unknown smooth functions evaluated at x. We call this model a switching nonparametric regression model. We develop an Expectation–Maximisation algorithm to estimate the parameters of the latent state process and the functions corresponding to the J states. We also obtain standard errors for the parameter estimates of the state process. We conduct simulation studies to analyse the frequentist properties of our estimates. We also apply the proposed methodology to the well-known motorcycle dataset treating the data as coming from more than one simulated accident run with unobserved run labels.     

Boundary bias correction in nonparametric density estimation

An approach for removing boundary bias in nonparametric density esti-mation is considered. The technique is based on suitable finite-dimensional projections in Hilbert space. Applications to boundary bias removal with kernel and trigonometric series estimators are presented.     

Local linear kernel estimation for discontinuous nonparametric regression functions

In this paper, we consider using a local linear (LL) smoothing method to estimate a class of discontinuous regression functions. We establish the asymptotic normality of the integrated square error (ISE) of a LL-type estimator and show that the ISE has an asymptotic rate of convergence as good as for smooth functions, and the asymptotic rate of convergence of the ISE of the LL estimator is better than that of the Nadaraya-Watson (NW) and the Gasser-Miiller (GM) estimators.     

Difference-based variance estimator for nonparametric regression in complex surveys

A difference-based variance estimator is proposed for nonparametric regression in complex surveys. By using a combined inference framework, the estimator is shown to be asymptotically normal and to converge to the true variance at a parametric rate. Simulation studies show that the proposed variance estimator works well for complex survey data and also reveals some finite sample properties of the estimator.     

Nonparametric Kernel Methods with Errors‐in‐Variables: Constructing Estimators,Computing them,and Avoiding Common Mistakes

Estimating a curve nonparametrically from data measured with error is a difficult problem that has been studied by many authors. Constructing a consistent estimator in this context can sometimes be quite challenging, and in this paper we review some of the tools that have been developed in the literature for kernel‐based approaches, founded on the Fourier transform and a more general unbiased score technique. We use those tools to rederive some of the existing nonparametric density and regression estimators for data contaminated by classical or Berkson errors, and discuss how to compute these estimators in practice. We also review some mistakes made by those working in the area, and highlight a number of problems with an existing R package decon .     

A new estimator combining the ridge regression and the restricted least squares methods of estimation

It is well-known in the literature on multicollinearity that one of the major consequences of multicollinearity on the ordinary least squares estimator is that the estimator produces large sampling variances, which in turn might inappropriately lead to exclusion of otherwise significant coefficients from the model. To circumvent this problem, two accepted estimation procedures which are often suggested are the restricted least squares method and the ridge regression method. While the former leads to a reduction in the sampling variance of the estimator, the later ensures a smaller mean square error value for the estimator. In this paper we have proposed a new estimator which is based on a criterion that combines the ideas underlying these two estimators. The standard properties of this new estimator have been studied in the paper. It has also been shown that this estimator is superior to both the restricted least squares as well as the ordinary ridge regression estimators by the criterion of mean sauare error of the estimator of the regression coefficients when the restrictions are indeed correct. The conditions for superiority of this estimator over the other two have also been derived for the situation when the restrictions are not correct.     

Testing heteroscedasticity in nonparametric regression

The importance of being able to detect heteroscedasticity in regression is widely recognized because efficient inference for the regression function requires that heteroscedasticity is taken into account. In this paper a simple consistent test for heteroscedasticity is proposed in a nonparametric regression set-up. The test is based on an estimator for the best L ²-approximation of the variance function by a constant. Under mild assumptions asymptotic normality of the corresponding test statistic is established even under arbitrary fixed alternatives. Confidence intervals are obtained for a corresponding measure of heteroscedasticity. The finite sample performance and robustness of these procedures are investigated in a simulation study and Box-type corrections are suggested for small sample sizes.     

Nonparametric density estimation from data with a mixture of Berkson and classical errors

The author considers density estimation from contaminated data where the measurement errors come from two very different sources. A first error, of Berkson type, is incurred before the experiment: the variable X of interest is unobservable and only a surrogate can be measured. A second error, of classical type, is incurred after the experiment: the surrogate can only be observed with measurement error. The author develops two nonparametric estimators of the density of X, valid whenever Berkson, classical or a mixture of both errors are present. Rates of convergence of the estimators are derived and a fully data‐driven procedure is proposed. Finite sample performance is investigated via simulations and on a real data example.     

A plug-in technique in nonparametric regression with dependence

The problem addressed is that of smoothing parameter selection in kernel nonparametric regression in the fixed design regression model with dependent noise. An asymptotic expression of the optimum bandwidth parameter has been obtained in recent studies, where this takes the form h = C ₀ n ^?1/5. This paper proposes to use a plug-in methodology, in order to obtain an optimum estimation of the bandwidth parameter, through preliminary estimation of the unknown value of C ₀.     

Error variance estimation via least squares for small sample nonparametric regression

In this paper we explore statistical properties of some difference-based approaches to estimate an error variance for small sample based on nonparametric regression which satisfies Lipschitz condition. Our study is motivated by Tong and Wang (2005), who estimated error variance using a least squares approach. They considered the error variance as the intercept in a simple linear regression which was obtained from the expectation of their lag-k Rice estimator. Their variance estimators are highly dependent on the setting of a regressor and weight of their simple linear regression. Although this regressor and weight can be varied based on the characteristic of an unknown nonparametric mean function, Tong and Wang (2005) have used a fixed regressor and weight in a large sample and gave no indication of how to determine the regressor and the weight. In this paper, we propose a new approach via local quadratic approximation to determine this regressor and weight. Using our proposed regressor and weight, we estimate the error variance as the intercept of simple linear regression using both ordinary least squares and weighted least squares. Our approach applies to both small and large samples, while most existing difference-based methods are appropriate solely for large samples. We compare the performance of our approach with other existing approaches using extensive simulation study. The advantage of our approach is demonstrated using a real data set.