On two-step estimation for varying coefficient models

In this note we discuss two-step kernel estimation of varying coefficient regression models that have a common smoothing variable. The method allows one to use different bandwidths for different coefficient functions. We consider local polynomial fitting and present explicit formulas for the asymptotic biases and variances of the estimators.     

On the Local Polynomial Estimators of the Counting Process Intensity Function and its Derivatives

Abstract. We consider the properties of the local polynomial estimators of a counting process intensity function and its derivatives. By expressing the local polynomial estimators in a kernel smoothing form via effective kernels, we show that the bias and variance of the estimators at boundary points are of the same magnitude as at interior points and therefore the local polynomial estimators in the context of intensity estimation also enjoy the automatic boundary correction property as they do in other contexts such as regression. The asymptotically optimal bandwidths and optimal kernel functions are obtained through the asymptotic expressions of the mean square error of the estimators. For practical purpose, we suggest an effective and easy‐to‐calculate data‐driven bandwidth selector. Simulation studies are carried out to assess the performance of the local polynomial estimators and the proposed bandwidth selector. The estimators and the bandwidth selector are applied to estimate the rate of aftershocks of the Sichuan earthquake and the rate of the Personal Emergency Link calls in Hong Kong.     

Robust nonparametric estimation of the intensity function of point data

Intensity functions—which describe the spatial distribution of the occurrences of point processes—are useful for risk assessment. This paper deals with the robust nonparametric estimation of the intensity function of space–time data from events such as earthquakes. The basic approach consists of smoothing the frequency histograms with the local polynomial regression (LPR) estimator. This method allows for automatic boundary corrections, and its jump-preserving ability can be improved with robustness. We derive a robust local smoother from the weighted-average approach to M-estimation and we select its bandwidths with robust cross-validation (RCV). Further, we develop a robust recursive algorithm for sequential processing of the data binned in time. An extensive application to the Northern California earthquake catalog in the San Francisco, CA, area illustrates the method and proves its validity.     

Nonparametric estimation of a log-variance function in scale-space

In a nonparametric regression setting, we consider the kernel estimation of the logarithm of the error variance function, which might be assumed to be homogeneous or heterogeneous. The objective of the present study is to discover important features in the variation of the data at multiple locations and scales based on a nonparametric kernel smoothing technique. Traditional kernel approaches estimate the function by selecting an optimal bandwidth, but it often turns out to be unsatisfying in practice. In this paper, we develop a SiZer (SIgnificant ZERo crossings of derivatives) tool based on a scale-space approach that provides a more flexible way of finding meaningful features in the variation. The proposed approach utilizes local polynomial estimators of a log-variance function using a wide range of bandwidths. We derive the theoretical quantile of confidence intervals in SiZer inference and also study the asymptotic properties of the proposed approach in scale-space. A numerical study via simulated and real examples demonstrates the usefulness of the proposed SiZer tool.     

Local Adaptivity of Kernel Estimates with Plug-in Local Bandwidth Selectors

In non-parametric function estimation selection of a smoothing parameter is one of the most important issues. The performance of smoothing techniques depends highly on the choice of this parameter. Preferably the bandwidth should be determined via a data-driven procedure. In this paper we consider kernel estimators in a white noise model, and investigate whether locally adaptive plug-in bandwidths can achieve optimal global rates of convergence. We consider various classes of functions: Sobolev classes, bounded variation function classes, classes of convex functions and classes of monotone functions. We study the situations of pilot estimation with oversmoothing and without oversmoothing. Our main finding is that simple local plug-in bandwidth selectors can adapt to spatial inhomogeneity of the regression function as long as there are no local oscillations of high frequency. We establish the pointwise asymptotic distribution of the regression estimator with local plug-in bandwidth.     

Robust non-parametric smoothing of non-stationary time series

Motivated by the need of extracting local trends and low frequency components in non-stationary time series, this paper discusses methods of robust non-parametric smoothing. Basic approach is the combination of the parametric M-estimation with kernel and local polynomial regression methods. The result is an iterative estimator that retains a linear structure, but has kernel weights also in the direction of the prediction errors. The design of smoothing coefficients is carried out with robust cross-validation criteria and rules of thumb. The method works well both to remove the influence of patches of outliers and to detect the local breaks and persistent structural change in time series.     

Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion

Many different methods have been proposed to construct nonparametric estimates of a smooth regression function, including local polynomial, (convolution) kernel and smoothing spline estimators. Each of these estimators uses a smoothing parameter to control the amount of smoothing performed on a given data set. In this paper an improved version of a criterion based on the Akaike information criterion (AIC), termed AIC_C, is derived and examined as a way to choose the smoothing parameter. Unlike plug-in methods, AIC_C can be used to choose smoothing parameters for any linear smoother, including local quadratic and smoothing spline estimators. The use of AIC_C avoids the large variability and tendency to undersmooth (compared with the actual minimizer of average squared error) seen when other 'classical' approaches (such as generalized cross-validation (GCV) or the AIC) are used to choose the smoothing parameter. Monte Carlo simulations demonstrate that the AIC_C-based smoothing parameter is competitive with a plug-in method (assuming that one exists) when the plug-in method works well but also performs well when the plug-in approach fails or is unavailable.     

Local polynomial regression and simulation–extrapolation

Summary.  The paper introduces a new local polynomial estimator and develops supporting asymptotic theory for nonparametric regression in the presence of covariate measurement error. We address the measurement error with Cook and Stefanski's simulation–extrapolation (SIMEX) algorithm. Our method improves on previous local polynomial estimators for this problem by using a bandwidth selection procedure that addresses SIMEX's particular estimation method and considers higher degree local polynomial estimators. We illustrate the accuracy of our asymptotic expressions with a Monte Carlo study, compare our method with other estimators with a second set of Monte Carlo simulations and apply our method to a data set from nutritional epidemiology. SIMEX was originally developed for parametric models. Although SIMEX is, in principle, applicable to nonparametric models, a serious problem arises with SIMEX in nonparametric situations. The problem is that smoothing parameter selectors that are developed for data without measurement error are no longer appropriate and can result in considerable undersmoothing. We believe that this is the first paper to address this difficulty.     

Local CQR Smoothing: An Efficient and Safe Alternative to Local Polynomial Regression

Summary.  Local polynomial regression is a useful non-parametric regression tool to explore fine data structures and has been widely used in practice. We propose a new non-parametric regression technique called local composite quantile regression smoothing to improve local polynomial regression further. Sampling properties of the estimation procedure proposed are studied. We derive the asymptotic bias, variance and normality of the estimate proposed. The asymptotic relative efficiency of the estimate with respect to local polynomial regression is investigated. It is shown that the estimate can be much more efficient than the local polynomial regression estimate for various non-normal errors, while being almost as efficient as the local polynomial regression estimate for normal errors. Simulation is conducted to examine the performance of the estimates proposed. The simulation results are consistent with our theoretical findings. A real data example is used to illustrate the method proposed.     

Consistent bandwidth selection for kernel binary regression

The use of nonparametric regression techniques for binary regression is a promising alternative to parametric methods. As in other nonparametric smoothing problems, the choice of smoothing parameter is critical to the performance of the estimator and the appearance of the resulting estimate. In this paper, we discuss the use of selection criteria based on estimates of squared prediction risk and show consistency and asymptotic normality of the selected bandwidths. The usefulness of the methods is explored on a data set and in a small simulation study.     

Kernel spline regression

The authors propose «kernel spline regression,» a method of combining spline regression and kernel smoothing by replacing the polynomial approximation for local polynomial kernel regression with the spline basis. The new approach retains the local weighting scheme and the use of a bandwidth to control the size of local neighborhood. The authors compute the bias and variance of the kernel linear spline estimator, which they compare with local linear regression. They show that kernel spline estimators can succeed in capturing the main features of the underlying curve more effectively than local polynomial regression when the curvature changes rapidly. They also show through simulation that kernel spline regression often performs better than ordinary spline regression and local polynomial regression.     

Kernel adjusted nonparametric regression

In this paper we propose and study a new kernel regression estimator in which the kernel is taken from a properly adapted location-scale family of the design distribution. We show that, while the original smoothing may be performed with sub-optimal bandwidths, adaptation of proper scale parameters yields overall optimal estimators. Unlike traditional smoothing methodology, our approach does not aim at estimating pivotal higher order derivatives.     

Bias-corrected confidence bands in nonparametric regression

Bias-corrected confidence bands for general nonparametric regression models are considered. We use local polynomial fitting to construct the confidence bands and combine the cross-validation method and the plug-in method to select the bandwidths. Related asymptotic results are obtained. Our simulations show that confidence bands constructed by local polynomial fitting have much better coverage than those constructed by using the Nadaraya–Watson estimator. The results are also applicable to nonparametric autoregressive time series models.     

Estimation of global temperature fields from scattered observations by a spherical-wavelet-based spatially adaptive method

Summary.  The paper considers the problem of estimating the entire temperature field for every location on the globe from scattered surface air temperatures observed by a network of weather-stations. Classical methods such as spherical harmonics and spherical smoothing splines are not efficient in representing data that have inherent multiscale structures. The paper presents an estimation method that can adapt to the multiscale characteristics of the data. The method is based on a spherical wavelet approach that has recently been developed for a multiscale representation and analysis of scattered data. Spatially adaptive estimators are obtained by coupling the spherical wavelets with different thresholding (selective reconstruction) techniques. These estimators are compared for their spatial adaptability and extrapolation performance by using the surface air temperature data.     

Relative smoothing of discrete distributions with sparse observations

Quite often we are faced with a sparse number of observations over a finite number of cells and are interested in estimating the cell probabilities. Some local polynomial smoothers or local likelihood estimators have been proposed to improve on the histogram, which would produce too many zero values. We propose a relativized local polynomial smoothing for this problem, weighting heavier the estimating errors in small probability cells. A simulation study about the estimators that are proposed show a good behaviour with respect to natural error criteria, especially when dealing with sparse observations.     

A Flexible Method for Estimating the ROC Curve

In this paper we propose a flexible method for estimating a receiver operating characteristic (ROC) curve that is based on a continuous-scale test. The approach is easily understood and efficiently computed, and robust to the smooth parameter selection, which needs intensive computation when using local polynomial and smoothing spline techniques. The results from our simulation experiment indicate that the moderate-sample numerical performance of our estimator is better than the empirical ROC curve estimator and comparable to the local linear estimator. The availability of easy implementation is also illustrated by our simulation. We apply the proposed method to two real data sets.     

Local linear smoothing of receiver operating characteristic (ROC) curves

For measuring the accuracy of a continuous diagnostic test, the receiver operating characteristic (ROC) curve is often used. The empirical ROC curve is the most commonly used non-parametric estimator for the ROC curve. Recently, Lloyd (J. Amer. Statist. Assoc. 93(1998) 1356) proposed a kernel smoothing estimator for the ROC curve and showed his estimator has better mean square error than the empirical ROC curve estimator. However, Lloyd's estimator involves two bandwidths and has a boundary problem. In addition, his choice of bandwidths is ad hoc. In this paper we propose another kernel smoothing estimator which involves only one bandwidth and does not have the boundary problem. Furthermore, our choice of the bandwidth is asymptotically optimal.     

Two-step estimation of functional linear models with applications to longitudinal data

Functional linear models are useful in longitudinal data analysis. They include many classical and recently proposed statistical models for longitudinal data and other functional data. Recently, smoothing spline and kernel methods have been proposed for estimating their coefficient functions nonparametrically but these methods are either intensive in computation or inefficient in performance. To overcome these drawbacks, in this paper, a simple and powerful two-step alternative is proposed. In particular, the implementation of the proposed approach via local polynomial smoothing is discussed. Methods for estimating standard deviations of estimated coefficient functions are also proposed. Some asymptotic results for the local polynomial estimators are established. Two longitudinal data sets, one of which involves time-dependent covariates, are used to demonstrate the approach proposed. Simulation studies show that our two-step approach improves the kernel method proposed by Hoover and co-workers in several aspects such as accuracy, computational time and visual appeal of the estimators.     

NONPARAMETRIC AUTOCOVARIANCE FUNCTION ESTIMATION

Nonparametric estimators of autocovariance functions for non-stationary time series are developed. The estimators are based on straightforward nonparametric mean function estimation ideas and allow use of any linear smoother (e.g. smoothing spline, local polynomial). The paper studies the properties of the estimators, and illustrates their usefulness through application to some meteorological and seismic time series.     

Adaptive smoothing in associated kernel discrete functions estimation using Bayesian approach

This paper demonstrates that cross-validation (CV) and Bayesian adaptive bandwidth selection can be applied in the estimation of associated kernel discrete functions. This idea is originally proposed by Brewer [A Bayesian model for local smoothing in kernel density estimation, Stat. Comput. 10 (2000), pp. 299–309] to derive variable bandwidths in adaptive kernel density estimation. Our approach considers the adaptive binomial kernel estimator and treats the variable bandwidths as parameters with beta prior distribution. The best variable bandwidth selector is estimated by the posterior mean in the Bayesian sense under squared error loss. Monte Carlo simulations are conducted to examine the performance of the proposed Bayesian adaptive approach in comparison with the performance of the Asymptotic mean integrated squared error estimator and CV technique for selecting a global (fixed) bandwidth proposed in Kokonendji and Senga Kiessé [Discrete associated kernels method and extensions, Stat. Methodol. 8 (2011), pp. 497–516]. The Bayesian adaptive bandwidth estimator performs better than the global bandwidth, in particular for small and moderate sample sizes.