Bootstrap bandwidth selection method for local linear estimator in exponential family models

Many biological experiments involve data whose distribution belongs to the exponential family. Such data are often analysed using generalised linear models but this method requires specification of the link function which can have strong influence on the resulting estimate. Instead a local method based on quasi-likelihood can be used, but the choice of the smoothing parameter is crucial for its performance. A bootstrap bandwidth selection method is proposed and shown to be consistent. Examples of application to data from biological and psychometric experiments are given.     

Interactive local bandwidth choice

A tool for user choice of the local bandwidth function for kernel density and nonparametric regression estimates is developed using KDE, a graphical object-oriented package for interactive kernel density estimation written in LISP-STAT. The bandwidth function is a parameterized spline, whose knots are manipulated by the user in one window, while the resulting estimate appears in another window. A real data illustration of this method raises concerns, because an extremely large family of estimates is available. Suggestions are made to overcome this problem so that this tool can be used effectively for presenting final results of a data analysis.     

A weighted polynomial regression method for local fitting of spatial data

Typically, parametric approaches to spatial problems require restrictive assumptions. On the other hand, in a wide variety of practical situations nonparametric bivariate smoothing techniques has been shown to be successfully employable for estimating small or large scale regularity factors, or even the signal content of spatial data taken as a whole.We propose a weighted local polynomial regression smoother suitable for fitting of spatial data. To account for spatial variability, we both insert a spatial contiguity index in the standard formulation, and construct a spatial-adaptive bandwidth selection rule. Our bandwidth selector depends on the Gearys local indicator of spatial association. As illustrative example, we provide a brief Monte Carlo study case on equally spaced data, the performances of our smoother and the standard polynomial regression procedure are compared.This note, though it is the result of a close collaboration, was specifically elaborated as follows: paragraphs 1 and 2 by T. Sclocco and the remainder by M. Di Marzio. The authors are grateful to the referees for constructive comments and suggestions.     

Further theoretical and practical insight to the do-validated bandwidth selector

Recent contributions to kernel smoothing show that the performance of cross-validated bandwidth selectors improves significantly from indirectness and that the recent do-validated method seems to provide the most practical alternative among these methods. In this paper we show step by step how classical cross-validation improves in theory, as well as in practice, from indirectness and that do-validated estimators improve in theory, but not in practice, from further indirectness. This paper therefore provides a strong support for the practical and theoretical properties of do-validated bandwidth selection. Do-validation is currently being introduced to survival analysis in a number of contexts and this paper provides evidence that this might be the immediate step forward.     

Smoothed bootstrap bandwidth selection for nonparametric hazard rate estimation

A smoothed bootstrap method is presented for the purpose of bandwidth selection in nonparametric hazard rate estimation for iid data. In this context, two new bootstrap bandwidth selectors are established based on the exact expression of the bootstrap version of the mean integrated squared error of some approximations of the kernel hazard rate estimator. This is very useful since Monte Carlo approximation is no longer needed for the implementation of the two bootstrap selectors. A simulation study is carried out in order to show the empirical performance of the new bootstrap bandwidths and to compare them with other existing selectors. The methods are illustrated by applying them to a diabetes data set.     

A weighted least-squares cross-validation bandwidth selector for kernel density estimation

Since the late 1980s, several methods have been considered in the literature to reduce the sample variability of the least-squares cross-validation bandwidth selector for kernel density estimation. In this article, a weighted version of this classical method is proposed and its asymptotic and finite-sample behavior is studied. The simulation results attest that the weighted cross-validation bandwidth performs quite well, presenting a better finite-sample performance than the standard cross-validation method for “easy-to-estimate” densities, and retaining the good finite-sample performance of the standard cross-validation method for “hard-to-estimate” ones.     

Multivariate bandwidth selection for local linear regression

The existence and properties of optimal bandwidths for multivariate local linear regression are established, using either a scalar bandwidth for all regressors or a diagonal bandwidth vector that has a different bandwidth for each regressor. Both involve functionals of the derivatives of the unknown multivariate regression function. Estimating these functionals is difficult primarily because they contain multivariate derivatives. In this paper, an estimator of the multivariate second derivative is obtained via local cubic regression with most cross-terms left out. This estimator has the optimal rate of convergence but is simpler and uses much less computing time than the full local estimator. Using this as a pilot estimator, we obtain plug-in formulae for the optimal bandwidth, both scalar and diagonal, for multivariate local linear regression. As a simpler alternative, we also provide rule-of-thumb bandwidth selectors. All these bandwidths have satisfactory performance in our simulation study.     

Convergence rates for uniform confidence intervals based on local polynomial regression estimators

We investigate the convergence rates of uniform bias-corrected confidence intervals for a smooth curve using local polynomial regression for both the interior and boundary region. We discuss the cases when the degree of the polynomial is odd and even. The uniform confidence intervals are based on the volume-of-tube formula modified for biased estimators. We empirically show that the proposed uniform confidence intervals attain, at least approximately, nominal coverage. Finally, we investigate the performance of the volume-of-tube based confidence intervals for independent non-Gaussian errors.     

A block bootstrap comparison for sparse chains

In this paper we apply the sequential bootstrap method proposed by Collet et al. [Bootstrap Central Limit theorem for chains of infinite order via Markov approximations, Markov Processes and Related Fields 11(3) (2005), pp. 443–464] to estimate the variance of the empirical mean of a special class of chains of infinite order called sparse chains. For this process, we show that we are able to compute numerically the true value of the standard error with any fixed error.

Our main goal is to present a comparison, for sparse chains, among sequential bootstrap, the block bootstrap method proposed by Künsch [The jackknife and the Bootstrap for general stationary observations, Ann. Statist. 17 (1989), pp. 1217–1241] and improved by Liu and Singh [Moving blocks jackknife and Bootstrap capture week dependence, in Exploring the limits of the Bootstrap, R. Lepage and L. Billard, eds., Wiley, New York, 1992, pp. 225–248] and the bootstrap method proposed by Bühlmann [Blockwise bootstrapped empirical process for stationary sequences, Ann. Statist. 22 (1994), pp. 995–1012].     

On kernel density derivative estimation

Estimators of derivatives of a density function based on polynomial multiples of kernels are compared with those based on differentiated kernels.     

Linex discrepancy for bandwidth selection

A bandwidth selection based on Linex discrepancy is proposed for kernel smoothing of periodogram. The selection minimizes Linex discrepancy between the smoothed and true spectrums. Two estimators are introduced for Linex discrepancy. The bandwidth choice outperforms some common bandwidth choices.     

A hybrid bandwidth selection methodology for kernel density estimation

A bandwidth selection method that combines the concept of least-squares cross-validation and the plug-in approach is being introduced in connection with kernel density estimation. A simulation study reveals that this hybrid methodology outperforms some commonly used bandwidth selection rules. It is shown that the proposed approach can also be readily employed in the context of variable kernel density estimation. We conclude with two illustrative examples.     

Improved local polynomial estimation in time series regression

We propose a modification of local polynomial estimation which improves the efficiency of the conventional method when the observation errors are correlated. The procedure is based on a pre-transformation of the data as a generalization of the pre-whitening procedure introduced by Xiao et al. [(2003), ‘More Efficient Local Polynomial Estimation in Nonparametric Regression with Autocorrelated Errors’, Journal of the American Statistical Association, 98, 980–992]. While these authors assumed a linear process representation for the error process, we avoid any structural assumption. We further allow the regressors and the errors to be dependent. More importantly, we show that the inclusion of both leading and lagged variables in the approximation of the error terms outperforms the best approximation based on lagged variables only. Establishing its asymptotic distribution, we show that the proposed estimator is more efficient than the standard local polynomial estimator. As a by-product we prove a suitable version of a central limit theorem which allows us to improve the asymptotic normality result for local polynomial estimators by Masry and Fan [(1997), ‘Local Polynomial Estimation of Regression Functions for Mixing Processes’, Scandinavian Journal of Statistics, 24, 165–179]. A simulation study confirms the efficiency of our estimator on finite samples. An application to climate data also shows that our new method leads to an estimator with decreased variability.     

Boundary Aware Estimators of Integrated Density Derivative Products

Integrated squared density derivatives are important to the plug-in type of bandwidth selector for kernel density estimation. Conventional estimators of these quantities are inefficient when there is a non-smooth boundary in the support of the density. We introduce estimators that utilize density derivative estimators obtained from local polynomial fitting. They retain the rates of convergence in mean-squared error that are familiar from non-boundary cases, and the constant coefficients have similar forms. The estimators and the formula for their asymptotically optimal bandwidths, which depend on integrated products of density derivatives, are applied to automatic bandwidth selection for local linear density estimation. Simulation studies show that the constructed bandwidth rule and the Sheather–Jones bandwidth are competitive in non-boundary cases, but the former overcomes boundary problems whereas the latter does not.     

Bootstrap inference in local polynomial regression of time series

In this paper we consider the inferential aspect of the nonparametric estimation of a conditional function , where X _t,m represents the vector containing the m conditioning lagged values of the series. Here is an arbitrary measurable function. The local polynomial estimator of order p is used for the estimation of the function g, and of its partial derivatives up to a total order p. We consider α-mixing processes, and we propose the use of a particular resampling method, the local polynomial bootstrap, for the approximation of the sampling distribution of the estimator. After analyzing the consistency of the proposed method, we present a simulation study which gives evidence of its finite sample behaviour.     

Automatic Local Smoothing for Spectral Density Estimation

This article uses local polynomial techniques to fit Whittle's likelihood for spectral density estimation. Asymptotic sampling properties of the proposed estimators are derived, and adaptation of the proposed estimator to the boundary effect is demonstrated. We show that the Whittle likelihood-based estimator has advantages over the least-squares based log-periodogram. The bandwidth for the Whittle likelihood-based method is chosen by a simple adjustment of a bandwidth selector proposed in Fan & Gijbels (1995). The effectiveness of the proposed procedure is demonstrated by a few simulated and real numerical examples. Our simulation results support the asymptotic theory that the likelihood based spectral density and log-spectral density estimators are the most appealing among their peers     

A simple,automatic and adaptive bivariate density estimator based on conditional densities

The standard approach to non-parametric bivariate density estimation is to use a kernel density estimator. Practical performance of this estimator is hindered by the fact that the estimator is not adaptive (in the sense that the level of smoothing is not sensitive to local properties of the density). In this paper a simple, automatic and adaptive bivariate density estimator is proposed based on the estimation of marginal and conditional densities. Asymptotic properties of the estimator are examined, and guidance to practical application of the method is given. Application to two examples illustrates the usefulness of the estimator as an exploratory tool, particularly in situations where the local behaviour of the density varies widely. The proposed estimator is also appropriate for use as a pilot estimate for an adaptive kernel estimate, since it is relatively inexpensive to calculate.     

A new information criterion-based bandwidth selection method for non-parametric regressions

ABSTRACT

Local linear estimator is a popularly used method to estimate the non-parametric regression functions, and many methods have been derived to estimate the smoothing parameter, or the bandwidth in this case. In this article, we propose an information criterion-based bandwidth selection method, with the degrees of freedom originally derived for non-parametric inferences. Unlike the plug-in method, the new method does not require preliminary parameters to be chosen in advance, and is computationally efficient compared to the cross-validation (CV) method. Numerical study shows that the new method performs better or comparable to existing plug-in method or CV method in terms of the estimation of the mean functions, and has lower variability than CV selectors. Real data applications are also provided to illustrate the effectiveness of the new method.     

Gains from joint cross validation bandwidth selection for derivatives of conditional multidimensional densities

This paper studies bandwidth selection for kernel estimation of derivatives of multidimensional conditional densities, a non-parametric realm unexplored in the literature. This paper extends Baird [Cross validation bandwidth selection for derivatives of multidimensional densities. RAND Working Paper series, WR-1060; 2014] in its examination of conditional multivariate densities, derives and presents criteria for arbitrary kernel order and density dimension, shows consistency of the estimators, and investigates a minimization criterion which jointly estimates numerator and denominator bandwidths. I conduct a Monte Carlo simulation study for various orders of kernels in the Gaussian family and compare the new cross validation criterion with those implied by Baird [Cross validation bandwidth selection for derivatives of multidimensional densities. RAND Working Paper series, WR-1060; 2014]. The paper finds that higher order kernels become increasingly important as the dimension of the distribution increases. I find that the cross validation criterion developed in this paper that jointly estimates the derivative of the joint density (numerator) and the marginal density (denominator) does orders of magnitude better than criteria that estimate the bandwidths separately. I further find that using the infinite order Dirichlet kernel tends to have the best results.