A Unified View of Nonparametric Trend-Cycle Predictors Via Reproducing Kernel Hilbert Spaces

We provide a common approach for studying several nonparametric estimators used for smoothing functional time series data. Linear filters based on different building assumptions are transformed into kernel functions via reproducing kernel Hilbert spaces. For each estimator, we identify a density function or second order kernel, from which a hierarchy of higher order estimators is derived. These are shown to give excellent representations for the currently applied symmetric filters. In particular, we derive equivalent kernels of smoothing splines in Sobolev and polynomial spaces. The asymmetric weights are obtained by adapting the kernel functions to the length of the various filters, and a theoretical and empirical comparison is made with the classical estimators used in real time analysis. The former are shown to be superior in terms of signal passing, noise suppression and speed of convergence to the symmetric filter.     

Local Estimation of the Second-Order Parameter in Extreme Value Statistics and Local Unbiased Estimation of the Tail Index

We develop and study in the framework of Pareto-type distributions a class of nonparametric kernel estimators for the conditional second order tail parameter. The estimators are obtained by local estimation of the conditional second order parameter using a moving window approach. Asymptotic normality of the proposed class of kernel estimators is proven under some suitable conditions on the kernel function and the conditional tail quantile function. The nonparametric estimators for the second order parameter are subsequently used to obtain a class of bias-corrected kernel estimators for the conditional tail index. In particular it is shown how for a given kernel function one obtains a bias-corrected kernel function, and that replacing the second order parameter in the latter with a consistent estimator does not change the limiting distribution of the bias-corrected estimator for the conditional tail index. The finite sample behavior of some specific estimators is illustrated with a simulation experiment. The developed methodology is also illustrated on fire insurance claim data.     

On finite sample properties of nonparametric discrete asymmetric kernel estimators

The discrete kernel method was developed to estimate count data distributions, distinguishing discrete associated kernels based on their asymptotic behaviour. This study investigates the class of discrete asymmetric kernels and their resulting non-consistent estimators, but this theoretical drawback of the estimators is balanced by some interesting features in small/medium samples. The role of modal probability and variance of discrete asymmetric kernels is highlighted to help better understand the performance of these kernels, in particular how the binomial kernel outperforms other asymmetric kernels. The performance of discrete asymmetric kernel estimators of probability mass functions is illustrated using simulations, in addition to applications to real data sets.     

Local Linear Least Squares Kernel Regression for Linear and Circular Predictors

We extend nonparametric regression models with local linear least squares fitting using kernel weights to the case of linear and circular predictors. We derive the asymptotic properties of the conditional bias and variance of bivariate local linear least squares kernel estimators. A small simulation study and a real experiment are given.     

Estimating the error variance in nonparametric regression by a covariate-matched u-statistic

For nonparametric regression models with fixed and random design, two classes of estimators for the error variance have been introduced: second sample moments based on residuals from a nonparametric fit, and difference-based estimators. The former are asymptotically optimal but require estimating the regression function; the latter are simple but have larger asymptotic variance. For nonparametric regression models with random covariates, we introduce a class of estimators for the error variance that are related to difference-based estimators: covariate-matched U-statistics. We give conditions on the random weights involved that lead to asymptotically optimal estimators of the error variance. Our explicit construction of the weights uses a kernel estimator for the covariate density.     

Minimax properties of beta kernel estimators

In this paper, we are interested in the study of beta kernel density estimators from an asymptotic minimax point of view. These estimators allows to estimate density functions with support in [0,1]. It is well-known that beta kernel estimators are - on the contrary of classical kernel estimators - “free of boundary effect” and thus are very useful in practice. The goal of this paper is to prove that there is a price to pay: for very regular density functions or for certain losses, these estimators are not minimax. Nevertheless they are minimax for classical regularities such as regularity of order two or less than two, supposed commonly in the practice and for some classical losses.     

Correction of Density Estimators that are not Densities

Abstract. Several old and new density estimators may have good theoretical performance, but are hampered by not being bona fide densities; they may be negative in certain regions or may not integrate to 1. One can therefore not simulate from them, for example. This paper develops general modification methods that turn any density estimator into one which is a bona fide density, and which is always better in performance under one set of conditions and arbitrarily close in performance under a complementary set of conditions. This improvement-for-free procedure can, in particular, be applied for higher-order kernel estimators, classes of modern h ⁴ bias kernel type estimators, superkernel estimators, the sinc kernel estimator, the k -NN estimator, orthogonal expansion estimators, and for various recently developed semi-parametric density estimators.     

Bernstein conditional density estimation with application to conditional distribution and regression functions

In this paper we propose a smooth nonparametric estimation for the conditional probability density function based on a Bernstein polynomial representation. Our estimator can be written as a finite mixture of beta densities with data-driven weights. Using the Bernstein estimator of the conditional density function, we derive new estimators for the distribution function and conditional mean. We establish the asymptotic properties of the proposed estimators, by proving their asymptotic normality and by providing their asymptotic bias and variance. Simulation results suggest that the proposed estimators can outperform the Nadaraya–Watson estimator and, in some specific setups, the local linear kernel estimators. Finally, we use our estimators for modeling the income in Italy, conditional on year from 1951 to 1998, and have another look at the well known Old Faithful Geyser data.     

Partial deconvolution estimation in nonparametric regression

In this article, we propose a class of partial deconvolution kernel estimators for the nonparametric regression function when some covariates are measured with error and some are not. The estimation procedure combines the classical kernel methodology and the deconvolution kernel technique. According to whether the measurement error is ordinarily smooth or supersmooth, we establish the optimal local and global convergence rates for these proposed estimators, and the optimal bandwidths are also identified. Furthermore, lower bounds for the convergence rates of all possible estimators for the nonparametric regression functions are developed. It is shown that, in both the super and ordinarily smooth cases, the convergence rates of the proposed partial deconvolution kernel estimators attain the lower bound. The Canadian Journal of Statistics 48: 535–560; 2020 © 2020 Statistical Society of Canada     

Some Asymptotic Results of Kernel Density Estimators Under Random Left-Truncation and Dependent Data

Problems with truncated data arise frequently in survival analyses and reliability applications. The estimation of the density function of the lifetimes is often of interest. In this article, the estimation of density function by the kernel method is considered, when truncated data are showing some kind of dependence. We apply the strong Gaussian approximation technique to study the strong uniform consistency for kernel estimators of the density function under a truncated dependent model. We also apply the strong approximation results to study the integrated square error properties of the kernel density estimators under the truncated dependent scheme.     

相依函数型数据的局部回归估计的渐近正态性

函数型数据研究近年来为越来越多的学者所重视,其在天文,医药,经济现象,生态环境及工业制造等诸多方面均有重要应用.非参数统计是统计研究的一个重要方面,其中核函数估计和局部多项式方法是这一类研究中重要常用方法.函数型数据的非参数方法中以核函数估计方法较为常见,且其收敛速度与极限分布无论在独立情形还是相依情形都有理论结果.而局部多项式的研究在函数型数据背景下较为少见,原因在于将局部多项式方法推广到函数型数据背景一直是一个难题. Marin, Ferraty, Vieu [Journal of Nonparametric Statistics, 22 (5) (2010), pp.617-632] 提出了非参函数型模型的局部回归估计. 这种估计可以看作是局部多项式估计在函数型数据背景下的一个推广.这种方法提出后,许多学者进一步研究了这种方法,考察了这种方法的收敛速度和极限分布,并将这种方法应用到不同的模型中以适应实际需求.但是,前人的研究都要求数据具有独立同分布的性质.然而许多实际数据并不符合这一假设.本文研究了在相依函数型数据情形下局部回归估计的渐近正态性.由于估计方法有差异,核函数估计的研究方法无法直接推广到局部回归估计,而相依性结构也给研究带来了一些挑战,我们采用Bernstein分块方法将相依性问题转化为渐近独立的问题,从而得到了估计的渐近正态性.此外我们还采用数据模拟的方法进一步验证了渐近正态的结果.     

On adaptive estimation

Point estimates that are weighted averages of other estimates are considered. They are adaptive because the weights are also functions of the sample observations.In particular, the weights are functions of new measures of peakedness and skewness. Five adaptive estimators are compared (in a Monte Carlo study using the swindle) to some of the usual estimators, including those robust ones of Huber and Tukey. In addition, the swindle constant is considered in some detail. All of the adaptive estimators do extremely well with an adaptive biweight statistic being the best one in this study. Suggestions are made about future directions in this area.     

Semiparametric multiple kernel estimators and model diagnostics for count regression functions

Abstract

This study concerns semiparametric approaches to estimate discrete multivariate count regression functions. The semiparametric approaches investigated consist of combining discrete multivariate nonparametric kernel and parametric estimations such that (i) a prior knowledge of the conditional distribution of model response may be incorporated and (ii) the bias of the traditional nonparametric kernel regression estimator of Nadaraya-Watson may be reduced. We are precisely interested in combination of the two estimations approaches with some asymptotic properties of the resulting estimators. Asymptotic normality results were showed for nonparametric correction terms of parametric start function of the estimators. The performance of discrete semiparametric multivariate kernel estimators studied is illustrated using simulations and real count data. In addition, diagnostic checks are performed to test the adequacy of the parametric start model to the true discrete regression model. Finally, using discrete semiparametric multivariate kernel estimators provides a bias reduction when the parametric multivariate regression model used as start regression function belongs to a neighborhood of the true regression model.     

Estimating the conditional tail index by integrating a kernel conditional quantile estimator

This paper deals with the estimation of the tail index of a heavy-tailed distribution in the presence of covariates. A class of estimators is proposed in this context and its asymptotic normality established under mild regularity conditions. These estimators are functions of a kernel conditional quantile estimator depending on some tuning parameters. The finite sample properties of our estimators are illustrated on a small simulation study.     

Comparison of Kernel Density Estimators with Assumption on Number of Modes

A data-driven bandwidth choice for a kernel density estimator called critical bandwidth is investigated. This procedure allows the estimation to have as many modes as assumed for the density to estimate. Both Gaussian and uniform kernels are considered. For the Gaussian kernel, asymptotic results are given. For the uniform kernel, an argument against these properties is mentioned. These theoretical results are illustrated with a simulation study that compares the kernel estimators that rely on critical bandwidth with another one that uses a plug-in method to select its bandwidth. An estimator that consists in estimates of density contour clusters and takes assumptions on number of modes into account is also considered. Finally, the methodology is illustrated using environment monitoring data.     

A comparison study of nonparametric imputation methods

Consider estimation of a population mean of a response variable when the observations are missing at random with respect to the covariate. Two common approaches to imputing the missing values are the nonparametric regression weighting method and the Horvitz-Thompson (HT) inverse weighting approach. The regression approach includes the kernel regression imputation and the nearest neighbor imputation. The HT approach, employing inverse kernel-estimated weights, includes the basic estimator, the ratio estimator and the estimator using inverse kernel-weighted residuals. Asymptotic normality of the nearest neighbor imputation estimators is derived and compared to kernel regression imputation estimator under standard regularity conditions of the regression function and the missing pattern function. A comprehensive simulation study shows that the basic HT estimator is most sensitive to discontinuity in the missing data patterns, and the nearest neighbors estimators can be insensitive to missing data patterns unbalanced with respect to the distribution of the covariate. Empirical studies show that the nearest neighbor imputation method is most effective among these imputation methods for estimating a finite population mean and for classifying the species of the iris flower data.     

Local regression for vector responses

We explore a class of vector smoothers based on local polynomial regression for fitting nonparametric regression models which have a vector response. The asymptotic bias and variance for the class of estimators are derived for two different ways of representing the variance matrices within both a seemingly unrelated regression and a vector measurement error framework. We show that the asymptotic behaviour of the estimators is different in these four cases. In addition, the placement of the kernel weights in weighted least squares estimators is very important in the seeming unrelated regressions problem (to ensure that the estimator is asymptotically unbiased) but not in the vector measurement error model. It is shown that the component estimators are asymptotically uncorrelated in the seemingly unrelated regressions model but asymptotically correlated in the vector measurement error model. These new and interesting results extend our understanding of the problem of smoothing dependent data.     

Boundary and Bias Correction in Kernel Hazard Estimation

A new class of local linear hazard estimators based on weighted least square kernel estimation is considered. The class includes the kernel hazard estimator of Ramlau-Hansen (1983), which has the same boundary correction property as the local linear regression estimator (see Fan & Gijbels, 1996). It is shown that all the local linear estimators in the class have the same pointwise asymptotic properties. We derive the multiplicative bias correction of the local linear estimator. In addition we propose a new bias correction technique based on bootstrap estimation of additive bias. This latter method has excellent theoretical properties. Based on an extensive simulation study where we compare the performance of competing estimators, we also recommend the use of the additive bias correction in applied work.     

Robust kernel estimators for additive models with dependent observations

Robust nonparametric estimators for additive regression or autoregression models under an α-mixing condition are proposed. They are based on local M-estimators or local medians with kernel weights, and their asymptotic behaviour is studied. Moreover, diese local M-estimators achieve the same univariate rate of convergence as their linear relatives.     

Bandwidth selection procedures tor kernel density estimates

A crucial problem in kernel density estimates of a probability density function is the selection of the bandwidth. The aim of this study is to propose a procedure for selecting both fixed and variable bandwidths. The present study also addresses the question of how different variable bandwidth kernel estimators perform in comparison with each other and to the fixed type of bandwidth estimators. The appropriate algorithms for implementation of the proposed method are given along with a numerical simulation.The numerical results serve as a guide to determine which bandwidth selection method is most appropriate for a given type of estimator over a vide class of probability density functions, Also, we obtain a numerical comparison of the different types of kernel estimators under various types of bandwidths.