Nonparametric density estimation for multivariate bounded data

We propose a new nonparametric estimator for the density function of multivariate bounded data. As frequently observed in practice, the variables may be partially bounded (e.g. nonnegative) or completely bounded (e.g. in the unit interval). In addition, the variables may have a point mass. We reduce the conditions on the underlying density to a minimum by proposing a nonparametric approach. By using a gamma, a beta, or a local linear kernel (also called boundary kernels), in a product kernel, the suggested estimator becomes simple in implementation and robust to the well known boundary bias problem. We investigate the mean integrated squared error properties, including the rate of convergence, uniform strong consistency and asymptotic normality. We establish consistency of the least squares cross-validation method to select optimal bandwidth parameters. A detailed simulation study investigates the performance of the estimators. Applications using lottery and corporate finance data are provided.     

Nonparametric density estimation based on beta prime kernel

Abstract

In this work, we propose beta prime kernel estimator for estimation of a probability density functions defined with nonnegative support. For the proposed estimator, beta prime probability density function used as a kernel. It is free of boundary bias and nonnegative with a natural varying shape. We obtained the optimal rate of convergence for the mean squared error (MSE) and the mean integrated squared error (MISE). Also, we use adaptive Bayesian bandwidth selection method with Lindley approximation for heavy tailed distributions and compare its performance with the global least squares cross-validation bandwidth selection method. Simulation studies are performed to evaluate the average integrated squared error (ISE) of the proposed kernel estimator against some asymmetric competitors using Monte Carlo simulations. Moreover, real data sets are presented to illustrate the findings.     

Posterior consistency of logistic Gaussian process priors in density estimation

We establish weak and strong posterior consistency of Gaussian process priors studied by Lenk [1988. The logistic normal distribution for Bayesian, nonparametric, predictive densities. J. Amer. Statist. Assoc. 83 (402), 509–516] for density estimation. Weak consistency is related to the support of a Gaussian process in the sup-norm topology which is explicitly identified for many covariance kernels. In fact we show that this support is the space of all continuous functions when the usual covariance kernels are chosen and an appropriate prior is used on the smoothing parameters of the covariance kernel. We then show that a large class of Gaussian process priors achieve weak as well as strong posterior consistency (under some regularity conditions) at true densities that are either continuous or piecewise continuous.     

On multivariate associated kernels to estimate general density functions

Multivariate associated kernel estimators, which depend on both target point and bandwidth matrix, are appropriate for distributions with partially or totally bounded supports and generalize the classical ones such as the Gaussian. Previous studies on multivariate associated kernels have been restricted to products of univariate associated kernels, also considered having diagonal bandwidth matrices. However, it has been shown in classical cases that, for certain forms of target density such as multimodal ones, the use of full bandwidth matrices offers the potential for significantly improved density estimation. In this paper, general associated kernel estimators with correlation structure are introduced. Asymptotic properties of these estimators are presented; in particular, the boundary bias is investigated. Generalized bivariate beta kernels are handled in more details. The associated kernel with a correlation structure is built with a variant of the mode-dispersion method and two families of bandwidth matrices are discussed using the least squared cross validation method. Simulation studies are done. In the particular situation of bivariate beta kernels, a very good performance of associated kernel estimators with correlation structure is observed compared to the diagonal case. Finally, an illustration on a real dataset of paired rates in a framework of political elections is presented.     

On the Local Polynomial Estimators of the Counting Process Intensity Function and its Derivatives

Abstract. We consider the properties of the local polynomial estimators of a counting process intensity function and its derivatives. By expressing the local polynomial estimators in a kernel smoothing form via effective kernels, we show that the bias and variance of the estimators at boundary points are of the same magnitude as at interior points and therefore the local polynomial estimators in the context of intensity estimation also enjoy the automatic boundary correction property as they do in other contexts such as regression. The asymptotically optimal bandwidths and optimal kernel functions are obtained through the asymptotic expressions of the mean square error of the estimators. For practical purpose, we suggest an effective and easy‐to‐calculate data‐driven bandwidth selector. Simulation studies are carried out to assess the performance of the local polynomial estimators and the proposed bandwidth selector. The estimators and the bandwidth selector are applied to estimate the rate of aftershocks of the Sichuan earthquake and the rate of the Personal Emergency Link calls in Hong Kong.     

Beta-Bernstein Smoothing for Regression Curves with Compact Support

ABSTRACT. The problem of boundary bias is associated with kernel estimation for regression curves with compact support. This paper proposes a simple and uni(r)ed approach for remedying boundary bias in non-parametric regression, without dividing the compact support into interior and boundary areas and without applying explicitly different smoothing treatments separately. The approach uses the beta family of density functions as kernels. The shapes of the kernels vary according to the position where the curve estimate is made. Theyare symmetric at the middle of the support interval, and become more and more asymmetric nearer the boundary points. The kernels never put any weight outside the data support interval, and thus avoid boundary bias. The method is a generalization of classical Bernstein polynomials, one of the earliest methods of statistical smoothing. The proposed estimator has optimal mean integrated squared error at an order of magnitude n ^−4/5, equivalent to that of standard kernel estimators when the curve has an unbounded support.     

Deconvolution boundary kernel method in nonparametric density estimation

This paper considers the nonparametric deconvolution problem when the true density function is left (or right) truncated. We propose to remove the boundary effect of the conventional deconvolution density estimator by using a special class of kernels: the deconvolution boundary kernels. Methods for constructing such kernels are provided. The mean squared error properties, including the rates of convergence, are investigated for supersmooth and ordinary smooth errors. Numerical simulations show that the deconvolution boundary kernel estimator successfully removes the boundary effects of the conventional deconvolution density estimator.     

A new class of boundary kernels for distribution function estimation

In this note we introduce a new class of boundary kernels for distribution function estimation which shows itself to be especially performing when the classical kernel distribution function estimator suffers from severe boundary problems.     

Density estimation using asymmetric kernels and Bayes bandwidths with censored data

We propose a modification to the regular kernel density estimation method that use asymmetric kernels to circumvent the spill over problem for densities with positive support. First a pivoting method is introduced for placement of the data relative to the kernel function. This yields a strongly consistent density estimator that integrates to one for each fixed bandwidth in contrast to most density estimators based on asymmetric kernels proposed in the literature. Then a data-driven Bayesian local bandwidth selection method is presented and lognormal, gamma, Weibull and inverse Gaussian kernels are discussed as useful special cases. Simulation results and a real-data example illustrate the advantages of the new methodology.     

Local orthogonal polynomial expansion for density estimation

A local orthogonal polynomial expansion (LOrPE) of the empirical density function is proposed as a novel method to estimate the underlying density. The estimate is constructed by matching localised expectation values of orthogonal polynomials to the values observed in the sample. LOrPE is related to several existing methods, and generalises straightforwardly to multivariate settings. By manner of construction, it is similar to local likelihood density estimation (LLDE). In the limit of small bandwidths, LOrPE functions as kernel density estimation (KDE) with high-order (effective) kernels inherently free of boundary bias, a natural consequence of kernel reshaping to accommodate endpoints. Consistency and faster asymptotic convergence rates follow. In the limit of large bandwidths LOrPE is equivalent to orthogonal series density estimation (OSDE) with Legendre polynomials, thereby inheriting its consistency. We compare the performance of LOrPE to KDE, LLDE, and OSDE, in a number of simulation studies. In terms of mean integrated squared error, the results suggest that with a proper balance of the two tuning parameters, bandwidth and degree, LOrPE generally outperforms these competitors when estimating densities with sharply truncated supports.     

Discrete associated kernels method and extensions

Discrete kernel estimation of a probability mass function (p.m.f.), often mentioned in the literature, has been far less investigated in comparison with continuous kernel estimation of a probability density function (p.d.f.). In this paper, we are concerned with a general methodology of discrete kernels for smoothing a p.m.f. f. We give a basic of mathematical tools for further investigations. First, we point out a generalizable notion of discrete associated kernel which is defined at each point of the support of f and built from any parametric discrete probability distribution. Then, some properties of the corresponding estimators are shown, in particular pointwise and global (asymptotical) properties. Other discrete kernels are constructed from usual discrete probability distributions such as Poisson, binomial and negative binomial. For small samples sizes, underdispersed discrete kernel estimators are more interesting than the empirical estimator; thus, an importance of discrete kernels is illustrated. The choice of smoothing bandwidth is classically investigated according to cross-validation and, novelly, to excess of zeros methods. Finally, a unification way of this method concerning the general probability function is discussed.     

Simple boundary correction for kernel density estimation

If a probability density function has bounded support, kernel density estimates often overspill the boundaries and are consequently especially biased at and near these edges. In this paper, we consider the alleviation of this boundary problem. A simple unified framework is provided which covers a number of straightforward methods and allows for their comparison: generalized jackknifing generates a variety of simple boundary kernel formulae. A well-known method of Rice (1984) is a special case. A popular linear correction method is another: it has close connections with the boundary properties of local linear fitting (Fan and Gijbels, 1992). Links with the optimal boundary kernels of Müller (1991) are investigated. Novel boundary kernels involving kernel derivatives and generalized reflection arise too. In comparisons, various generalized jackknifing methods perform rather similarly, so this, together with its existing popularity, make linear correction as good a method as any. In an as yet unsuccessful attempt to improve on generalized jackknifing, a variety of alternative approaches is considered. A further contribution is to consider generalized jackknife boundary correction for density derivative estimation. En route to all this, a natural analogue of local polynomial regression for density estimation is defined and discussed.     

A new non parametric estimator for Pdf based on inverse gamma distribution

ABSTRACT

The non parametric approach is considered to estimate probability density function (Pdf) which is supported on(0, ∞). This approach is the inverse gamma kernel. We show that it has same properties as gamma, reciprocal inverse Gaussian, and inverse Gaussian kernels such that it is free of the boundary bias, non negative, and it achieves the optimal rate of convergence for the mean integrated squared error. Also some properties of the estimator were established such as bias and variance. Comparison of the bandwidth selection methods for inverse gamma kernel estimation of Pdf is done.     

Optimal kernels when estimating non-smooth densities

The derivation of new kernel functions for the kernel estimator of an unknown density function is given. These kernels are shown to be optimal in some sense when the underlying density f is continuous but its derivative f′ is not, and consequently a solu tion is presented for an unsolved problem which was stated by van Eeden (1985). Other attractive features of these kernels are also discussed and a number of graphs are listed.     

Gains from joint cross validation bandwidth selection for derivatives of conditional multidimensional densities

This paper studies bandwidth selection for kernel estimation of derivatives of multidimensional conditional densities, a non-parametric realm unexplored in the literature. This paper extends Baird [Cross validation bandwidth selection for derivatives of multidimensional densities. RAND Working Paper series, WR-1060; 2014] in its examination of conditional multivariate densities, derives and presents criteria for arbitrary kernel order and density dimension, shows consistency of the estimators, and investigates a minimization criterion which jointly estimates numerator and denominator bandwidths. I conduct a Monte Carlo simulation study for various orders of kernels in the Gaussian family and compare the new cross validation criterion with those implied by Baird [Cross validation bandwidth selection for derivatives of multidimensional densities. RAND Working Paper series, WR-1060; 2014]. The paper finds that higher order kernels become increasingly important as the dimension of the distribution increases. I find that the cross validation criterion developed in this paper that jointly estimates the derivative of the joint density (numerator) and the marginal density (denominator) does orders of magnitude better than criteria that estimate the bandwidths separately. I further find that using the infinite order Dirichlet kernel tends to have the best results.     

Nonparametric density deconvolution by weighted kernel estimators

Nonparametric density estimation in the presence of measurement error is considered. The usual kernel deconvolution estimator seeks to account for the contamination in the data by employing a modified kernel. In this paper a new approach based on a weighted kernel density estimator is proposed. Theoretical motivation is provided by the existence of a weight vector that perfectly counteracts the bias in density estimation without generating an excessive increase in variance. In practice a data driven method of weight selection is required. Our strategy is to minimize the discrepancy between a standard kernel estimate from the contaminated data on the one hand, and the convolution of the weighted deconvolution estimate with the measurement error density on the other hand. We consider a direct implementation of this approach, in which the weights are optimized subject to sum and non-negativity constraints, and a regularized version in which the objective function includes a ridge-type penalty. Numerical tests suggest that the weighted kernel estimation can lead to tangible improvements in performance over the usual kernel deconvolution estimator. Furthermore, weighted kernel estimates are free from the problem of negative estimation in the tails that can occur when using modified kernels. The weighted kernel approach generalizes to the case of multivariate deconvolution density estimation in a very straightforward manner.     

Estimating Percentiles of Time-to-Failure Distribution Obtained from a Linear Degradation Model Using Kernel Density Method

In this article, we propose a nonparametric estimator for percentiles of the time-to-failure distribution obtained from a linear degradation model using the kernel density method. The properties of the proposed kernel estimator are investigated and compared with well-known maximum likelihood and ordinary least squares estimators via a simulation technique. The mean squared error and the length of the bootstrap confidence interval are used as the basis criteria of the comparisons. The simulation study shows that the performance of the kernel estimator is acceptable as a general estimator. When the distribution of the data is assumed to be known, the maximum likelihood and ordinary least squares estimators perform better than the kernel estimator, while the kernel estimator is superior when the assumption of our knowledge of the data distribution is violated. A comparison among different estimators is achieved using a real data set.     

A generalized reflection method of boundary correction in kernel density estimation

The kernel method of estimation of curves is now popular and widely used in statistical applications. Kernel estimators suffer from boundary effects, however, when the support of the function to be estimated has finite endpoints. Several solutions to this problem have already been proposed. Here the authors develop a new method of boundary correction for kernel density estimation. Their technique is a kind of generalized reflection involving transformed data. It generates a class of boundary corrected estimators having desirable properties such as local smoothness and nonnegativity. Simulations show that the proposed method performs quite well when compared with the existing methods for almost all shapes of densities. The authors present the theory behind this new methodology, and they determine the bias and variance of their estimators.     

BIAS REDUCTION AND ELIMINATION WITH KERNEL ESTIMATORS

A great deal of research has focused on improving the bias properties of kernel estimators. One proposal involves removing the restriction of non-negativity on the kernel to construct “higher-order” kernels that eliminate additional terms in the Taylor's series expansion of the bias. This paper considers an alternative that uses a local approach to bandwidth selection to not only reduce the bias, but to eliminate it entirely. These so-called “zero-bias bandwidths” are shown to exist for univariate and multivariate kernel density estimation as well as kernel regression. Implications of the existence of such bandwidths are discussed. An estimation strategy is presented, and the extent of the reduction or elimination of bias in practice is studied through simulation and example.     

Robust Likelihood Cross-Validation for Kernel Density Estimation

Likelihood cross-validation for kernel density estimation is known to be sensitive to extreme observations and heavy-tailed distributions. We propose a robust likelihood-based cross-validation method to select bandwidths in multivariate density estimations. We derive this bandwidth selector within the framework of robust maximum likelihood estimation. This method establishes a smooth transition from likelihood cross-validation for nonextreme observations to least squares cross-validation for extreme observations, thereby combining the efficiency of likelihood cross-validation and the robustness of least-squares cross-validation. We also suggest a simple rule to select the transition threshold. We demonstrate the finite sample performance and practical usefulness of the proposed method via Monte Carlo simulations and a real data application on Chinese air pollution.