Adaptive Nonparametric Comparison of Regression Curves

In this article, we develop a new and novel kernel density estimator for a sum of weighted averages from a single population based on utilizing the well defined kernel density estimator in conjunction with classic inversion theory. This idea is further developed for a kernel density estimator for the difference of weighed averages from two independent populations. The resulting estimator is “bootstrap-like” in terms of its properties with respect to the derivation of approximate confidence intervals via a “plug-in” approach. This new approach is distinct from the bootstrap methodology in that it is analytically and computationally feasible to provide an exact estimate of the distribution function through direct calculation. Thus, our approach eliminates the error due to Monte Carlo resampling that arises within the context of simulation based approaches that are oftentimes necessary in order to derive bootstrap-based confidence intervals for statistics involving weighted averages of i.i.d. random variables. We provide several examples and carry forth a simulation study to show that our kernel density estimator performs better than the standard central limit theorem based approximation in term of coverage probability.     

Lp-estimators in ARCH models

For ergodic ARCH processes, we introduce a one-parameter family of L_p-estimators. The construction is based on the concept of weighted M-estimators. Under weak assumptions on the error distribution, the consistency is established. The asymptotic normality is proved for the special cases p=1 and 2. To prove the asymptotic normality of the L₁-estimator, one needs the existence of a density of the squares of the errors, whereas for the L₂-estimator the existence of fourth moments is assumed. The asymptotic covariance matrix of the estimator depends on the unknown parameter which can be substituted by consistent estimators. For the L₁-estimator we construct a kernel estimator for the unknown density of the square of the errors.     

A semiparametric estimator of the distribution function of a variable measured with error

The estimation of the distribution functon of a random variable X measured with error is studied. Let the i-th observation on X be denoted by Y_iX_i+ε_i where ε_i is the measuremen error. Let {Y_i} (i=1,2,…,n) be a sample of independent observations. It is assumed that {X_i} and {∈_i} are mutually independent and each is identically distributed. As is standard in the literature for this problem, the distribution of e is assumed known in the development of the methodology. In practice, the measurement error distribution is estimated from replicate observations.

The proposed semiparametric estimator is derived by estimating the quantises of X on a set of n transformed V-values and smoothing the estimated quantiles using a spline function. The number of parameters of the spline function is determined by the data with a simple criterion, such as AIC. In a simulation study, the semiparametric estimator dominates an optimal kernel estimator and a normal mixture estimator for a wide class of densities.

The proposed estimator is applied to estimate the distribution function of the mean pH value in a field plot. The density function of the measurement error is estimated from repeated measurements of the pH values in a plot, and is treated as known for the estimation of the distribution function of the mean pH value.     

Deconvolution of supersmooth densities with smooth noise

The author considers the estimation of the common probability density of independent and identically distributed random variables observed with added white noise. She assumes that the unknown density belongs to some class of supersmooth functions, and that the error distribution is ordinarily smooth, meaning that its characteristic function decays polynomially asymptotically. In this context, the author evaluates the minimax rate of convergence of the pointwise risk and describes a kernel estimator having this rate. She computes upper bounds for the L₂ risk of this estimator.     

CORRECTING FOR KURTOSIS IN DENSITY ESTIMATION

Using a global window width kernel estimator to estimate an approximately symmetric probability density with high kurtosis usually leads to poor estimation because good estimation of the peak of the distribution leads to unsatisfactory estimation of the tails and vice versa. The technique proposed corrects for kurtosis via a transformation of the data before using a global window width kernel estimator. The transformation depends on a “generalised smoothing parameter” consisting of two real-valued parameters and a window width parameter which can be selected either by a simple graphical method or, for a completely data-driven implementation, by minimising an estimate of mean integrated squared error. Examples of real and simulated data demonstrate the effectiveness of this approach, which appears suitable for a wide range of symmetric, unimodal densities. Its performance is similar to ordinary kernel estimation in situations where the latter is effective, e.g. Gaussian densities. For densities like the Cauchy where ordinary kernel estimation is not satisfactory, our methodology offers a substantial improvement.     

Density Estimation by Total Variation Penalized Likelihood Driven by the Sparsity ℓ1 Information Criterion

Abstract. We propose a non‐linear density estimator, which is locally adaptive, like wavelet estimators, and positive everywhere, without a log‐ or root‐transform. This estimator is based on maximizing a non‐parametric log‐likelihood function regularized by a total variation penalty. The smoothness is driven by a single penalty parameter, and to avoid cross‐validation, we derive an information criterion based on the idea of universal penalty. The penalized log‐likelihood maximization is reformulated as an ?₁‐penalized strictly convex programme whose unique solution is the density estimate. A Newton‐type method cannot be applied to calculate the estimate because the ?₁‐penalty is non‐differentiable. Instead, we use a dual block coordinate relaxation method that exploits the problem structure. By comparing with kernel, spline and taut string estimators on a Monte Carlo simulation, and by investigating the sensitivity to ties on two real data sets, we observe that the new estimator achieves good L ₁ and L ₂ risk for densities with sharp features, and behaves well with ties.     

A note on recursive smoothers for semimartingales

The kernel function method developed by Yamato (1971) to estimate a probability density function essentially is a way of smoothing the empirical distribution function. This paper shows how one can generalize this method to estimate signals for a semimartingale model. A recursive convolution smoothed estimate is used to obtain an absolutely continuous estimate for an absolutely continuous signal of a semimartingale model. It is also shown that the estimator obtained has a smaller asymptotic variance than the one obtained in Thavaneswaran (1988).     

Asymptotic Unbiased Kernel Estimator for Line Transect Sampling

In this article, we propose a new estimator for the density of objects using line transect data. The proposed estimator combines the nonparametric kernel estimator with parametric detection function: the exponential or the half normal detection function to estimate the density of objects. The selection of the detection function depends on the testing of the shoulder condition assumption. If the shoulder condition is true then the half-normal detection function is introduced together with the kernel estimator. Otherwise, the negative exponential is combined with the kernel estimator. Under these assumptions, the proposed estimator is asymptotically unbiased and it is strongly consistent estimator for the density of objects using line transect data. The simulation results indicate that the proposed estimator is very successful in taking the advantage of the parametric detection function available.     

EMPIRICAL LIKELIHOOD-BASED KERNEL DENSITY ESTIMATION

This paper considers the estimation of a probability density function when extra distributional information is available (e.g. the mean of the distribution is known or the variance is a known function of the mean). The standard kernel method cannot exploit such extra information systematically as it uses an equal probability weight n^-1 at each data point. The paper suggests using empirical likelihood to choose the probability weights under constraints formulated from the extra distributional information. An empirical likelihood-based kernel density estimator is given by replacing n^-1 by the empirical likelihood weights, and has these advantages: it makes systematic use of the extra information, it is able to reflect the extra characteristics of the density function, and its variance is smaller than that of the standard kernel density estimator.     

Minimum local distance density estimation

We present a local density estimator based on first-order statistics. To estimate the density at a point, x, the original sample is divided into subsets and the average minimum sample distance to x over all such subsets is used to define the density estimate at x. The tuning parameter is thus the number of subsets instead of the typical bandwidth of kernel or histogram-based density estimators. The proposed method is similar to nearest-neighbor density estimators but it provides smoother estimates. We derive the asymptotic distribution of this minimum sample distance statistic to study globally optimal values for the number and size of the subsets. Simulations are used to illustrate and compare the convergence properties of the estimator. The results show that the method provides good estimates of a wide variety of densities without changes of the tuning parameter, and that it offers competitive convergence performance.     

Moment density estimation for positive random variables

An unknown moment-determinate cumulative distribution function or its density function can be recovered from corresponding moments and estimated from the empirical moments. This method of estimating an unknown density is natural in certain inverse estimation models like multiplicative censoring or biased sampling when the moments of unobserved distribution can be estimated via the transformed moments of the observed distribution. In this paper, we introduce a new nonparametric estimator of a probability density function defined on the positive real line, motivated by the above. Some fundamental properties of proposed estimator are studied. The comparison with traditional kernel density estimator is discussed.     

Estimation of a probability density function using interval aggregated data

ABSTRACT

In economics and government statistics, aggregated data instead of individual level data are usually reported for data confidentiality and for simplicity. In this paper we develop a method of flexibly estimating the probability density function of the population using aggregated data obtained as group averages when individual level data are grouped according to quantile limits. The kernel density estimator has been commonly applied to such data without taking into account the data aggregation process and has been shown to perform poorly. Our method models the quantile function as an integral of the exponential of a spline function and deduces the density function from the quantile function. We match the aggregated data to their theoretical counterpart using least squares, and regularize the estimation by using the squared second derivatives of the density function as the penalty function. A computational algorithm is developed to implement the method. Application to simulated data and US household income survey data show that our penalized spline estimator can accurately recover the density function of the underlying population while the common use of kernel density estimation is severely biased. The method is applied to study the dynamic of China's urban income distribution using published interval aggregated data of 1985–2010.     

Histogram Model with Plausible Parametric Detection Function for Line Transect Data

This article develops a new model that combines between the histogram and plausible parametric detection function to estimate the population density (abundance) by using line transects technique. A parametric detection function is introduced to improve the properties of the classical histogram estimator. Asymptotic properties of the resulting estimator are derived and an expression for the asymptotic mean square error (AMSE) is given. A general formula for the optimal choice of the histogram bin width based on AMSE is derived. Moreover, other possible alternative procedures to select the bin width are suggested and studied via simulation technique. The results show the superiority of the proposed estimators over both the classical histogram and the usual kernel estimators in most reasonable cases. In addition, the simulation results indicate that the choice of a plausible detection function is less sensitive than the choice of a bin width on the performance of the proposed estimator.     

Characterization of weak convergence for smoothed empirical and quantile processes under ϕ-mixing

Let X_n, n ⩾ 1 be a sequence of ϕ-mixing random variables having a smooth common distribution function F. The smoothed empirical distribution function is obtained by integrating a kernel type density estimator. In this paper we provide necessary and sufficient conditions for the central limit theorem to hold for smoothed empirical distribution functions and smoothed sample quantiles. Also, necessary and sufficient conditions are given for weak convergence of the smoothed empirical process and the smoothed uniform quantile process.     

Inversion Theorem Based Kernel Density Estimation for the Ordinary Least Squares Estimator of a Regression Coefficient

The traditional confidence interval associated with the ordinary least squares estimator of linear regression coefficient is sensitive to non-normality of the underlying distribution. In this article, we develop a novel kernel density estimator for the ordinary least squares estimator via utilizing well-defined inversion based kernel smoothing techniques in order to estimate the conditional probability density distribution of the dependent random variable. Simulation results show that given a small sample size, our method significantly increases the power as compared with Wald-type CIs. The proposed approach is illustrated via an application to a classic small data set originally from Graybill (1961 Graybill, F.A. (1961). Introduction to Linear Statistical Models. Vol. 1. New York: McGraw-Hill Book Company. [Google Scholar]).     

Asymptotic results in gamma kernel regression

Abstract

Based on the Gamma kernel density estimation procedure, this article constructs a nonparametric kernel estimate for the regression functions when the covariate are nonnegative. Asymptotic normality and uniform almost sure convergence results for the new estimator are systematically studied, and the finite performance of the proposed estimate is discussed via a simulation study and a comparison study with an existing method. Finally, the proposed estimation procedure is applied to the Geyser data set.     

Quasi-universal bandwidth selection for kernel density estimators

Let f?_n, h denote the kernel density estimate based on a sample of size n drawn from an unknown density f. Using techniques from L₂ projection density estimators, the author shows how to construct a data-driven estimator f?_n, h which satisfies This paper is inspired by work of Stone (1984), Devroye and Lugosi (1996) and Birge and Massart (1997).     

Estimating Percentiles of Time-to-Failure Distribution Obtained from a Linear Degradation Model Using Kernel Density Method

In this article, we propose a nonparametric estimator for percentiles of the time-to-failure distribution obtained from a linear degradation model using the kernel density method. The properties of the proposed kernel estimator are investigated and compared with well-known maximum likelihood and ordinary least squares estimators via a simulation technique. The mean squared error and the length of the bootstrap confidence interval are used as the basis criteria of the comparisons. The simulation study shows that the performance of the kernel estimator is acceptable as a general estimator. When the distribution of the data is assumed to be known, the maximum likelihood and ordinary least squares estimators perform better than the kernel estimator, while the kernel estimator is superior when the assumption of our knowledge of the data distribution is violated. A comparison among different estimators is achieved using a real data set.     

A Remedy for Kernel Estimation Under Random Design

Two common kernel-based methods for non-parametric regression estimation suffer from well-known drawbacks when the design is random. The Gasser-Müller estimator is inadmissible due to its high variance while the Nadaraya-Watson estimator has zero asymptotic efficiency because of poor bias behavior. Under asymptotic consideration, the local linear estimator avoids these two drawbacks of kernel estimators and achieves minimax optimality. However, when based on compact support kernels its finite sample behavior is disappointing because sudden kinks may show up in the estimate.

This paper proposes a modification of the kernel estimator, called the binned convolution estimator leading to a fast O(n) method. Provided the design density is continously differentiable and the conditional fourth moments exist the binned convolution estimator has asymptotic properties identical with those of the local linear estimator.