Adaptive wavelet estimation of a function from an m-dependent process with possibly unbounded m

The estimation of a multivariate function from a stationary m-dependent process is investigated, with a special focus on the case where m is large or unbounded. We develop an adaptive estimator based on wavelet methods. Under flexible assumptions on the nonparametric model, we prove the good performances of our estimator by determining sharp rates of convergence under two kinds of errors: the pointwise mean squared error and the mean integrated squared error. We illustrate our theoretical result by considering the multivariate density estimation problem, the derivatives density estimation problem, the density estimation problem in a GARCH-type model and the multivariate regression function estimation problem. The performance of proposed estimator has been shown by a numerical study for a simulated and real data sets.     

Estimation of the Spectral Density with Assigned Risk

It is well known that adaptive sequential nonparametric estimation of differentiable functions with assigned mean integrated squared error and minimax expected stopping time is impossible. In other words, no sequential estimator can compete with an oracle estimator that knows how many derivatives an estimated curve has. Differentiable functions are typical in probability density and regression models but not in spectral density models, where considered functions are typically smoother. This paper shows that for a large class of spectral densities, which includes spectral densities of classical autoregressive moving average processes, an adaptive minimax sequential estimation with assigned mean integrated squared error is possible. Furthermore, a two‐stage sequential procedure is proposed, which is minimax and adaptive to smoothness of an underlying spectral density.     

A note on Bayesian nonparametric regression function estimation

In this note the problem of nonparametric regression function estimation in a random design regression model with Gaussian errors is considered from the Bayesian perspective. It is assumed that the regression function belongs to a class of functions with a known degree of smoothness. A prior distribution on the given class can be induced by a prior on the coefficients in a series expansion of the regression function through an orthonormal system. The rate of convergence of the resulting posterior distribution is employed to provide a measure of the accuracy of the Bayesian estimation procedure defined by the posterior expected regression function. We show that the Bayes’ estimator achieves the optimal minimax rate of convergence under mean integrated squared error over the involved class of regression functions, thus being comparable to other popular frequentist regression estimators.     

Optimal Sequential Design in a Controlled Non-parametric Regression

Abstract. In a non‐parametric regression, the heteroscedasticity (dependence of the variance of the regression error on the predictor) can be a serious complication in estimation or visualization of an underlying regression function. If a controlled sampling is permitted, then the statistician can choose the design of predictors which attenuates the effect of heteroscedasticity. It is proposed to use a design which minimizes the mean integrated squared error of the regression function estimation. Then the corresponding optimal design density is proportional to the standard deviation of the regression error (the so‐called scale function). Because in general the statistician does not know an underlying scale function, the natural question is as follows: is it possible to suggest a sequential design which performs as well as an oracle that knows the underlying scale function? The answer is ‘yes’, and a corresponding sequential procedure is developed. It is proved, for the first time in the literature, that a data‐driven sequential design, together with an adaptive regression estimator, can mimic the oracle and be sharp minimax. Further, it is shown that the suggested method is feasible for small samples.     

Adaptive nonparametric estimation in the presence of dependence

We consider nonparametric estimation problems in the presence of dependent data, notably nonparametric regression with random design and nonparametric density estimation. The proposed estimation procedure is based on a dimension reduction. The minimax optimal rate of convergence of the estimator is derived assuming a sufficiently weak dependence characterised by fast decreasing mixing coefficients. We illustrate these results by considering classical smoothness assumptions. However, the proposed estimator requires an optimal choice of a dimension parameter depending on certain characteristics of the function of interest, which are not known in practice. The main issue addressed in our work is an adaptive choice of this dimension parameter combining model selection and Lepski's method. It is inspired by the recent work of Goldenshluger and Lepski [(2011), ‘Bandwidth Selection in Kernel Density Estimation: Oracle Inequalities and Adaptive Minimax Optimality’, The Annals of Statistics, 39, 1608–1632]. We show that this data-driven estimator can attain the lower risk bound up to a constant provided a fast decay of the mixing coefficients.     

Variable kernel density estimation

This paper considers the problem of selecting optimal bandwidths for variable (sample‐point adaptive) kernel density estimation. A data‐driven variable bandwidth selector is proposed, based on the idea of approximating the log‐bandwidth function by a cubic spline. This cubic spline is optimized with respect to a cross‐validation criterion. The proposed method can be interpreted as a selector for either integrated squared error (ISE) or mean integrated squared error (MISE) optimal bandwidths. This leads to reflection upon some of the differences between ISE and MISE as error criteria for variable kernel estimation. Results from simulation studies indicate that the proposed method outperforms a fixed kernel estimator (in terms of ISE) when the target density has a combination of sharp modes and regions of smooth undulation. Moreover, some detailed data analyses suggest that the gains in ISE may understate the improvements in visual appeal obtained using the proposed variable kernel estimator. These numerical studies also show that the proposed estimator outperforms existing variable kernel density estimators implemented using piecewise constant bandwidth functions.     

Nonparametric regression with responses missing at random

For the case of a complete sample of univariate predictors and responses, the modern nonparametric regression matches results known for parametric and semiparametric regressions. The situation changes dramatically if some values in a sample are missing. This paper develops the theory of nonparametric regression for the classical case of responses missing at random. The main conclusion is that an adaptive estimator, based on a complete-case subsample, is asymptotically sharp minimax over all possible oracle-estimators that know: an underlying sample with missing responses; probability of observing the response given the predictor; smoothness of an underlying regression function; design density of the predictor; scale function of the regression error.     

Dilation-Invariant Wavelet Estimation

A wavelet method is proposed that reduces function estimation error and provides smooth reconstructions, while still estimating jumps in the function well. It is based on analyzing multiple dilated versions of the sampled function. In simulation studies, the estimator exhibits low mean squared errors without sacrificing smoothness or jump detection ability when compared to other wavelet methods.     

Nonparametric density estimation based on beta prime kernel

Abstract

In this work, we propose beta prime kernel estimator for estimation of a probability density functions defined with nonnegative support. For the proposed estimator, beta prime probability density function used as a kernel. It is free of boundary bias and nonnegative with a natural varying shape. We obtained the optimal rate of convergence for the mean squared error (MSE) and the mean integrated squared error (MISE). Also, we use adaptive Bayesian bandwidth selection method with Lindley approximation for heavy tailed distributions and compare its performance with the global least squares cross-validation bandwidth selection method. Simulation studies are performed to evaluate the average integrated squared error (ISE) of the proposed kernel estimator against some asymmetric competitors using Monte Carlo simulations. Moreover, real data sets are presented to illustrate the findings.     

Minimax Rates for Estimating the Variance and its Derivatives in Non–Parametric Regression

In this paper the van Trees inequality is applied to obtain lower bounds for the quadratic risk of estimators for the variance function and its derivatives in non–parametric regression models. This approach yields a much simpler proof compared to previously applied methods for minimax rates. Furthermore, the informative properties of the van Trees inequality reveal why the optimal rates for estimating the variance are not affected by the smoothness of the signal g . A Fourier series estimator is constructed which achieves the optimal rates. Finally, a second–order correction is derived which suggests that the initial estimator of g must be undersmoothed for the estimation of the variance.     

Adaptive Estimation Using Weighted Least Squares

This paper proposes an adaptive estimator that is more precise than the ordinary least squares estimator if the distribution of random errors is skewed or has long tails. The adaptive estimates are computed using a weighted least squares approach with weights based on the lengths of the tails of the distribution of residuals. Smaller weights are assigned to those observations that have residuals in the tails of long-tailed distributions and larger weights are assigned to observations having residuals in the tails of short-tailed distributions. Monte Carlo methods are used to compare the performance of the proposed estimator and the performance of the ordinary least squares estimator. The estimates that were studied in this simulation include the difference between the means of two populations, the mean of a symmetric distribution, and the slope of a regression line. The adaptive estimators are shown to have lower mean squared errors than those for the ordinary least squares estimators for short-tailed, long-tailed, and skewed distributions, provided the sample size is at least 20. The ordinary least squares estimator has slightly lower mean squared error for normally distributed errors. The adaptive estimator is recommended for general use for studies having sample sizes of at least 20 observations unless the random errors are known to be normally distributed.     

Note on the minimum mean integrated squared error of kernel estimates of a distribution function and its derivatives

An exact expiession for the minimum integrated squared error associated with the kernel distribution function and its derivatives is given. Furthermore, the virtual optimality of the Fourier integral estimate in density estimation, shown by Davis (1977), is extended to estimation of a distibution function and its derivatives.     

Robust signed-rank estimation and variable selection for semi-parametric additive partial linear models

A fully nonparametric model may not perform well or when the researcher wants to use a parametric model but the functional form with respect to a subset of the regressors or the density of the errors is not known. This becomes even more challenging when the data contain gross outliers or unusual observations. However, in practice the true covariates are not known in advance, nor is the smoothness of the functional form. A robust model selection approach through which we can choose the relevant covariates components and estimate the smoothing function may represent an appealing tool to the solution. A weighted signed-rank estimation and variable selection under the adaptive lasso for semi-parametric partial additive models is considered in this paper. B-spline is used to estimate the unknown additive nonparametric function. It is shown that despite using B-spline to estimate the unknown additive nonparametric function, the proposed estimator has an oracle property. The robustness of the weighted signed-rank approach for data with heavy-tail, contaminated errors, and data containing high-leverage points are validated via finite sample simulations. A practical application to an economic study is provided using an updated Canadian household gasoline consumption data.     

On MISE of a Non linear Wavelet Estimator of the Regression Function Based on Biased Data under Strong Mixing

In this paper, we consider the adaptation of the non linear wavelet-based estimator of the regression function for the biased data setup under strong mixing. We provide an asymptotic sharp bound for the mean integrated squared error (MISE) of the estimator, that is nearly optimal in the minimax sense over a large range of Besov function classes.     

Adaptive wavelet estimation of a density from mixtures under multiplicative censoring

In this paper, a mixture model under multiplicative censoring is considered. We investigate the estimation of a component of the mixture (a density) from the observations. A new adaptive estimator based on wavelets and a hard thresholding rule is constructed for this problem. Under mild assumptions on the model, we study its asymptotic properties by determining an upper bound of the mean integrated squared error over a wide range of Besov balls. We prove that the obtained upper bound is sharp.     

Comparison of estimators for heteroskedastic models

Ordinary least squares (OLS) yield inefficient parameter estimates and inconsistent estimates of the covariance matrix in case of heteroskedastic errors. Robinson's adaptive estimator and the Cragg estimator avoid any explicit parameterization of heteroskedasticity, and reduce the danger of misspecification. A small Monte Carlo experiment is performed to compare the behavior of the adaptive estimator with the performance of the Cragg estimator. The Monte Carlo experiment includes simulations of the Generalized Least Squares (GLS) estimator. Indeed, an interesting question is how more sophisticated techniques, like the adaptive estimator, compare with GLS when the latter relies on an incorrect specification of the heteroskedastic process. It turns out that the regression parameters, when estimated adaptively, display small mean squared errors and great efficiency in case of medium or high heteroskedasticity. The covariance matrix, instead, is better estimated by the Cragg estimator or by GLS based on a misspecified error term, since the adaptive estimator overpredicts the standard errors of the regression parameters.     

On the pointwise mean squared error of a multidimensional term-by-term thresholding wavelet estimator

In this paper we provide a theoretical contribution to the pointwise mean squared error of an adaptive multidimensional term-by-term thresholding wavelet estimator. A general result exhibiting fast rates of convergence under mild assumptions on the model is proved. It can be applied for a wide range of non parametric models including possible dependent observations. We give applications of this result for the non parametric regression function estimation problem (with random design) and the conditional density estimation problem.     

SEMIPARAMETRIC COMPARISON OF REGRESSION CURVES VIA NORMAL LIKELIHOODS

Härdle & Marron (1990) treated the problem of semiparametric comparison of nonparametric regression curves by proposing a kernel-based estimator derived by minimizing a version of weighted integrated squared error. The resulting estimators of unknown transformation parameters are n-consistent, which prompts a consideration of issues. of optimality. We show that when the unknown mean function is periodic, an optimal nonparametric estimator may be motivated by an elegantly simple argument based on maximum likelihood estimation in a parametric model with normal errors. Strikingly, the asymptotic variance of an optimal estimator of θ does not depend at all on the manner of estimating error variances, provided they are estimated n-consistently. The optimal kernel-based estimator derived via these considerations is asymptotically equivalent to a periodic version of that suggested by Härdle & Marron, and so the latter technique is in fact optimal in this sense. We discuss the implications of these conclusions for the aperiodic case.     

Estimation of a probability density function using interval aggregated data

ABSTRACT

In economics and government statistics, aggregated data instead of individual level data are usually reported for data confidentiality and for simplicity. In this paper we develop a method of flexibly estimating the probability density function of the population using aggregated data obtained as group averages when individual level data are grouped according to quantile limits. The kernel density estimator has been commonly applied to such data without taking into account the data aggregation process and has been shown to perform poorly. Our method models the quantile function as an integral of the exponential of a spline function and deduces the density function from the quantile function. We match the aggregated data to their theoretical counterpart using least squares, and regularize the estimation by using the squared second derivatives of the density function as the penalty function. A computational algorithm is developed to implement the method. Application to simulated data and US household income survey data show that our penalized spline estimator can accurately recover the density function of the underlying population while the common use of kernel density estimation is severely biased. The method is applied to study the dynamic of China's urban income distribution using published interval aggregated data of 1985–2010.     

On finite-sample properties of adaptive least squares regression estimates

This paper is mainly concerned with deriving finite-sample properties of least squares estimators for the regression function in a nonparametric regression situation under some simplifying assumptions such as normally distributed errors with a common known variance. The selection of basis functions to be used for the construction of an estimator may be regarded as a smoothing problem, and will usually be done in a data-dependent way, A straightforward application of a result by P. J. Kernpthorne yields that, under a squared error loss, all selection procedures are admissible. Furthermore, the minimax approach provides an interpolating estimator, which is often impractical, Therefore, within a certain class of selection procedures an optimal one is determined using the minimax regret principle. It can be seen to behave similarly to the procedure minimizing either an unbiased risk estimator or, equivalently, the C_p-criterion.