Nonparametric estimation of a two dimensional continuous–discrete density function by wavelets

We consider the estimation of a two dimensional continuous–discrete density function. A new methodology based on wavelets is proposed. We construct a linear wavelet estimator and a non-linear wavelet estimator based on a term-by-term thresholding. Their rates of convergence are established under the mean integrated squared error over Besov balls. In particular, we prove that our adaptive wavelet estimator attains a fast rate of convergence. A simulation study illustrates the usefulness of the proposed estimators.     

Adaptive nonparametric empirical Bayes estimation via wavelet series: The minimax study

In the present paper, we derive lower bounds for the risk of the nonparametric empirical Bayes estimators. In order to attain the optimal convergence rate, we propose generalization of the linear empirical Bayes estimation method which takes advantage of the flexibility of the wavelet techniques. We present an empirical Bayes estimator as a wavelet series expansion and estimate coefficients by minimizing the prior risk of the estimator. As a result, estimation of wavelet coefficients requires solution of a well-posed low-dimensional sparse system of linear equations. The dimension of the system depends on the size of wavelet support and smoothness of the Bayes estimator. An adaptive choice of the resolution level is carried out using Lepski et al. (1997) method. The method is computationally efficient and provides asymptotically optimal adaptive EB estimators. The theory is supplemented by numerous examples.     

A new Bayesian wavelet thresholding estimator of nonparametric regression

The methods of estimation of nonparametric regression function are quite common in statistical application. In this paper, the new Bayesian wavelet thresholding estimation is considered. The new mixture prior distributions for the estimation of nonparametric regression function by applying wavelet transformation are investigated. The reversible jump algorithm to obtain the appropriate prior distributions and value of thresholding is used. The performance of the proposed estimator is assessed with simulated data from well-known test functions by comparing the convergence rate of the proposed estimator with respect to another by evaluating the average mean square error and standard deviations. Finally by applying the developed method, density function of galaxy data is estimated.     

Nonparametric Regression with Sample Design Following a Random Process

Nonparametric regression is considered where the sample point placement is not fixed and equispaced, but generated by a random process with rate n. Conditions are found for the random processes that result in optimal rates of convergence for nonparametric regression when using a block thresholded wavelet estimator. Previous results on nonparametric regression via wavelets on both fixed and random sample point placement are shown to be special cases of the general result given here. The estimator is adaptive over a large range of Hölder function spaces and the convergence rate exhibited is an improvement over term-by-term wavelet estimators. Threshold selection is implemented in a data-adaptive fashion, rather than using a fixed threshold as is usually done in block thresholding. This estimator, BlockSure, is compared against fixed-threshold block estimators and the more traditional term-by-term threshold wavelet estimators on several random design schemes via simulations.     

Wavelet Estimator in Nonparametric Regression Model with Dependent Error’s Structure

In this article, we consider a nonparametric regression model with replicated observations based on the dependent error’s structure, for exhibiting dependence among the units. The wavelet procedures are developed to estimate the regression function. The moment consistency, the strong consistency, strong convergence rate and asymptotic normality of wavelet estimator are established under suitable conditions. A simulation study is undertaken to assess the finite sample performance of the proposed method.     

Instrumental variable based variable selection for generalized linear models with endogenous covariates

We study the variable selection problem for a class of generalized linear models with endogenous covariates. Based on the instrumental variable adjustment technology and the smooth-threshold estimating equation (SEE) method, we propose an instrumental variable based variable selection procedure. The proposed variable selection method can attenuate the effect of endogeneity in covariates, and is easy for application in practice. Some theoretical results are also derived such as the consistency of the proposed variable selection procedure and the convergence rate of the resulting estimator. Further, some simulation studies and a real data analysis are conducted to evaluate the performance of the proposed method, and simulation results show that the proposed method is workable.     

A trimmed translation-invariant denoising estimator

A popular wavelet method for estimating jumps in functions is through the use of the translation-invariant (TI) estimator. The TI estimator addresses a particular problem, the susceptibility of the wavelet estimates to the location of the features in a function with respect to the support of the wavelet basis functions. The TI estimator reduces this reliance by cycling the data through a set of shifts, thus changing the relation between the wavelet support and the jump location. However, a drawback of the TI estimator is that it includes every shifted analysis in the reconstruction, even those that may reduce, rather than improve, the effectiveness of the method. In this paper, we propose a method that modifies the TI estimator to improve the jump reconstruction in terms of the mean squared errors of the reconstructions and visual performance. Information from the set of shifted data sets is used to mimic the performance of an oracle which knows exactly which are the best TI shifts to retain in the reconstruction. The TI estimate is a special case of the proposed method. A simulation study comparing this proposed method to the existing wavelet estimators and the oracle is provided.     

Log-density Deconvolution by Wavelet Thresholding

Abstract.  This paper proposes a new wavelet-based method for deconvolving a density. The estimator combines the ideas of non-linear wavelet thresholding with periodized Meyer wavelets and estimation by information projection. It is guaranteed to be in the class of density functions, in particular it is positive everywhere by construction. The asymptotic optimality of the estimator is established in terms of the rate of convergence of the Kullback–Leibler discrepancy over Besov classes. Finite sample properties are investigated in detail, and show the excellent empirical performance of the estimator, compared with other recently introduced estimators.     

Wavelet estimation of density for censored data with censoring indicator missing at random

In this paper, we define the nonlinear wavelet estimator of density for the right censoring model with the censoring indicator missing at random (MAR), and develop its asymptotic expression for mean integrated squared error (MISE). Unlike for kernel estimator, the MISE expression of the estimator is not affected by the presence of discontinuities in the curve. Meanwhile, asymptotic normality of the estimator is established. The proposed estimator can reduce to the estimator defined by Li [Non-linear wavelet-based density estimators under random censorship. J Statist Plann Inference. 2003;117(1):35–58] when the censoring indicator MAR does not occur and a bandwidth in non-parametric estimation is close to zero. Also, we define another two nonlinear wavelet estimators of the density. A simulation is done to show the performance of the three proposed estimators.     

Adaptive nonparametric estimation in the presence of dependence

We consider nonparametric estimation problems in the presence of dependent data, notably nonparametric regression with random design and nonparametric density estimation. The proposed estimation procedure is based on a dimension reduction. The minimax optimal rate of convergence of the estimator is derived assuming a sufficiently weak dependence characterised by fast decreasing mixing coefficients. We illustrate these results by considering classical smoothness assumptions. However, the proposed estimator requires an optimal choice of a dimension parameter depending on certain characteristics of the function of interest, which are not known in practice. The main issue addressed in our work is an adaptive choice of this dimension parameter combining model selection and Lepski's method. It is inspired by the recent work of Goldenshluger and Lepski [(2011), ‘Bandwidth Selection in Kernel Density Estimation: Oracle Inequalities and Adaptive Minimax Optimality’, The Annals of Statistics, 39, 1608–1632]. We show that this data-driven estimator can attain the lower risk bound up to a constant provided a fast decay of the mixing coefficients.     

Multinomial Probability Estimation by Wavelet Thresholding

In this article, we introduce a wavelet threshold estimator to estimate multinomial probabilities. The advantages of the estimator are its adaptability to the roughness and sparseness of the data. The asymptotic behavior of the estimator is investigated through an often-used criteria: the mean sum of squared error (MSSE). We show that the MSSE of the estimator achieves the optimal rate of convergence. Its performance on finite samples is examined through simulation studies which show favorable results for the new estimator over the commonly used kernel estimator.     

A shift parameter estimation based on smoothed Kolmogorov–Smirnov

A new procedure of shift parameter estimation in the two-sample location problem is investigated and compared with existing estimators. The proposed procedure smooths the empirical distribution functions of each random sample and replaces empirical distribution functions in the two-sample Kolmogorov–Smirnov method. The smoothed Kolmogorov–Smirnov is minimized with respect to an arbitrary shift variable in order to find an estimate of the shift parameter. The proposed procedure can be considered the smoothed version of a very little known method of shift parameter estimation from Rao-Schuster-Littell (RSL) [Rao et al., Estimation of shift and center of symmetry based on Kolmogorov–Smirnov statistics, Ann. Stat. 3(4) (1975), pp. 862–873]. Their estimator will be discussed and compared with the proposed estimator in this paper. An example and simulation studies have been performed to compare the proposed procedure with existing shift parameter estimators such as Hodges–Lehmann (H–L) and least squares in addition to RSL's estimator. The results show that the proposed estimator has lower mean-squared error as well as higher relative efficiency against RSL's estimator under normal or contaminated normal model assumptions. Moreover, the proposed estimator performs competitively against H–L and least-squares shift estimators. Smoother function and bandwidth selections are also discussed and several alternatives are proposed in the study.     

Block thresholding wavelet regression using SCAD penalty

This paper concerns wavelet regression using a block thresholding procedure. Block thresholding methods utilize neighboring wavelet coefficients information to increase estimation accuracy. We propose to construct a data-driven block thresholding procedure using the smoothly clipped absolute deviation (SCAD) penalty. A simulation study demonstrates competitive finite sample performance of the proposed estimator compared to existing methods. We also show that the proposed estimator achieves optimal convergence rates in Besov spaces.     

Nonparametric curve estimation with missing data: A general empirical process approach

A general nonparametric imputation procedure, based on kernel regression, is proposed to estimate points as well as set- and function-indexed parameters when the data are missing at random (MAR). The proposed method works by imputing a specific function of a missing value (and not the missing value itself), where the form of this specific function is dictated by the parameter of interest. Both single and multiple imputations are considered. The associated empirical processes provide the right tool to study the uniform convergence properties of the resulting estimators. Our estimators include, as special cases, the imputation estimator of the mean, the estimator of the distribution function proposed by Cheng and Chu [1996. Kernel estimation of distribution functions and quantiles with missing data. Statist. Sinica 6, 63–78], imputation estimators of a marginal density, and imputation estimators of regression functions.     

Super efficient frequency estimation

In this paper we propose a modified Newton-Raphson method to obtain super efficient estimators of the frequencies of a sinusoidal signal in presence of stationary noise. It is observed that if we start from an initial estimator with convergence rate O_p(n⁻¹) and use Newton-Raphson algorithm with proper step factor modification, then it produces super efficient frequency estimator in the sense that its asymptotic variance is lower than the asymptotic variance of the corresponding least squares estimator. The proposed frequency estimator is consistent and it has the same rate of convergence, namely O_p(n^−3/2), as the least squares estimator. Monte Carlo simulations are performed to observe the performance of the proposed estimator for different sample sizes and for different models. The results are quite satisfactory. One real data set has been analyzed for illustrative purpose.     

Estimation for the censored partially linear quantile regression models

In this article, we develop estimation procedures for partially linear quantile regression models, where some of the responses are censored by another random variable. The nonparametric function is estimated by basis function approximations. The estimation procedure is easy to implement through existing weighted quantile regression, and it requires no specification of the error distributions. We show the large-sample properties of the resulting estimates, the proposed estimator of the regression parameter is root-n consistent and asymptotically normal and the estimator of the functional component achieves the optimal convergence rate of the nonparametric function. The proposed method is studied via simulations and illustrated with the analysis of a primary biliary cirrhosis (BPC) data.     

M-estimation for functional linear regression

This paper studies M-estimation in functional linear regression in which the dependent variable is scalar while the covariate is a function. An estimator for the slope function is obtained based on the functional principal component basis. The global convergence rate of the M-estimator of unknown slope function is established. The convergence rate of the mean-squared prediction error for the proposed estimators is also established. Monte Carlo simulation studies are conducted to examine the finite-sample performance of the proposed procedure. Finally, the proposed method is applied to analyze the Berkeley growth data.     

Non-parametric Estimation for the Location of a Change-point in an Otherwise Smooth Hazard Function under Random Censoring

A non-parametric wavelet based estimator is proposed for the location of a change-point in an otherwise smooth hazard function under non-informative random right censoring. The proposed estimator is based on wavelet coefficients differences via an appropriate parametrization of the time-frequency plane. The study of the estimator is facilitated by the strong representation theorem for the Kaplan–Meier estimator established by Lo and Singh (1986). The performance of the estimator is checked via simulations and two real examples conclude the paper.     

Series estimation of functional-coefficient partially linear regression model

This paper develops an alternative and complement estimation procedure for functional coefficient partially linear regression (FCPLR) model based on series method. We derive the convergence rates and asymptotic normality of the proposed estimator. We examine its finite sample performance and compare it with the two-step local linear estimator via a small scale Monte Carlo simulation.     

A Non‐Parametric Estimator of the Spectral Density of a Continuous‐Time Gaussian Process Observed at Random Times

Abstract. In numerous applications data are observed at random times and an estimated graph of the spectral density may be relevant for characterizing and explaining phenomena. By using a wavelet analysis, one derives a non‐parametric estimator of the spectral density of a Gaussian process with stationary increments (or a stationary Gaussian process) from the observation of one path at random discrete times. For every positive frequency, this estimator is proved to satisfy a central limit theorem with a convergence rate depending on the roughness of the process and the moment of random durations between successive observations. In the case of stationary Gaussian processes, one can compare this estimator with estimators based on the empirical periodogram. Both estimators reach the same optimal rate of convergence, but the estimator based on wavelet analysis converges for a different class of random times. Simulation examples and an application to biological data are also provided.