Estimating smooth monotone functions

Many situations call for a smooth strictly monotone function f of arbitrary flexibility. The family of functions defined by the differential equation D  ² f  = w Df , where w is an unconstrained coefficient function comprises the strictly monotone twice differentiable functions. The solution to this equation is f = C ₀ + C ₁  D ⁻¹{exp( D ⁻¹ w )}, where C ₀ and C ₁ are arbitrary constants and D ⁻¹ is the partial integration operator. A basis for expanding w is suggested that permits explicit integration in the expression of f . In fitting data, it is also useful to regularize f by penalizing the integral of w ² since this is a measure of the relative curvature in f . Applications are discussed to monotone nonparametric regression, to the transformation of the dependent variable in non-linear regression and to density estimation.     

Likelihood Ratio Tests Under Local and Fixed Alternatives in Monotone Function Problems

Abstract.  We focus on a class of non-standard problems involving non-parametric estimation of a monotone function that is characterized by n ^1/3 rate of convergence of the maximum likelihood estimator, non-Gaussian limit distributions and the non-existence of     -regular estimators. We have shown elsewhere that under a null hypothesis of the type ψ ( z ₀) =  θ ₀ ( ψ being the monotone function of interest) in non-standard problems of the above kind, the likelihood ratio statistic has a 'universal' limit distribution that is free of the underlying parameters in the model. In this paper, we illustrate its limiting behaviour under local alternatives of the form ψ _n ( z ), where ψ _n (·) and ψ (·) vary in O ( n ^−1/3) neighbourhoods around z ₀ and ψ _n converges to ψ at rate n ^1/3 in an appropriate metric. Apart from local alternatives, we also consider the behaviour of the likelihood ratio statistic under fixed alternatives and establish the convergence in probability of an appropriately scaled version of the same to a constant involving a Kullback–Leibler distance.     

Estimating the Upper Support Point in Deconvolution

Abstract.  We consider estimation of the upper boundary point F ⁻¹ (1) of a distribution function F with finite upper boundary or 'frontier' in deconvolution problems, primarily focusing on deconvolution models where the noise density is decreasing on the positive halfline. Our estimates are based on the (non-parametric) maximum likelihood estimator (MLE) of F . We show that (1) is asymptotically never too small. If the convolution kernel has bounded support the estimator (1) can generally be expected to be consistent. In this case, we establish a relation between the extreme value index of F and the rate of convergence of (1) to the upper support point for the 'boxcar' deconvolution model. If the convolution density has unbounded support, (1) can be expected to overestimate the upper support point. We define consistent estimators , for appropriately chosen vanishing sequences ( β _n ) and study these in a particular case.     

Estimation of the Quantile Function of an IFRA Distribution

Let F and G be lifetime distributions and consider the problem of estimating F ⁻¹ when it is known that G ⁻¹ F is star-shaped. Estimators of F ⁻¹ are considered here which are shown to be uniformly strongly consistent. The case of censored data is also presented. Asymptotic confidence intervals and bands for F ⁻¹ are provided. The result are applicable, for example, to the estimation of quantile functions of k -out-of- n systems in reliability. The special case of an IFRA distribution follows immediately from the more general case presented here     

Zero-Bias Locally Adaptive Density Estimators

Strategies for improving fixed non-negative kernel estimators have focused on reducing the bias, either by employing higher-order kernels or by adjusting the bandwidth locally. Intuitively, bandwidths in the tails should be relatively larger in order to reduce wiggles since there is less data available in the tails. We show that in regions where the density function is convex, it is theoretically possible to find local bandwidths such that the pointwise bias is exactly zero. The corresponding pointwise mean squared error converges at the parametric rate of O ( n ⁻¹ ) rather than the slower O ( n ^−4/5). These so-called zero-bias bandwidths are constant and are usually orders of magnitude larger than the optimal locally adaptive bandwidths predicted by asymptotic mean squared error analysis. We describe data-based algorithms for estimating zero-bias bandwidths over intervals where the density is convex. We find that our particular density estimator attains the usual O ( n ^−4/5) rate. However, we demonstrate that the algorithms can provide significant improvement in mean squared error, often clearly visually superior curves, and a new operating point in the usual bias-variance tradeoff.     

The NPMLE for Doubly Censored Current Status Data

In biostatistical applications interest often focuses on the estimation of the distribution of time T between two consecutive events. If the initial event time is observed and the subsequent event time is only known to be larger or smaller than an observed point in time, then the data is described by the well understood singly censored current status model, also known as interval censored data, case I. Jewell et al. (1994) extended this current status model by allowing the initial time to be unobserved, but with its distribution over an observed interval ' A, B ' known to be uniformly distributed; the data is referred to as doubly censored current status data. These authors used this model to handle application in AIDS partner studies focusing on the NPMLE of the distribution G of T . The model is a submodel of the current status model, but the distribution G is essentially the derivative of the distribution of interest F in the current status model. In this paper we establish that the NPMLE of G is uniformly consistent and that the resulting estimators for the n ^1/2-estimable parameters are efficient. We propose an iterative weighted pool-adjacent-violator-algorithm to compute the estimator. It is also shown that, without smoothness assumptions, the NPMLE of F converges at rate n ^−2/5 in L ²-norm while the NPMLE of F in the non-parametric current status data model converges at rate n ^−1/3 in L ²-norm, which shows that there is a substantial gain in using the submodel information.     

An Adaptive Two-stage Estimation Method for Additive Models

Abstract.  In this paper, a two-stage estimation method for non-parametric additive models is investigated. Differing from Horowitz and Mammen's two-stage estimation, our first-stage estimators are designed not only for dimension reduction but also as initial approximations to all of the additive components. The second-stage estimators are obtained by using one-dimensional non-parametric techniques to refine the first-stage ones. From this procedure, we can reveal a relationship between the regression function spaces and convergence rate, and then provide estimators that are optimal in the sense that, better than the usual one-dimensional mean-squared error (MSE) of the order n ^−4/5 , the MSE of the order n ^{− 1} can be achieved when the underlying models are actually parametric. This shows that our estimation procedure is adaptive in a certain sense. Also it is proved that the bandwidth that is selected by cross-validation depends only on one-dimensional kernel estimation and maintains the asymptotic optimality. Simulation studies show that the new estimators of the regression function and all components outperform the existing estimators, and their behaviours are often similar to that of the oracle estimator.     

Some Bounds on the Distribution of Certain Quadratic Forms in Normal Random Variables

The paper derives bounds on the distribution of the quadratic forms Z = y ^H( X Γ X ^H)⁻¹ y and W = y ^H(σ² I + X Γ X ^H)⁻¹ y , where the elements of the M × 1 vector y and the M × N matrix X are independent identically distributed (i.i.d.) complex zero mean Normal variables, Γ is some N × N diagonal matrix with positive diagonal elements, I , is the identity, σ² is a constant and ^H denotes the Hermitian transpose. The bounds are convenient for numerical work and appear to be tight for small values of M . This work has applications in digital mobile radio for a specific channel where M antennas are used to receive a signal with N interferers. Some of these applications in radio communication systems are discussed.     

Kernel Smoothing to Improve Bootstrap Confidence Intervals

Some studies of the bootstrap have assessed the effect of smoothing the estimated distribution that is resampled, a process usually known as the smoothed bootstrap. Generally, the smoothed distribution for resampling is a kernel estimate and is often rescaled to retain certain characteristics of the empirical distribution. Typically the effect of such smoothing has been measured in terms of the mean-squared error of bootstrap point estimates. The reports of these previous investigations have not been encouraging about the efficacy of smoothing. In this paper the effect of resampling a kernel-smoothed distribution is evaluated through expansions for the coverage of bootstrap percentile confidence intervals. It is shown that, under the smooth function model, proper bandwidth selection can accomplish a first-order correction for the one-sided percentile method. With the objective of reducing the coverage error the appropriate bandwidth for one-sided intervals converges at a rate of n ^−1/4, rather than the familiar n ^−1/5 for kernel density estimation. Applications of this same approach to bootstrap t and two-sided intervals yield optimal bandwidths of order n ^−1/2. These bandwidths depend on moments of the smooth function model and not on derivatives of the underlying density of the data. The relationship of this smoothing method to both the accelerated bias correction and the bootstrap t methods provides some insight into the connections between three quite distinct approximate confidence intervals.     

On Rates of Convergence for Bayesian Density Estimation

Abstract.  We consider the problem of estimating a compactly supported density taking a Bayesian nonparametric approach. We define a Dirichlet mixture prior that, while selecting piecewise constant densities, has full support on the Hellinger metric space of all commonly dominated probability measures on a known bounded interval. We derive pointwise rates of convergence for the posterior expected density by studying the speed at which the posterior mass accumulates on shrinking Hellinger neighbourhoods of the sampling density. If the data are sampled from a strictly positive, α -Hölderian density, with α  ∈ ( 0,1] , then the optimal convergence rate n^{− α / (2 α +1)} is obtained up to a logarithmic factor. Smoothing histograms by polygons, a continuous piecewise linear estimator is obtained that for twice continuously differentiable, strictly positive densities satisfying boundary conditions attains a rate comparable up to a logarithmic factor to the convergence rate n ^−4/5 for integrated mean squared error of kernel type density estimators.     

On distribution-weighted partial least squares with diverging number of highly correlated predictors

Summary.  Because highly correlated data arise from many scientific fields, we investigate parameter estimation in a semiparametric regression model with diverging number of predictors that are highly correlated. For this, we first develop a distribution-weighted least squares estimator that can recover directions in the central subspace, then use the distribution-weighted least squares estimator as a seed vector and project it onto a Krylov space by partial least squares to avoid computing the inverse of the covariance of predictors. Thus, distrbution-weighted partial least squares can handle the cases with high dimensional and highly correlated predictors. Furthermore, we also suggest an iterative algorithm for obtaining a better initial value before implementing partial least squares. For theoretical investigation, we obtain strong consistency and asymptotic normality when the dimension p of predictors is of convergence rate O { n ^1/2/ log ( n )} and o ( n ^1/3) respectively where n is the sample size. When there are no other constraints on the covariance of predictors, the rates n ^1/2 and n ^1/3 are optimal. We also propose a Bayesian information criterion type of criterion to estimate the dimension of the Krylov space in the partial least squares procedure. Illustrative examples with a real data set and comprehensive simulations demonstrate that the method is robust to non-ellipticity and works well even in 'small n –large p ' problems.     

Beta-Bernstein Smoothing for Regression Curves with Compact Support

ABSTRACT. The problem of boundary bias is associated with kernel estimation for regression curves with compact support. This paper proposes a simple and uni(r)ed approach for remedying boundary bias in non-parametric regression, without dividing the compact support into interior and boundary areas and without applying explicitly different smoothing treatments separately. The approach uses the beta family of density functions as kernels. The shapes of the kernels vary according to the position where the curve estimate is made. Theyare symmetric at the middle of the support interval, and become more and more asymmetric nearer the boundary points. The kernels never put any weight outside the data support interval, and thus avoid boundary bias. The method is a generalization of classical Bernstein polynomials, one of the earliest methods of statistical smoothing. The proposed estimator has optimal mean integrated squared error at an order of magnitude n ^−4/5, equivalent to that of standard kernel estimators when the curve has an unbounded support.     

Non-parametric Kernel Estimation of the Coefficient of a Diffusion

In this work we exhibit a non-parametric estimator of kernel type, for the diffusion coefficient when one observes a one-dimensional diffusion process at times i / n for i = , ..., n and study its asymptotics as n ←∞. When the diffusion coefficient has regularity r ≥ 1, we obtain a rate 1/ n ^{r /(1+2 r )}, both for pointwise estimation and for estimation on a compact subset of R: this is the same rate as for non-parametric estimation of a density with i.i.d. observations.     

Test inversion bootstrap confidence intervals

In this paper we explore the theoretical and practical implications of using bootstrap test inversion to construct confidence intervals. In the presence of nuisance parameters, we show that the coverage error of such intervals is O ( n ^−1/2) which may be reduced to O ( n ⁻¹) if a Studentized statistic is used. We present three simulation studies and compare the performance of test inversion methods with established methods on the problem of estimating a confidence interval for the dose–response parameter in models of the Japanese atomic bomb survivors data.     

Penalized Projection Estimator for Volatility Density

Abstract.  In this paper, we consider a stochastic volatility model ( Y _t , V _t ), where the volatility (V _t ) is a positive stationary Markov process. We assume that ( ln V _t ) admits a stationary density f that we want to estimate. Only the price process Y _t is observed at n discrete times with regular sampling interval Δ . We propose a non-parametric estimator for f obtained by a penalized projection method. Under mixing assumptions on ( V _t ), we derive bounds for the quadratic risk of the estimator. Assuming that Δ=Δ _n tends to 0 while the number of observations and the length of the observation time tend to infinity, we discuss the rate of convergence of the risk. Examples of models included in this framework are given.     

IMPROVING UPON THE BEST INVARIANT ESTIMATOR IN MULTIVARIATE LOCATION PROBLEMS

We are concerned with estimators which improve upon the best invariant estimator, in estimating a location parameter θ. If the loss function is L(θ - a) with L convex, we give sufficient conditions for the inadmissibility of δ₀(X) = X. If the loss is a weighted sum of squared errors, we find various classes of estimators δ which are better than δ₀. In general, δ is the convolution of δ₁ (an estimator which improves upon δ₀ outside of a compact set) with a suitable probability density in R^p. The critical dimension of inadmissibility depends on the estimator δ₁ We also give several examples of estimators δ obtained in this way and state some open problems.     

Optimal Bootstrap Sample Size in Construction of Percentile Confidence Bounds

In traditional bootstrap applications the size of a bootstrap sample equals the parent sample size, n say. Recent studies have shown that using a bootstrap sample size different from n may sometimes provide a more satisfactory solution. In this paper we apply the latter approach to correct for coverage error in construction of bootstrap confidence bounds. We show that the coverage error of a bootstrap percentile method confidence bound, which is of order O ( n ^−2/2) typically, can be reduced to O ( n ⁻¹) by use of an optimal bootstrap sample size. A simulation study is conducted to illustrate our findings, which also suggest that the new method yields intervals of shorter length and greater stability compared to competitors of similar coverage accuracy.     

Testing a Regression Model When We Have Smooth Alternatives in Mind

Goodness-of-fit tests based on residual sums of squares are standard procedures used when fitting regression models. Often we have a smooth alternative in mind, a qualitative feature that the χ²-test does not take into account. We show that the power of detecting a smooth alternative increases when we smooth the current model as well. The proposed test is shown to be able to detect any continuous local alternative tending to zero slower than n ^−1/2. Theoretical results also address minimax non-parametric hypothesis testing in Sobolev spaces. A simulation study is presented, and the procedure is applied to expenditure curve estimation.     

A TWO-PHASE SAMPLING ESTIMATOR IN SAMPLE SURVEYS

A two-phase sampling estimator of the ratio-type for estimating the mean of a finite population, has been considered where the value of ρC_y/C_x can be guessed or estimated in advance. Here C_y and C_x denote respectively the coefficients of variation of the characteristic under study, y, and the auxiliary characteristic x and ρ denotes the coefficient of correlation between y and x. When the value of ρC_y/C_x is guessed or estimated exactly, the estimator has a smaller large-sample variance compared with either an ordinary ratio estimator or an ordinary linear regression estimator in two-phase sampling in the case where the first-phase sample is drawn independently from the second-phase sample. If the sample at the second phase is a subsample of the first-phase sample, the estimator has variance equal to that of the linear regression estimator. The largest value of the difference between the assumed value and the actual value of ρC_y/C_x has been obtained so as not to result in the variance of the estimator being larger than the variances of either an ordinary ratio estimator or an ordinary linear regression estimator.     

Approximate Representation of Estimators in Constrained Regression Problems

The estimators of inequality-constrained regression problems can be computed by iterative algorithms of mathematical programming, but they do not have analytical expressions in terms of the given data. This situation brings obstacles to further studies on the constrained regression. In this paper we derive approximate representations of the estimators with a remainder of magnitude ( N ⁻¹ log log N )^1/2. From these representations one can clearly see the concrete structure of the estimators of these problems. It will be very helpful for further regression analysis.