Likelihood Ratio Tests Under Local and Fixed Alternatives in Monotone Function Problems

Abstract.  We focus on a class of non-standard problems involving non-parametric estimation of a monotone function that is characterized by n ^1/3 rate of convergence of the maximum likelihood estimator, non-Gaussian limit distributions and the non-existence of     -regular estimators. We have shown elsewhere that under a null hypothesis of the type ψ ( z ₀) =  θ ₀ ( ψ being the monotone function of interest) in non-standard problems of the above kind, the likelihood ratio statistic has a 'universal' limit distribution that is free of the underlying parameters in the model. In this paper, we illustrate its limiting behaviour under local alternatives of the form ψ _n ( z ), where ψ _n (·) and ψ (·) vary in O ( n ^−1/3) neighbourhoods around z ₀ and ψ _n converges to ψ at rate n ^1/3 in an appropriate metric. Apart from local alternatives, we also consider the behaviour of the likelihood ratio statistic under fixed alternatives and establish the convergence in probability of an appropriately scaled version of the same to a constant involving a Kullback–Leibler distance.     

Consistency of Bernstein polynomial posteriors

A Bernstein prior is a probability measure on the space of all the distribution functions on [0, 1]. Under very general assumptions, it selects absolutely continuous distribution functions, whose densities are mixtures of known beta densities. The Bernstein prior is of interest in Bayesian nonparametric inference with continuous data. We study the consistency of the posterior from a Bernstein prior. We first show that, under mild assumptions, the posterior is weakly consistent for any distribution function P ₀ on [0, 1] with continuous and bounded Lebesgue density. With slightly stronger assumptions on the prior, the posterior is also Hellinger consistent. This implies that the predictive density from a Bernstein prior, which is a Bayesian density estimate, converges in the Hellinger sense to the true density (assuming that it is continuous and bounded). We also study a sieve maximum likelihood version of the density estimator and show that it is also Hellinger consistent under weak assumptions. When the order of the Bernstein polynomial, i.e. the number of components in the beta distribution mixture, is truncated, we show that under mild restrictions the posterior concentrates on the set of pseudotrue densities. Finally, we study the behaviour of the predictive density numerically and we also study a hybrid Bayes–maximum likelihood density estimator.     

Estimating smooth monotone functions

Many situations call for a smooth strictly monotone function f of arbitrary flexibility. The family of functions defined by the differential equation D  ² f  = w Df , where w is an unconstrained coefficient function comprises the strictly monotone twice differentiable functions. The solution to this equation is f = C ₀ + C ₁  D ⁻¹{exp( D ⁻¹ w )}, where C ₀ and C ₁ are arbitrary constants and D ⁻¹ is the partial integration operator. A basis for expanding w is suggested that permits explicit integration in the expression of f . In fitting data, it is also useful to regularize f by penalizing the integral of w ² since this is a measure of the relative curvature in f . Applications are discussed to monotone nonparametric regression, to the transformation of the dependent variable in non-linear regression and to density estimation.     

On the Distance Between Cumulative Sum Diagram and Its Greatest Convex Minorant for Unequally Spaced Design Points

Abstract.  The supremum difference between the cumulative sum diagram, and its greatest convex minorant (GCM), in case of non-parametric isotonic regression is considered. When the regression function is strictly increasing, and the design points are unequally spaced, but approximate a positive density in even a slow rate ( n ^−1/3), then the difference is shown to shrink in a very rapid (close to n ^−2/3) rate. The result is analogous to the corresponding result in case of a monotone density estimation established by Kiefer and Wolfowitz, but uses entirely different representation. The limit distribution of the GCM as a process on the unit interval is obtained when the design variables are i.i.d. with a positive density. Finally, a pointwise asymptotic normality result is proved for the smooth monotone estimator, obtained by the convolution of a kernel with the classical monotone estimator.     

Non-parametric Regression with Dependent Censored Data

Abstract.  Let ( X _i , Y _i ) ( i = 1 ,…, n ) be n replications of a random vector ( X , Y  ), where Y is supposed to be subject to random right censoring. The data ( X _i , Y _i ) are assumed to come from a stationary α -mixing process. We consider the problem of estimating the function m ( x ) = E ( φ ( Y ) |  X = x ), for some known transformation φ . This problem is approached in the following way: first, we introduce a transformed variable     , that is not subject to censoring and satisfies the relation     , and then we estimate m ( x ) by applying local linear regression techniques. As a by-product, we obtain a general result on the uniform rate of convergence of kernel type estimators of functionals of an unknown distribution function, under strong mixing assumptions.     

Estimating the Upper Support Point in Deconvolution

Abstract.  We consider estimation of the upper boundary point F ⁻¹ (1) of a distribution function F with finite upper boundary or 'frontier' in deconvolution problems, primarily focusing on deconvolution models where the noise density is decreasing on the positive halfline. Our estimates are based on the (non-parametric) maximum likelihood estimator (MLE) of F . We show that (1) is asymptotically never too small. If the convolution kernel has bounded support the estimator (1) can generally be expected to be consistent. In this case, we establish a relation between the extreme value index of F and the rate of convergence of (1) to the upper support point for the 'boxcar' deconvolution model. If the convolution density has unbounded support, (1) can be expected to overestimate the upper support point. We define consistent estimators , for appropriately chosen vanishing sequences ( β _n ) and study these in a particular case.     

Penalized Projection Estimator for Volatility Density

Abstract.  In this paper, we consider a stochastic volatility model ( Y _t , V _t ), where the volatility (V _t ) is a positive stationary Markov process. We assume that ( ln V _t ) admits a stationary density f that we want to estimate. Only the price process Y _t is observed at n discrete times with regular sampling interval Δ . We propose a non-parametric estimator for f obtained by a penalized projection method. Under mixing assumptions on ( V _t ), we derive bounds for the quadratic risk of the estimator. Assuming that Δ=Δ _n tends to 0 while the number of observations and the length of the observation time tend to infinity, we discuss the rate of convergence of the risk. Examples of models included in this framework are given.     

On a Posterior Predictive Density Sample Size Criterion

Abstract.  Let Ω be a space of densities with respect to some σ -finite measure μ and let Π be a prior distribution having support Ω with respect to some suitable topology. Conditional on f , let X ⁿ  = ( X ₁ ,…, X _n ) be an independent and identically distributed sample of size n from f . This paper introduces a Bayesian non-parametric criterion for sample size determination which is based on the integrated squared distance between posterior predictive densities. An expression for the sample size is obtained when the prior is a Dirichlet mixture of normal densities.     

ON THE CONVERGENCE OF EMPIRICAL PROCESSES OF MIXING VARIABLES1

Conditions are given for the weak convergence of (t—t²)L_N(a_N^-1( t )) to a Gaussian process where v<1/2, a _N is a cdf and L _N is the normalized weighted empirical cumulative distribution function (cdf) for an α-mixing sample of random variables in R which may be non-stationary with discontinuous marginals.     

Bayesian model selection for join point regression with application to age-adjusted cancer rates

Summary.  The method of Bayesian model selection for join point regression models is developed. Given a set of K +1 join point models M ₀,  M ₁, …,  M _K with 0, 1, …,  K join points respec-tively, the posterior distributions of the parameters and competing models M _k are computed by Markov chain Monte Carlo simulations. The Bayes information criterion BIC is used to select the model M _k with the smallest value of BIC as the best model. Another approach based on the Bayes factor selects the model M _k with the largest posterior probability as the best model when the prior distribution of M _k is discrete uniform. Both methods are applied to analyse the observed US cancer incidence rates for some selected cancer sites. The graphs of the join point models fitted to the data are produced by using the methods proposed and compared with the method of Kim and co-workers that is based on a series of permutation tests. The analyses show that the Bayes factor is sensitive to the prior specification of the variance σ ², and that the model which is selected by BIC fits the data as well as the model that is selected by the permutation test and has the advantage of producing the posterior distribution for the join points. The Bayesian join point model and model selection method that are presented here will be integrated in the National Cancer Institute's join point software ( http://www.srab.cancer.gov/joinpoint/ ) and will be available to the public.     

Estimation and testing stationarity for double-autoregressive models

Summary.  The paper considers the double-autoregressive model y _t  =  φ y _{t −1}+ ɛ _t with ɛ _t  =     . Consistency and asymptotic normality of the estimated parameters are proved under the condition E  ln | φ  +√ α η _t |<0, which includes the cases with | φ |=1 or | φ |>1 as well as     . It is well known that all kinds of estimators of φ in these cases are not normal when ɛ _t are independent and identically distributed. Our result is novel and surprising. Two tests are proposed for testing stationarity of the model and their asymptotic distributions are shown to be a function of bivariate Brownian motions. Critical values of the tests are tabulated and some simulation results are reported. An application to the US 90-day treasury bill rate series is given.     

Computation of the Generalized F Distribution

Exact expressions for the cumulative distribution function of a random variable of the form ( α ₁ X ₁+ α ₂ X ₂)/ Y are given where X ₁, X ₂ and Y are independent chi-squared random variables. The expressions are applied to the detection of joint outliers and Hotelling's mis-specified T ² distribution.     

MINIMAX ESTIMATION OF A MULTIVARIATE NORMAL MEAN UNDER A CONVEX LOSS FUNCTION

Let X = (X₁, - X_p)prime; ˜ N_p (μ, Σ) where μ= (μ₁, -, μ_p)' and Σ= diag (Σ²₁, -, Σ²_p) are both unknown and p3. Let (n_i - 2) w_i/Σ²_i! X²_ni, independent. of w_i (I ≠ j = 1, -, p). Assume that (w₁, -, w_p) and X are independent. Define W = diag (w₁, -, w_p) and ¶ X ¶²_w= X'W^-1Q^-1W^-1X where Q = diag (q₁, -,n q_p), q_i > 0, i = 1, -, p. In this paper, the minimax estimator of Berger & Bock (1976), given by δ (X, W) = [I_p - r(X, W) ¶ X ¶^-2_w Q^-1W^-1] X, is shown to be minimax relative to the convex loss (δ - μ)'[αQ + (1 - α) Σ^-1] δ - μ)/C, where C =α tr (Σ) + (1 - α)p and 0 α 1, under certain conditions on r(X, W). This generalizes the above mentioned result of Berger & Bock.     

Consistency of the Semi-parametric MLE in Linear Regression Models with Interval-censored Data

Abstract.  Consider the model Y = β ^' X + ε . Let F ₀ be the unknown cumulative distribution function of the random variable ε . Consistency of the semi-parametric Maximum likelihood estimator of ( β , F ₀), denoted by     , has not been established under any interval censorship (IC) model. We prove in this paper that     is consistent under the mixed case IC model and some mild assumptions.     

On distribution-weighted partial least squares with diverging number of highly correlated predictors

Summary.  Because highly correlated data arise from many scientific fields, we investigate parameter estimation in a semiparametric regression model with diverging number of predictors that are highly correlated. For this, we first develop a distribution-weighted least squares estimator that can recover directions in the central subspace, then use the distribution-weighted least squares estimator as a seed vector and project it onto a Krylov space by partial least squares to avoid computing the inverse of the covariance of predictors. Thus, distrbution-weighted partial least squares can handle the cases with high dimensional and highly correlated predictors. Furthermore, we also suggest an iterative algorithm for obtaining a better initial value before implementing partial least squares. For theoretical investigation, we obtain strong consistency and asymptotic normality when the dimension p of predictors is of convergence rate O { n ^1/2/ log ( n )} and o ( n ^1/3) respectively where n is the sample size. When there are no other constraints on the covariance of predictors, the rates n ^1/2 and n ^1/3 are optimal. We also propose a Bayesian information criterion type of criterion to estimate the dimension of the Krylov space in the partial least squares procedure. Illustrative examples with a real data set and comprehensive simulations demonstrate that the method is robust to non-ellipticity and works well even in 'small n –large p ' problems.     

Wavelet deconvolution in a periodic setting

Summary.  Deconvolution problems are naturally represented in the Fourier domain, whereas thresholding in wavelet bases is known to have broad adaptivity properties. We study a method which combines both fast Fourier and fast wavelet transforms and can recover a blurred function observed in white noise with O { n    log ( n )²} steps. In the periodic setting, the method applies to most deconvolution problems, including certain 'boxcar' kernels, which are important as a model of motion blur, but having poor Fourier characteristics. Asymptotic theory informs the choice of tuning parameters and yields adaptivity properties for the method over a wide class of measures of error and classes of function. The method is tested on simulated light detection and ranging data suggested by underwater remote sensing. Both visual and numerical results show an improvement over competing approaches. Finally, the theory behind our estimation paradigm gives a complete characterization of the 'maxiset' of the method: the set of functions where the method attains a near optimal rate of convergence for a variety of L ^p loss functions.     

Bayes and maximum likelihood for $$L^1$$-Wasserstein deconvolution of Laplace mixtures

We consider the problem of recovering a distribution function on the real line from observations additively contaminated with errors following the standard Laplace distribution. Assuming that the latent distribution is completely unknown leads to a nonparametric deconvolution problem. We begin by studying the rates of convergence relative to the \(L^2\)-norm and the Hellinger metric for the direct problem of estimating the sampling density, which is a mixture of Laplace densities with a possibly unbounded set of locations: the rate of convergence for the Bayes’ density estimator corresponding to a Dirichlet process prior over the space of all mixing distributions on the real line matches, up to a logarithmic factor, with the \(n^{-3/8}\log ^{1/8}n\) rate for the maximum likelihood estimator. Then, appealing to an inversion inequality translating the \(L^2\)-norm and the Hellinger distance between general kernel mixtures, with a kernel density having polynomially decaying Fourier transform, into any \(L^p\)-Wasserstein distance, \(p\ge 1\), between the corresponding mixing distributions, provided their Laplace transforms are finite in some neighborhood of zero, we derive the rates of convergence in the \(L^1\)-Wasserstein metric for the Bayes’ and maximum likelihood estimators of the mixing distribution. Merging in the \(L^1\)-Wasserstein distance between Bayes and maximum likelihood follows as a by-product, along with an assessment on the stochastic order of the discrepancy between the two estimation procedures.     

A NOTE ON ESTIMATING THE LOCATION AND SCALE PARAMETERS OF THE EXPONENTIAL DISTRIBUTION FROM A CENSORED SAMPLE

   
Replacing f (x)/F (x) by α+β(x- θ)/σ in the maximum likelihood equations ∂L/∂θ and ∂L/∂σ calculated from a censored sample, a pair of estimators θ^e and σ^e, is obtained. The variances and covariances of these estimators are calculated and compared with the corresponding values for the best linear unbiassed (BLU) estimators.     

Non-parametric tests for distributional treatment effect for randomly censored responses

Summary.  For a binary treatment ν =0, 1 and the corresponding 'potential response' Y ⁰ for the control group ( ν =0) and Y ¹ for the treatment group ( ν =1), one definition of no treatment effect is that Y ⁰ and Y ¹ follow the same distribution given a covariate vector X . Koul and Schick have provided a non-parametric test for no distributional effect when the realized response (1− ν ) Y ⁰+ ν Y ¹ is fully observed and the distribution of X is the same across the two groups. This test is thus not applicable to censored responses, nor to non-experimental (i.e. observational) studies that entail different distributions of X across the two groups. We propose ' X -matched' non-parametric tests generalizing the test of Koul and Schick following an idea of Gehan. Our tests are applicable to non-experimental data with randomly censored responses. In addition to these motivations, the tests have several advantages. First, they have the intuitive appeal of comparing all available pairs across the treatment and control groups, instead of selecting a number of matched controls (or treated) in the usual pair or multiple matching. Second, whereas most matching estimators or tests have a non-overlapping support (of X ) problem across the two groups, our tests have a built-in protection against the problem. Third, Gehan's idea allows the tests to make good use of censored observations. A simulation study is conducted, and an empirical illustration for a job training effect on the duration of unemployment is provided.     

Estimating the proportion of true null hypotheses, with application to DNA microarray data

Summary.  We consider the problem of estimating the proportion of true null hypotheses, π ₀, in a multiple-hypothesis set-up. The tests are based on observed p -values. We first review published estimators based on the estimator that was suggested by Schweder and Spjøtvoll. Then we derive new estimators based on nonparametric maximum likelihood estimation of the p -value density, restricting to decreasing and convex decreasing densities. The estimators of π ₀ are all derived under the assumption of independent test statistics. Their performance under dependence is investigated in a simulation study. We find that the estimators are relatively robust with respect to the assumption of independence and work well also for test statistics with moderate dependence.