Shape constrained kernel density estimation

In this paper, a method for estimating monotone, convex and log-concave densities is proposed. The estimation procedure consists of an unconstrained kernel estimator which is modified in a second step with respect to the desired shape constraint by using monotone rearrangements. It is shown that the resulting estimate is a density itself and shares the asymptotic properties of the unconstrained estimate. A short simulation study shows the finite sample behavior.     

A Jacobian Inequality for Gradient Maps on the Sphere and Its Application to Directional Statistics

In the field of optimal transport theory, an optimal map is known to be a gradient map of a potential function satisfying cost-convexity. In this article, the Jacobian determinant of a gradient map is shown to be log-concave with respect to a convex combination of the potential functions when the underlying manifold is the sphere and the cost function is the distance squared.

As an application to statistics, a new family of probability densities on the sphere is defined in terms of cost-convex functions. The log-concave property of the likelihood function follows from the inequality.     

Computing the log concave NPMLE for interval censored data

In analyzing interval censored data, a non-parametric estimator is often desired due to difficulties in assessing model fits. Because of this, the non-parametric maximum likelihood estimator (NPMLE) is often the default estimator. However, the estimates for values of interest of the survival function, such as the quantiles, have very large standard errors due to the jagged form of the estimator. By forcing the estimator to be constrained to the class of log concave functions, the estimator is ensured to have a smooth survival estimate which has much better operating characteristics than the unconstrained NPMLE, without needing to specify a parametric family or smoothing parameter. In this paper, we first prove that the likelihood can be maximized under a finite set of parameters under mild conditions, although the log likelihood function is not strictly concave. We then present an efficient algorithm for computing a local maximum of the likelihood function. Using our fast new algorithm, we present evidence from simulated current status data suggesting that the rate of convergence of the log-concave estimator is faster (between \(n^{2/5}\) and \(n^{1/2}\)) than the unconstrained NPMLE (between \(n^{1/3}\) and \(n^{1/2}\)).     

A note on the common location parameter of several exponential populations

The problem of estimation of an unknown common location parameter of several exponential populations with unknown and possibly unequal scale parameters is considered. A wide class of estimators, including both a modified maximum likelihood estimator (MLE), and the uniformly minimum variance unbiased estimator (Umvue) proposed by ghosh and razmpour(1984), is obtained under a class of convex loss functions.     

On admissible estimation for parametric functions in linear models

This paper deals with the linear model Ey∈K, Cov y∈V. The question is investigated when a parametric function (a,y) is an admissible or inadmissible estimator of some parametric function (p,Ey). It is also discussed when a linear mapping C:K→K has the property that (a,cy) is an admissible estimator of ((Ey),a) for all a∈K. Finall the question is raised how inadmissible estimators (a,y) can be replaced by admissible estimators superior to (a,y).     

Local Adaptivity of Kernel Estimates with Plug-in Local Bandwidth Selectors

In non-parametric function estimation selection of a smoothing parameter is one of the most important issues. The performance of smoothing techniques depends highly on the choice of this parameter. Preferably the bandwidth should be determined via a data-driven procedure. In this paper we consider kernel estimators in a white noise model, and investigate whether locally adaptive plug-in bandwidths can achieve optimal global rates of convergence. We consider various classes of functions: Sobolev classes, bounded variation function classes, classes of convex functions and classes of monotone functions. We study the situations of pilot estimation with oversmoothing and without oversmoothing. Our main finding is that simple local plug-in bandwidth selectors can adapt to spatial inhomogeneity of the regression function as long as there are no local oscillations of high frequency. We establish the pointwise asymptotic distribution of the regression estimator with local plug-in bandwidth.     

A Robust and Almost Fully Efficient M-Estimator

Simultaneous robust estimates of location and scale parameters are derived from a class of M-estimating equations. A coefficient p ( p > 0), which plays a role similar to that of a tuning constant in the theory of M-estimation, determines the estimating equations. These estimating equations may be obtained as the gradient of a strictly convex criterion function. This article shows that the estimators are uniquely defined, asymptotically bi-variate normal and have positive breakdown for some choices of p . When p = 0.12 and p = 0.3, the estimators are almost fully efficient for normal and exponential distributions: efficiencies with respect to the maximum likelihood estimators are 1.00 and 0.99, respectively. It is shown that the location estimator for known scale has the maximum breakdown point 0.5 independent of p , when the target model is symmetric. Also it is shown that the scale estimator has a positive breakdown point which depends on the choice of p . A simulation study finds that the proposed location estimator has smaller variance than the Hodges–Lehmann estimator, Huber's minimax and bisquare M-estimators.     

Modeling nonlinear time series with local mixtures of generalized linear models

The authors consider a novel class of nonlinear time series models based on local mixtures of regressions of exponential family models, where the covariates include functions of lags of the dependent variable. They give conditions to guarantee consistency of the maximum likelihood estimator for correctly specified models, with stationary and nonstationary predictors. They show that consistency of the maximum likelihood estimator still holds under model misspecification. They also provide probabilistic results for the proposed model when the vector of predictors contains only lags of transformations of the modeled time series. They illustrate the consistency of the maximum likelihood estimator and the probabilistic properties via Monte Carlo simulations. Finally, they present an application using real data.     

Smooth tail-index estimation

The two parametric distribution functions appearing in the extreme-value theory – the generalized extreme-value distribution and the generalized Pareto distribution – have log-concave densities if the extreme-value index γ∈[?1, 0]. Replacing the order statistics in tail-index estimators by their corresponding quantiles from the distribution function that is based on the estimated log-concave density ? f _n leads to novel smooth quantile and tail-index estimators. These new estimators aim at estimating the tail index especially in small samples. Acting as a smoother of the empirical distribution function, the log-concave distribution function estimator reduces estimation variability to a much greater extent than it introduces bias. As a consequence, Monte Carlo simulations demonstrate that the smoothed version of the estimators are well superior to their non-smoothed counterparts, in terms of mean-squared error.     

Proximal Markov chain Monte Carlo algorithms

This paper presents a new Metropolis-adjusted Langevin algorithm (MALA) that uses convex analysis to simulate efficiently from high-dimensional densities that are log-concave, a class of probability distributions that is widely used in modern high-dimensional statistics and data analysis. The method is based on a new first-order approximation for Langevin diffusions that exploits log-concavity to construct Markov chains with favourable convergence properties. This approximation is closely related to Moreau–Yoshida regularisations for convex functions and uses proximity mappings instead of gradient mappings to approximate the continuous-time process. The proposed method complements existing MALA methods in two ways. First, the method is shown to have very robust stability properties and to converge geometrically for many target densities for which other MALA are not geometric, or only if the step size is sufficiently small. Second, the method can be applied to high-dimensional target densities that are not continuously differentiable, a class of distributions that is increasingly used in image processing and machine learning and that is beyond the scope of existing MALA and HMC algorithms. To use this method it is necessary to compute or to approximate efficiently the proximity mappings of the logarithm of the target density. For several popular models, including many Bayesian models used in modern signal and image processing and machine learning, this can be achieved with convex optimisation algorithms and with approximations based on proximal splitting techniques, which can be implemented in parallel. The proposed method is demonstrated on two challenging high-dimensional and non-differentiable models related to image resolution enhancement and low-rank matrix estimation that are not well addressed by existing MCMC methodology.     

ROBUST ESTIMATION FOR A SIMPLE EXPONENTIAL MODEL1

A Hodges-Lehmann type estimator (Hodges and Lehmann, 1963) is proposed for the parameter p in the simple exponential model (1) y=1-exp.(-ρx)+ε Some exact and asymptotic properties of the proposed estimator are also given.     

Best Invariant and Minimax Estimation of Quantiles in Finite Populations

The theoretical literature on quantile and distribution function estimation in infinite populations is very rich, and invariance plays an important role in these studies. This is not the case for the commonly occurring problem of estimation of quantiles in finite populations. The latter is more complicated and interesting because an optimal strategy consists not only of an estimator, but also of a sampling design, and the estimator may depend on the design and on the labels of sampled individuals, whereas in iid sampling, design issues and labels do not exist.We study the estimation of finite population quantiles, with emphasis on estimators that are invariant under the group of monotone transformations of the data, and suitable invariant loss functions. Invariance under the finite group of permutation of the sample is also considered. We discuss nonrandomized and randomized estimators, best invariant and minimax estimators, and sampling strategies relative to different classes. Invariant loss functions and estimators in finite population sampling have a nonparametric flavor, and various natural combinatorial questions and tools arise as a result.     

Shape Constrained Non‐parametric Estimators of the Baseline Distribution in Cox Proportional Hazards Model

Abstract. We investigate non‐parametric estimation of a monotone baseline hazard and a decreasing baseline density within the Cox model. Two estimators of a non‐decreasing baseline hazard function are proposed. We derive the non‐parametric maximum likelihood estimator and consider a Grenander type estimator, defined as the left‐hand slope of the greatest convex minorant of the Breslow estimator. We demonstrate that the two estimators are strongly consistent and asymptotically equivalent and derive their common limit distribution at a fixed point. Both estimators of a non‐increasing baseline hazard and their asymptotic properties are obtained in a similar manner. Furthermore, we introduce a Grenander type estimator for a non‐increasing baseline density, defined as the left‐hand slope of the least concave majorant of an estimator of the baseline cumulative distribution function, derived from the Breslow estimator. We show that this estimator is strongly consistent and derive its asymptotic distribution at a fixed point.     

Double robust estimation in longitudinal marginal structural models

In this article we consider estimation of causal parameters in a marginal structural model for the discrete intensity of the treatment specific counting process (e.g. hazard of a treatment specific survival time) based on longitudinal observational data on treatment, covariates and survival. We define three estimators: the inverse probability of treatment weighted (IPTW) estimator, the maximum likelihood estimator (MLE), and a double robust (DR) estimator. The DR estimator is obtained by following a general methodology for constructing double robust estimating functions in censored data models as described in van der Laan and Robins (Unified Methods for Censored Longitudinal Data and Causality, 2002). The double-robust estimator is consistent and asymptotically linear when either the treatment mechanism or the partial likelihood of the observed data is consistently estimated. We illustrate the superiority of the DR estimator relative to the IPTW and ML estimators in a simulation study. The proposed methodology is also applied to estimate the causal effect of exercise on physical functioning in a longitudinal study of seniors in Sonoma County.     

A note on adaptation in garch models

In the framework of the Engle-type (G)ARCH models, I demonstrate that there is a family of symmetric and asymmetric density functions for which the asymptotic efficiency of the semiparametric estimator is equal to the asymptotic efficiency of the maximum likelihood estimator. This family of densities is bimodal (except for the normal). I also chracterize the solution to the problem of minimizing the mean squared distance between the parametric score and the semiparametric score in order to search for unimodal densities for which the semiparametric estimator is likely to perform well. The LaPlace density function emerges as one of these cases.     

Root-n-consistent Estimation in Partial Linear Models with Long-memory Errors

We consider estimation of β in the semiparametric regression model y ( i ) - x ^T( i )β + f ( i / n ) + ε( i ) where x ( i ) = g ( i )/ n ) + e ( i , f and g are unknown smooth functions and the processes ε( i ) and e ( i ) are stationary with short- or long-range dependence. For the case of i.i.d. errors, Speckman (1988) proposed a √ n –consistent estimator of β. In this paper it is shown that, under suitable regularity conditions, this estimator is asymptotically unbiased and √ n –consistent even if the errors exhibit long-range dependence. The orders of the finite sample bias and of the required bandwidth depend on the long-memory parameters. Simulations and a data example illustrate the method     

Some theoretical results regarding the polygonal distribution

Polygonal distributions are a class of distributions that can be defined via the mixture of triangular distributions over the unit interval. We demonstrate that the densities of polygonal distributions are dense in the class of continuous and concave densities with bounded second derivatives. Furthermore, we prove that polygonal density functions provide O(g^{? 2}) approximations (where g is the number of triangular distribution components), in the supremum distance, to any density function from the hypothesized class. Parametric consistency and Hellinger consistency results for the maximum likelihood (ML) estimator are obtained. A result regarding model selection via penalized ML estimation is proved.     

On Projection‐type Estimators of Multivariate Isotonic Functions

Abstract. Let M be an isotonic real‐valued function on a compact subset of and let be an unconstrained estimator of M. A feasible monotonizing technique is to take the largest (smallest) monotone function that lies below (above) the estimator or any convex combination of these two envelope estimators. When the process is asymptotically equicontinuous for some sequence r_n→∞, we show that these projection‐type estimators are r_n‐equivalent in probability to the original unrestricted estimator. Our first motivating application involves a monotone estimator of the conditional distribution function that has the distributional properties of the local linear regression estimator. Applications also include the estimation of econometric (probability‐weighted moment, quantile) and biometric (mean remaining lifetime) functions.