NONPARAMETRIC ESTIMATION OF MULTIVARIATE CONVEX-TRANSFORMED DENSITIES

We study estimation of multivariate densities p of the form p(x) = h(g(x)) for x ∈ ?(d) and for a fixed monotone function h and an unknown convex function g. The canonical example is h(y) = e(-y) for y ∈ ?; in this case, the resulting class of densities [Formula: see text]is well known as the class of log-concave densities. Other functions h allow for classes of densities with heavier tails than the log-concave class.We first investigate when the maximum likelihood estimator p? exists for the class P(h) for various choices of monotone transformations h, including decreasing and increasing functions h. The resulting models for increasing transformations h extend the classes of log-convex densities studied previously in the econometrics literature, corresponding to h(y) = exp(y).We then establish consistency of the maximum likelihood estimator for fairly general functions h, including the log-concave class P(e(-y)) and many others. In a final section, we provide asymptotic minimax lower bounds for the estimation of p and its vector of derivatives at a fixed point x(0) under natural smoothness hypotheses on h and g. The proofs rely heavily on results from convex analysis.     

Smooth tail-index estimation

The two parametric distribution functions appearing in the extreme-value theory – the generalized extreme-value distribution and the generalized Pareto distribution – have log-concave densities if the extreme-value index γ∈[?1, 0]. Replacing the order statistics in tail-index estimators by their corresponding quantiles from the distribution function that is based on the estimated log-concave density ? f _n leads to novel smooth quantile and tail-index estimators. These new estimators aim at estimating the tail index especially in small samples. Acting as a smoother of the empirical distribution function, the log-concave distribution function estimator reduces estimation variability to a much greater extent than it introduces bias. As a consequence, Monte Carlo simulations demonstrate that the smoothed version of the estimators are well superior to their non-smoothed counterparts, in terms of mean-squared error.     

Quadratic Artificial Likelihood Functions Using Estimating Functions

Abstract.  A vector-valued estimating function, such as the quasi-score, is typically not the gradient of any objective function. Consequently, an analogue of the likelihood function cannot be unambiguously defined by integrating the estimating function. This paper studies an analogue of the likelihood inference in the framework of optimal estimating functions. We propose a quadratic artificial likelihood function for an optimal estimating function. The objective function is uniquely identified as the potential function from the vector field decomposition by imposing some natural restriction on the divergence-free part. The artificial likelihood function is shown to resemble a genuine likelihood function in a number of respects. A bootstrap version of the artificial likelihood function is also studied, which may be used for selecting a root as an estimate from among multiple roots to an estimating equation.     

Minimization of the probabilistic p-frame potential

We investigate the optimal configurations of n points on the unit sphere for a class of potential functions. In particular, we characterize these optimal configurations in terms of their approximation properties within frame theory. Furthermore, we consider similar optimal configurations in terms of random distributions of points on the sphere. In this probabilistic setting, we characterize these optimal distributions by means of special classes of probabilistic frames. Our work also indicates some connections between statistical shape analysis and frame theory.     

Asymptotical improvement of maximum likelihood estimators on Kullback–Leibler loss

We discuss the general form of a first-order correction to the maximum likelihood estimator which is expressed in terms of the gradient of a function, which could for example be the logarithm of a prior density function. In terms of Kullback–Leibler divergence, the correction gives an asymptotic improvement over maximum likelihood under rather general conditions. The theory is illustrated for Bayes estimators with conjugate priors. The optimal choice of hyper-parameter to improve the maximum likelihood estimator is discussed. The results based on Kullback–Leibler risk are extended to a wide class of risk functions.     

The polar slice sampler

This paper investigates the polar slice sampler, a particular type of the Markov chain Monte Carlo algorithm known as the slice sampler. This algorithm is shown to have convergence properties which under some circumstances are essentially independent of the dimension of the problem. For log-concave densities, the algorithm probably converges (from an appropriate starting point) to within 0.01 of stationarity in total variation distance in a number of iterations given as a computable function of the spherical asymmetry of the density. In particular, for spherically symmetric log-concave densities, in arbitrary dimension, with an appropriate starting point, we prove that the algorithm converges in, at most, 525 iterations. Simulations are done which confirm the polar slice sampler's excellent performance.     

Bayesian estimation and prediction for the inverse weibull distribution under general progressive censoring

Abstract

This paper deals with Bayesian estimation and prediction for the inverse Weibull distribution with shape parameter α and scale parameter λ under general progressive censoring. We prove that the posterior conditional density functions of α and λ are both log-concave based on the assumption that λ has a gamma prior distribution and α follows a prior distribution with log-concave density. Then, we present the Gibbs sampling strategy to estimate under squared-error loss any function of the unknown parameter vector (α, λ) and find credible intervals, as well as to obtain prediction intervals for future order statistics. Monte Carlo simulations are given to compare the performance of Bayesian estimators derived via Gibbs sampling with the corresponding maximum likelihood estimators, and a real data analysis is discussed in order to illustrate the proposed procedure. Finally, we extend the developed methodology to other two-parameter distributions, including the Weibull, Burr type XII, and flexible Weibull distributions, and also to general progressive hybrid censoring.     

Probability inequalities for weighted distributions

In this paper, properties of weighted distributions for general weight functions are investigated. We establish the subadditivity [superadditivity] property of weighted distributions for log-concave [log-convex] weight functions in the sense of the usual stochastic order. The main result generalizes Lemma 2.3 in Brown (2006). Several interesting moment inequalities are presented.     

Adaptive rejection sampling with fixed number of nodes

The adaptive rejection sampling (ARS) algorithm is a universal random generator for drawing samples efficiently from a univariate log-concave target probability density function (pdf). ARS generates independent samples from the target via rejection sampling with high acceptance rates. Indeed, ARS yields a sequence of proposal functions that converge toward the target pdf, so that the probability of accepting a sample approaches one. However, sampling from the proposal pdf becomes more computational demanding each time it is updated. In this work, we propose a novel ARS scheme, called Cheap Adaptive Rejection Sampling (CARS), where the computational effort for drawing from the proposal remains constant, decided in advance by the user. For generating a large number of desired samples, CARS is faster than ARS.     

Global convergence of the log-concave MLE when the true distribution is geometric

Let X₁, …, X_n be i.i.d. from a discrete probability mass function (pmf) p. In Balabdaoui et al. [(2013), ‘Asymptotic Distribution of the Discrete Log-Concave mle and Some Applications’, JRSS-B, in press], the pointwise limit distribution of the log-concave maximum-likelihood estimator (MLE) was derived in both the well- and misspecified settings. In the well-specified setting, the geometric distribution was excluded, classified as being degenerate. In this article, we establish the global asymptotic theory of the log-concave MLE of a geometric pmf in all ?_q distances for q∈{1, 2, …}∪{∞}. We also show how these asymptotic results could be used in testing whether a pmf is geometric.     

Computing the log concave NPMLE for interval censored data

In analyzing interval censored data, a non-parametric estimator is often desired due to difficulties in assessing model fits. Because of this, the non-parametric maximum likelihood estimator (NPMLE) is often the default estimator. However, the estimates for values of interest of the survival function, such as the quantiles, have very large standard errors due to the jagged form of the estimator. By forcing the estimator to be constrained to the class of log concave functions, the estimator is ensured to have a smooth survival estimate which has much better operating characteristics than the unconstrained NPMLE, without needing to specify a parametric family or smoothing parameter. In this paper, we first prove that the likelihood can be maximized under a finite set of parameters under mild conditions, although the log likelihood function is not strictly concave. We then present an efficient algorithm for computing a local maximum of the likelihood function. Using our fast new algorithm, we present evidence from simulated current status data suggesting that the rate of convergence of the log-concave estimator is faster (between \(n^{2/5}\) and \(n^{1/2}\)) than the unconstrained NPMLE (between \(n^{1/3}\) and \(n^{1/2}\)).     

The generalized symmetric Laplace distribution on the sphere

A distribution on the unit sphere is generated by conditioning a normal mixture distribution with an inverse gamma distributed weighting function. It can be regarded as the generalized symmetric Laplace distribution on the unit sphere. The density involves a modified Bessel function of the third kind which can be approximated by other simpler functions in certain limiting cases. As a consequence, the von Mises–Fisher, cardioid and Jones–Pewsey distributions are limiting cases of the new distribution. No closed form expressions exist for the roots of the likelihood equations. However, given the normal mixture structure of the distribution, we propose an E–M-algorithm-based approach for finding the maximum-likelihood estimates of the parameters which assumes the weights in the mixture to be missing data. The modeling capabilities of the proposed distribution are illustrated by fitting it and some of its competitors to two circular data sets.     

Building adaptive estimating equations when inverse of covariance estimation is difficult

Summary. To construct an optimal estimating function by weighting a set of score functions, we must either know or estimate consistently the covariance matrix for the individual scores. In problems with high dimensional correlated data the estimated covariance matrix could be unreliable. The smallest eigenvalues of the covariance matrix will be the most important for weighting the estimating equations, but in high dimensions these will be poorly determined. Generalized estimating equations introduced the idea of a working correlation to minimize such problems. However, it can be difficult to specify the working correlation model correctly. We develop an adaptive estimating equation method which requires no working correlation assumptions. This methodology relies on finding a reliable approximation to the inverse of the variance matrix in the quasi-likelihood equations. We apply a multivariate generalization of the conjugate gradient method to find estimating equations that preserve the information well at fixed low dimensions. This approach is particularly useful when the estimator of the covariance matrix is singular or close to singular, or impossible to invert owing to its large size.     

Some Relationships Between Skew-Normal Distributions and Order Statistics from Exchangeable Normal Random Vectors

The article explores the relationship between distributions of order statistics from random vectors with exchangeable normal distributions and several skewed generalizations of the normal distribution. In particular, we show that the order statistics of exchangeable normal observations have closed skew-normal distributions, and that the corresponding density function is log-concave when the order statistic is extreme. Special attention is given to the bivariate case, which is related to the univariate skew-normal distribution. The applications discussed focus on the lifetimes of coherent systems.     

A penalized likelihood approach to image warping

A warping is a function that deforms images by mapping between image domains. The choice of function is formulated statistically as maximum penalized likelihood, where the likelihood measures the similarity between images after warping and the penalty is a measure of distortion of a warping. The paper addresses two issues simultaneously, of how to choose the warping function and how to assess the alignment. A new, Fourier–von Mises image model is identified, with phase differences between Fourier-transformed images having von Mises distributions. Also, new, null set distortion criteria are proposed, with each criterion uniquely minimized by a particular set of polynomial functions. A conjugate gradient algorithm is used to estimate the warping function, which is numerically approximated by a piecewise bilinear function. The method is motivated by, and used to solve, three applied problems: to register a remotely sensed image with a map, to align microscope images obtained by using different optics and to discriminate between species of fish from photographic images.     

On linear lon-odds and estimation of discriminant coefficients

Fisher's Linear Discriminant Function Can be used to classify an individual who has sampled from one of two multivariate normal Populations. In the following, this function is viewed as the other given his data vector it is assumed that the Population means and common covariance matrix are unknown. The vector of discriminant coeffients β(p×1) is the gradient of posterior log-odds and certain of its lineqar functions are directional derivatives which have a practical meaning. Accordingly, we treat the problems of estimating several linear functions of β The usual estimatoes of these functions are scaled versions of the unbiased estmators. In this Paper, these estimators are domainated by explicit alterenatives under a quadratic loss function. we reduce the problem of estimating β to that of estimating the inverse convariance matrix.     

A generalization of the adaptive rejection sampling algorithm

Rejection sampling is a well-known method to generate random samples from arbitrary target probability distributions. It demands the design of a suitable proposal probability density function (pdf) from which candidate samples can be drawn. These samples are either accepted or rejected depending on a test involving the ratio of the target and proposal densities. The adaptive rejection sampling method is an efficient algorithm to sample from a log-concave target density, that attains high acceptance rates by improving the proposal density whenever a sample is rejected. In this paper we introduce a generalized adaptive rejection sampling procedure that can be applied with a broad class of target probability distributions, possibly non-log-concave and exhibiting multiple modes. The proposed technique yields a sequence of proposal densities that converge toward the target pdf, thus achieving very high acceptance rates. We provide a simple numerical example to illustrate the basic use of the proposed technique, together with a more elaborate positioning application using real data.     

Generalized quasi-likelihood

The quasi-score function, as defined by Wedderburn (1974) and McCullagh (1983) and so on, is a linear function of observations. The generalized quasi-score function introduced in this paper is a linear function of some unbiased basis functions, where the unbiased basis functions may be some linear functions of the observations or not, and can be easily constructed by the meaning of the parameters such as mean and median and so on. The generalized quasi-likelihood estimate obtained by such a generalized quasi-score function is consistent and has an asymptotically normal distribution. As a result, the optimum generalized quasi-score is obtained and a method to construct the optimum unbiased basis function is introduced. In order to construct the potential function, a conservative generalized estimating function is defined. By conservative, a potential function for the projected score has many properties of a log-likelihood function. Finally, some examples are given to illustrate the theoretical results. This paper is supported by NNSF project (10371059) of China and Youth Teacher Foundation of Nankai University.     

Asymmetric matrix-valued covariances for multivariate random fields on spheres

ABSTRACT

Matrix-valued covariance functions are crucial to geostatistical modelling of multivariate spatial data. The classical assumption of symmetry of a multivariate covariance function is overly restrictive and has been considered as unrealistic for most of the real data applications. Despite of that, the literature on asymmetric covariance functions has been very sparse. In particular, there is some work related to asymmetric covariances on Euclidean spaces, depending on the Euclidean distance. However, for data collected over large portions of planet Earth, the most natural spatial domain is a sphere, with the corresponding geodesic distance being the natural metric. In this work, we propose a strategy based on spatial rotations to generate asymmetric covariances for multivariate random fields on the d-dimensional unit sphere. We illustrate through simulations as well as real data analysis that our proposal allows to achieve improvements in the predictive performance in comparison to the symmetric counterpart.     

Random variate generation for the generalized inverse Gaussian distribution

We provide a uniformly efficient and simple random variate generator for the entire parameter range of the generalized inverse Gaussian distribution. A general algorithm is provided as well that works for all densities that are proportional to a log-concave function φ, even if the normalization constant is not known. It requires only black box access to φ and its derivative.