Robust Bayesian analysis of the binomial empirical Bayes problem

Empirical Bayes estimation is considered for an i.i.d. sequence of binomial parameters θ_i arising from an unknown prior distribution G(.). This problem typically arises in industrial sampling, where samples from lots are routinely used to estimate the lot fraction defective of each lot. Two related issues are explored. The first concerns the fact that only the first few moments of G are typically estimable from the data. This suggests consideration of the interval of estimates (e.g., posterior means) corresponding to the different possible G with the specified moments. Such intervals can be obtained by application of well-known moment theory. The second development concerns the need to acknowledge the uncertainty in the estimation of the first few moments of G. Our proposal is to determine a credible set for the moments, and then find the range of estimates (e.g., posterior means) corresponding to the different possible G with moments in the credible set.     

Convergence of Heavy-tailed Monte Carlo Markov Chain Algorithms

Abstract.  In this paper, we use recent results of Jarner & Roberts ( Ann. Appl. Probab., 12, 2002, 224) to show polynomial convergence rates of Monte Carlo Markov Chain algorithms with polynomial target distributions, in particular random-walk Metropolis algorithms, Langevin algorithms and independence samplers. We also use similar methodology to consider polynomial convergence of the Gibbs sampler on a constrained state space. The main result for the random-walk Metropolis algorithm is that heavy-tailed proposal distributions lead to higher rates of convergence and thus to qualitatively better algorithms as measured, for instance, by the existence of central limit theorems for higher moments. Thus, the paper gives for the first time a theoretical justification for the common belief that heavy-tailed proposal distributions improve convergence in the context of random-walk Metropolis algorithms. Similar results are shown to hold for Langevin algorithms and the independence sampler, while results for the mixing of Gibbs samplers on uniform distributions on constrained spaces are rather different in character.     

Trend estimation of multivariate time series with controlled smoothness

This paper extends the univariate time series smoothing approach provided by penalized least squares to a multivariate setting, thus allowing for joint estimation of several time series trends. The theoretical results are valid for the general multivariate case, but particular emphasis is placed on the bivariate situation from an applied point of view. The proposal is based on a vector signal-plus-noise representation of the observed data that requires the first two sample moments and specifying only one smoothing constant. A measure of the amount of smoothness of an estimated trend is introduced so that an analyst can set in advance a desired percentage of smoothness to be achieved by the trend estimate. The required smoothing constant is determined by the chosen percentage of smoothness. Closed form expressions for the smoothed estimated vector and its variance-covariance matrix are derived from a straightforward application of generalized least squares, thus providing best linear unbiased estimates for the trends. A detailed algorithm applicable for estimating bivariate time series trends is also presented and justified. The theoretical results are supported by a simulation study and two real applications. One corresponds to Mexican and US macroeconomic data within the context of business cycle analysis, and the other one to environmental data pertaining to a monitored site in Scotland.     

联合广义线性模型中的变量选择

在联合广义线性模型中,散度参数与均值都被赋予了广义线性模型的结构,本文主要考虑在只有分布的一阶矩和二阶矩指定的条件下,联合广义线性模型中均值部分的变量选择问题。本文采用广义拟似然函数,提出了新的模型选择准则(EAIC);该准则是Akaike信息准则的推广。论文通过模拟研究验证了该准则的效果。     

Applied state space modelling of non-Gaussian time series using integration-based Kalman filtering

The main topic of the paper is on-line filtering for non-Gaussian dynamic (state space) models by approximate computation of the first two posterior moments using efficient numerical integration. Based on approximating the prior of the state vector by a normal density, we prove that the posterior moments of the state vector are related to the posterior moments of the linear predictor in a simple way. For the linear predictor Gauss-Hermite integration is carried out with automatic reparametrization based on an approximate posterior mode filter. We illustrate how further topics in applied state space modelling, such as estimating hyperparameters, computing model likelihoods and predictive residuals, are managed by integration-based Kalman-filtering. The methodology derived in the paper is applied to on-line monitoring of ecological time series and filtering for small count data.     

Optimal collapsing of mixture distributions in robust recursive estimation

Several authors have discussed Kalman filtering procedures using a mixture of normals as a model for the distributions of the noise in the observation and/or the state space equations. Under this model, resulting posteriors involve a mixture of normal distributions, and a “collapsing method” must be found in order to keep the recursive procedure simple. We prove that the Kullback-Leibler distance between the mixture posterior and that of a single normal distribution is minimized when we choose the mean and variance of the single normal distribution to be the mean and variance of the mixture posterior. Hence, “collapsing by moments” is optimal in this sense. We then develop the resulting optimal algorithm for “Kalman filtering” for this situation, and illustrate its performance with an example.     

Some remarks on experimental design for estimating the expectation of a stationary random process

For estimation of the mean of a stationary random process the variance-optimal choice of the observation points (the so called experimental design) is considered. For this discrete and continuous designs are introduced, some known results of process statistics are interpreted to experimental design and a proposal for simplification of the minimization problem is offered, moreover it is proved, that for monotone decreasing eovarianee functions a design, for which the points near the ends of the observation interval are more dense than in the middle, is better than the equidistant design.     

Comparison of different estimation techniques for portfolio selection

The main problem in applying the mean-variance portfolio selection consists of the fact that the first two moments of the asset returns are unknown. In practice the optimal portfolio weights have to be estimated. This is usually done by replacing the moments by the classical unbiased sample estimators. We provide a comparison of the exact and the asymptotic distributions of the estimated portfolio weights as well as a sensitivity analysis to shifts in the moments of the asset returns. Furthermore we consider several types of shrinkage estimators for the moments. The corresponding estimators of the portfolio weights are compared with each other and with the portfolio weights based on the sample estimators of the moments. We show how the uncertainty about the portfolio weights can be introduced into the performance measurement of trading strategies. The methodology explains the bad out-of-sample performance of the classical Markowitz procedures.     

A note on maximum entropy spectrum estimation

This article studies the maximum entropy spectrum estimation. After a bnei discussion on iiow co select ciic appropriate constraints and tiie objec¬tive functions, we decide to choose the constraints containing only the first four sample moments and, consequently, to employ the second order spectral entropy as the objective function. The resulting (maximum entropy) spectral estimate is the power spectral density of an ARMA sequence. Examples for comparing our proposal with the traditional maximum entropy spectral estimate follow at the end.     

Robust adaptive Metropolis algorithm with coerced acceptance rate

The adaptive Metropolis (AM) algorithm of Haario, Saksman and Tamminen (Bernoulli 7(2):223?C242, 2001) uses the estimated covariance of the target distribution in the proposal distribution. This paper introduces a new robust adaptive Metropolis algorithm estimating the shape of the target distribution and simultaneously coercing the acceptance rate. The adaptation rule is computationally simple adding no extra cost compared with the AM algorithm. The adaptation strategy can be seen as a multidimensional extension of the previously proposed method adapting the scale of the proposal distribution in order to attain a given acceptance rate. The empirical results show promising behaviour of the new algorithm in an example with Student target distribution having no finite second moment, where the AM covariance estimate is unstable. In the examples with finite second moments, the performance of the new approach seems to be competitive with the AM algorithm combined with scale adaptation.     

Generalized portmanteau statistics and tests of randomness

This paper considers the problem of testing the randomness of Gaussian and non–Gaussian time series. A general class of parametric portmanteau statistics, which include the Box–Pierce and the Ljung–Box statistics, is introduced. Using the exact first and second moments of the sample autocorrelations when the observations are i.i.d. normal with unknown mean, the exact expected value of any portmanteau statistics is obtained for this case. Two new portmanteau statistics, which exploit the exact moments of the sample autocorrelations, are studied. For the nonparametric case, a rank portmanteau statistic is introduced. The latter has the same distribution for any series of exchangeable random variables and uses the exact moments of the rank autocorrelations. We show that its asymptotic distribution is chi–squate. Simulation results indicate that the new portmanteau statistics are better approximated by the chi–square asymptotic distribution than the Ljung–Box statistics. Several analytical results presented in the paper were derived by usig a symbolic manipulation program.     

Generalization of Fisher's z-transformation

We deal with the asymptotic expansions of the means and the variances of the correlation coefficients in truncated bivariate normal populations. The Fisher's z-transformation is generalized for stabilizing variance in a truncated normal population. The Hermite moments are introduced, and the relationship among cross moments, central cross moments, and Hermite moments are discussed.     

On an adaptive version of the Metropolis–Hastings algorithm with independent proposal distribution

In this paper, we present a general formulation of an algorithm, the adaptive independent chain (AIC), that was introduced in a special context in Gåsemyr et al . [ Methodol. Comput. Appl. Probab. 3 (2001)]. The algorithm aims at producing samples from a specific target distribution Π, and is an adaptive, non-Markovian version of the Metropolis–Hastings independent chain. A certain parametric class of possible proposal distributions is fixed, and the parameters of the proposal distribution are updated periodically on the basis of the recent history of the chain, thereby obtaining proposals that get ever closer to Π. We show that under certain conditions, the algorithm produces an exact sample from Π in a finite number of iterations, and hence that it converges to Π. We also present another adaptive algorithm, the componentwise adaptive independent chain (CAIC), which may be an alternative in particular in high dimensions. The CAIC may be regarded as an adaptive approximation to the Gibbs sampler updating parametric approximations to the conditionals of Π.     

Layered adaptive importance sampling

Monte Carlo methods represent the de facto standard for approximating complicated integrals involving multidimensional target distributions. In order to generate random realizations from the target distribution, Monte Carlo techniques use simpler proposal probability densities to draw candidate samples. The performance of any such method is strictly related to the specification of the proposal distribution, such that unfortunate choices easily wreak havoc on the resulting estimators. In this work, we introduce a layered (i.e., hierarchical) procedure to generate samples employed within a Monte Carlo scheme. This approach ensures that an appropriate equivalent proposal density is always obtained automatically (thus eliminating the risk of a catastrophic performance), although at the expense of a moderate increase in the complexity. Furthermore, we provide a general unified importance sampling (IS) framework, where multiple proposal densities are employed and several IS schemes are introduced by applying the so-called deterministic mixture approach. Finally, given these schemes, we also propose a novel class of adaptive importance samplers using a population of proposals, where the adaptation is driven by independent parallel or interacting Markov chain Monte Carlo (MCMC) chains. The resulting algorithms efficiently combine the benefits of both IS and MCMC methods.     

An alternative version of zero-inflated logarithmic series distribution and some of its applications

An important drawback of the standard logarithmic series distribution (LSD) in several practical applications is that it excludes the zero observation from its support. The LSD with non-negative support is not much studied in the literature. Recently Kumar and Riyaz [On the zero-inflated LSD and its modification. Statistica (accepted for publication). 2013] considered a distribution in this respect namely ‘zero-inflated logarithmic series distribution (ZILSD)’. Through this paper we propose an alternative form of the ZILSD and study some of its properties. We obtain expressions for its probability-generating function, mean and variance, and develop certain recurrence relations for its probabilities, raw moments and factorial moments. The parameters of the model are estimated by the method of moments and the method of maximum likelihood, and certain test procedures are considered for testing the significance of the additional parameter of the distribution. The distribution has been fitted to certain real-life data sets for illustrating its usefulness compared with certain existing models available in the literature. Further, a simulation study is conducted for assessing the performance of the estimators.     

A full likelihood procedure of exchangeable negative binomials for modelling correlated and overdispersed count data

This paper introduces an exchangeable negative binomial distribution resulting from relaxing the independence of the Bernoulli sequence associated with a negative binomial distribution to exchangeability. It is demonstrated that the introduced distribution is a mixture of negative binomial distributions and can be characterized by infinitely many parameters that form a completely monotone sequence. The moments of the distribution are derived and a small simulation is conducted to illustrate the distribution. For data analytic purposes, two methods, truncation and completely-monotone links, are given for converting the saturated distribution of infinitely many parameters to parsimonious distributions of finitely many parameters. A full likelihood procedure is described which can be used to investigate correlated and overdispersed count data common in biomedical sciences and teratology. In the end, the introduced distribution is applied to analyze a real clinical data of burn wounds on patients.     

Moment density estimation for positive random variables

An unknown moment-determinate cumulative distribution function or its density function can be recovered from corresponding moments and estimated from the empirical moments. This method of estimating an unknown density is natural in certain inverse estimation models like multiplicative censoring or biased sampling when the moments of unobserved distribution can be estimated via the transformed moments of the observed distribution. In this paper, we introduce a new nonparametric estimator of a probability density function defined on the positive real line, motivated by the above. Some fundamental properties of proposed estimator are studied. The comparison with traditional kernel density estimator is discussed.     

Stable Autoregressive Models and Signal Estimation

This article studies the problem of model identification and estimation for stable autoregressive process observed in a symmetric stable noise environment. A new tool called partial auto-covariation function is introduced to identify the stable autoregressive signals. The signal and noise parameters are estimated using a modified version of Generalized Yule Walker type method and the method of moments. The proposed methods are illustrated through data simulated from autoregressive signals with symmetric stable innovations. The new technique is applied to analyze the time series of sea surface temperature anomaly and compared with its Gaussian counterpart.     

A bivariate distribution with gamma and beta marginals with application to drought data

The first known bivariate distribution with gamma and beta marginals is introduced. Various representations are derived for its joint probability density function (pdf), joint cumulative distribution function (cdf), product moments, conditional pdfs, conditional cdfs, conditional moments, joint moment generating function, joint characteristic function and entropies. The method of maximum likelihood and the method of moments are used to derive the associated estimation procedures as well as the Fisher information matrix, variance–covariance matrix and the profile likelihood confidence intervals. An application to drought data from Nebraska is provided. Some other applications are also discussed. Finally, an extension of the bivariate distribution to the multivariate case is proposed.     

Universal methods for generating random variables with a given characteristic function

Universal generators for absolutely-continuous and integer-valued random variables are introduced. The proposal is based on a generalization of the rejection technique proposed by Devroye [The computer generation of random variables with a given characteristic function. Computers and Mathematics with Applications. 1981;7:547–552]. The method involves a dominating function solely requiring the evaluation of integrals which depend on the characteristic function of the underlying random variable. The proposal gives rise to simple algorithms which may be implemented in a few code lines and which may show noticeable performance even if some classical families of distributions are considered.