Convergence Analysis of MCMC Algorithms for Bayesian Multivariate Linear Regression with Non‐Gaussian Errors

When Gaussian errors are inappropriate in a multivariate linear regression setting, it is often assumed that the errors are iid from a distribution that is a scale mixture of multivariate normals. Combining this robust regression model with a default prior on the unknown parameters results in a highly intractable posterior density. Fortunately, there is a simple data augmentation (DA) algorithm and a corresponding Haar PX‐DA algorithm that can be used to explore this posterior. This paper provides conditions (on the mixing density) for geometric ergodicity of the Markov chains underlying these Markov chain Monte Carlo algorithms. Letting d denote the dimension of the response, the main result shows that the DA and Haar PX‐DA Markov chains are geometrically ergodic whenever the mixing density is generalized inverse Gaussian, log‐normal, inverted Gamma (with shape parameter larger than d /2) or Fréchet (with shape parameter larger than d /2). The results also apply to certain subsets of the Gamma, F and Weibull families.     

Lévy‐based Modelling in Brain Imaging

Abstract. A substantive problem in neuroscience is the lack of valid statistical methods for non‐Gaussian random fields. In the present study, we develop a flexible, yet tractable model for a random field based on kernel smoothing of a so‐called Lévy basis. The resulting field may be Gaussian, but there are many other possibilities, including random fields based on Gamma, inverse Gaussian and normal inverse Gaussian (NIG) Lévy bases. It is easy to estimate the parameters of the model and accordingly to assess by simulation the quantiles of test statistics commonly used in neuroscience. We give a concrete example of magnetic resonance imaging scans that are non‐Gaussian. For these data, simulations under the fitted models show that traditional methods based on Gaussian random field theory may leave small, but significant changes in signal level undetected, while these changes are detectable under a non‐Gaussian Lévy model.     

On the asymptotic normality of the R-estimators of the slope parameters of simple linear regression models with associated errors

ABSTRACT

The purpose of this paper is to prove, under mild conditions, the asymptotic normality of the rank estimator of the slope parameter of a simple linear regression model with stationary associated errors. This result follows from a uniform linearity property for linear rank statistics that we establish under general conditions on the dependence of the errors. We prove also a tightness criterion for weighted empirical process constructed from associated triangular arrays. This criterion is needed for the proofs which are based on that of Koul [Behavior of robust estimators in the regression model with dependent errors. Ann Stat. 1977;5(4):681–699] and of Louhichi [Louhichi S. Weak convergence for empirical processes of associated sequences. Ann Inst Henri Poincaré Probabilités Statist. 2000;36(5):547–567].     

A note on the estimates of multivariate Gaussian probability

ABSTRACT

Some lower and upper bounds of multivariate Gaussian probability are given based on the univariate Mills’ ratio. These bounds are sharper than known ones on the multivariate Mills’ ratio in many case.     

On an exponential bound for the Kaplan–Meier estimator

We review limit theory and inequalities for the Kaplan–Meier Kaplan and Meier (J Am Stat Assoc 53:457–481, 1958) product limit estimator of a survival function on the whole line . Along the way we provide bounds for the constant in an interesting inequality due to Biotouzé et al. (Ann Inst H Poincaré Probab Stat 35:735–763, 1999), and provide some numerical evidence in support of one of their conjectures. Supported in part by NSF grant DMS-0503822 and by NI-AID grant 2R01 AI291968-04.     

The Consistency and Robustness of Modified Cramér–Von Mises and Kolmogorov–Cramér Estimators

This article focuses on the minimum distance estimators under two newly introduced modifications of Cramér–von Mises distance. The generalized power form of Cramér–von Mises distance is defined together with the so-called Kolmogorov–Cramér distance which includes both standard Kolmogorov and Cramér–von Mises distances as limiting special cases. We prove the consistency of Kolmogorov-Cramér estimators in the (expected) L₁-norm by direct technique employing domination relations between statistical distances. In our numerical simulation we illustrate the quality of consistency property for sample sizes of the most practical range from n = 10 to n = 500. We study dependence of consistency in L₁-norm on ?-contamination neighborhood of the true model and further the robustness of these two newly defined estimators for normal families and contaminated samples. Numerical simulations are used to compare statistical properties of the minimum Kolmogorov–Cramér, generalized Cramér–von Mises, standard Kolmogorov, and Cramér–von Mises distance estimators of the normal family scale parameter. We deal with the corresponding order of consistency and robustness. The resulting graphs are presented and discussed for the cases of the contaminated and uncontaminated pseudo-random samples.     

Flat and Multimodal Likelihoods and Model Lack of Fit in Curved Exponential Families

Abstract. It is well known that curved exponential families can have multimodal likelihoods. We investigate the relationship between flat or multimodal likelihoods and model lack of fit, the latter measured by the score (Rao) test statistic W _U of the curved model as embedded in the corresponding full model. When data yield a locally flat or convex likelihood (root of multiplicity >1, terrace point, saddle point, local minimum), we provide a formula for W _U in such points, or a lower bound for it. The formula is related to the statistical curvature of the model, and it depends on the amount of Fisher information. We use three models as examples, including the Behrens–Fisher model, to see how a flat likelihood, etc. by itself can indicate a bad fit of the model. The results are related (dual) to classical results by Efron from 1978.     

A Bayesian analysis for the Wilcoxon signed-rank statistic

A Bayesian analysis is provided for the Wilcoxon signed-rank statistic (T⁺). The Bayesian analysis is based on a sign-bias parameter φ on the (0, 1) interval. For the case of a uniform prior probability distribution for φ and for small sample sizes (i.e., 6 ? n ? 25), values for the statistic T⁺ are computed that enable probabilistic statements about φ. For larger sample sizes, approximations are provided for the asymptotic likelihood function P(T⁺|φ) as well as for the posterior distribution P(φ|T⁺). Power analyses are examined both for properly specified Gaussian sampling and for misspecified non Gaussian models. The new Bayesian metric has high power efficiency in the range of 0.9–1 relative to a standard t test when there is Gaussian sampling. But if the sampling is from an unknown and misspecified distribution, then the new statistic still has high power; in some cases, the power can be higher than the t test (especially for probability mixtures and heavy-tailed distributions). The new Bayesian analysis is thus a useful and robust method for applications where the usual parametric assumptions are questionable. These properties further enable a way to do a generic Bayesian analysis for many non Gaussian distributions that currently lack a formal Bayesian model.     

On generalized conditional cumulative residual inaccuracy measure

Abstract

Recently, the notion of cumulative residual Rényi’s entropy has been proposed in the literature as a measure of information that parallels Rényi’s entropy. Motivated by this, here we introduce a generalized measure of it, namely cumulative residual inaccuracy of order α. We study the proposed measure for conditionally specified models of two components having possibly different ages called generalized conditional cumulative residual inaccuracy measure. Several properties of generalized conditional cumulative residual inaccuracy measure including the effect of monotone transformation are investigated. Further, we provide some bounds on using the usual stochastic order and characterize some bivariate distributions using the concept of conditional proportional hazard rate model.     

Convergence in the Wasserstein Metric for Markov Chain Monte Carlo Algorithms with Applications to Image Restoration

Abstract

In this paper, we show how the time for convergence to stationarity of a Markov chain can be assessed using the Wasserstein metric, rather than the usual choice of total variation distance. The Wasserstein metric may be more easily applied in some applications, particularly those on continuous state spaces. Bounds on convergence time are established by considering the number of iterations required to approximately couple two realizations of the Markov chain to within ε tolerance. The particular application considered is the use of the Gibbs sampler in the Bayesian restoration of a degraded image, with pixels that are a continuous grey-scale and with pixels that can only take two colours. On finite state spaces, a bound in the Wasserstein metric can be used to find a bound in total variation distance. We use this relationship to get a precise O(N log N) bound on the convergence time of the stochastic Ising model that holds for appropriate values of its parameter as well as other binary image models. Our method employing convergence in the Wasserstein metric can also be applied to perfect sampling algorithms involving coupling from the past to obtain estimates of their running times.     

RAO'S STATISTIC FOR THE ANALYSIS OF UNIFORM ASSOCIATION IN CROSS-CLASSIFICATIONS

In this paper we introduce a new measure for the analysis of association in cross-classifications having ordered categories. Association is measured in terms of the odd-ratios in 2 × 2 subtables formed from adjacent rows and adjacent columns. We focus our attention in the uniform association model. Our measure is based in the family of divergences introduced by Burbea and Rao [1] Burbea, J. and Rao, C. R. 1982a. On the convexity of some divergence measures based on entropy functions. IEEE Transactions on Information Theory, 28: 489–495. [Crossref], [Web of Science ®] , [Google Scholar]. Some well-known sets of data are reanalyzed and a simulation study is presented to analyze the behavior of the new families of test statistics introduced in this paper.     

The Self‐Similar and Multifractal Nature of a Network Traffic Model

Abstract

We look at a family of models for Internet traffic with increasing input rates and consider approximation models which exhibit self‐similarity at large time scales and multifractality at small time scales. Depending on whether the input rate is fast or slow, the total cumulative input traffic can be approximated by a self‐similar stable Lévy motion or a self‐similar Gaussian process. The stable Lévy limit does not depend on the behavior of the individual transmission schedules but the Gaussian limit does. Also, the models and their approximations show multifractal behavior at small time scales.     

TWO-DIMENSIONAL BALANCED SAMPLING PLANS EXCLUDING CONTIGUOUS UNITS

ABSTRACT

A balanced sampling plan excluding contiguous units (or BSEC for short) was first introduced by Hedayat, Rao and Stufken in 1988. These designs can be used for survey sampling when the units are arranged in one-dimensional ordering and the contiguous units in this ordering provide similar information. In this paper, we generalize the concept of a BSEC to the two-dimensional situation and give constructions of two-dimensional BSECs with block size 3. The existence problem is completely solved in the case where λ = 1.     

Generalized Bhaskar Rao designs

Generalized Bhaskar Rao designs with non-zero elements from an abelian group G are constructed. In particular this paper shows that the necessary conditions are sufficient for the existence of generalized Bhaskar Rao designs with k=3 for the following groups: ?G? is odd, G=Z^r₂, and G=Z^r₂×H where 3? ?H? and r?1. It also constructs generalized Bhaskar Rao designs with υ=k, which is equivalent to υ rows of a generalized Hadamard matrix of order n where υ?n.     

On orthogonal arrays attaining Rao's bounds

Rao (1947) provided two inequalities on parameters of an orthogonal array OA(N,m,s,t). An orthogonal array attaining these Rao bounds is said to be complete. Noda (1979) characterized complete orthogonal arrays of t=4 (strength). We here investigate complete orthogonal arrays with s=2 (levels) and general t; and with t=2, 3 and general s.     

Estimation and selection procedures in regression: An L1 approach

The authors consider the problem of estimating a regression function go involving several variables by the closest functional element of a prescribed class _G that is closest to it in the L₁ norm. They propose a new estimator ? based on independent observations and give explicit finite sample bounds for the L₁distance between ?g and go. They apply their estimation procedure to the problem of selecting the smoothing parameter in nonparametric regression.     

THE LOG-EIG DISTRIBUTION: A NEW PROBABILITY MODEL FOR LIFETIME DATA

ABSTRACT

In this paper, we propose a new probability model called the log-EIG distribution for lifetime data analysis. Some important properties of the proposed model and maximum likelihood estimation of its parameters are discussed. Its relationship with the exponential inverse Gaussian distribution is similar to that of the lognormal and the normal distributions. Through applications to well-known datasets, we show that the log-EIG distribution competes well, and in some instances even provides a better fit than the commonly used lifetime models such as the gamma, lognormal, Weibull and inverse Gaussian distributions. It can accommodate situations where an increasing failure rate model is required as well as those with a decreasing failure rate at larger times.     

An efficient algorithm for estimating the parameters of superimposed exponential signals

An efficient computational algorithm is proposed for estimating the parameters of undamped exponential signals, when the parameters are complex valued. Such data arise in several areas of applications including telecommunications, radio location of objects, seismic signal processing and computer assisted medical diagnostics. It is observed that the proposed estimators are consistent and the dispersion matrix of these estimators is asymptotically the same as that of the least squares estimators. Moreover, the asymptotic variances of the proposed estimators attain the Cramer–Rao lower bounds, when the errors are Gaussian.     

On the BLUEs in Two Linear Models via C. R. Rao's Pandora's Box

We consider the estimation of the parameters in two partitioned linear models, denoted by 𝒜 = {y, X ₁ β ₁ + X ₂ β ₂, V _𝒜} and ? = {y, X ₁ β ₁ + X ₂ β ₂, V _?}, which we call full models. Correspondingly, we define submodels 𝒜₁ = {y, X ₁ β ₁, V _𝒜} and ?₁ = {y, X ₁ β ₁, V _?}. Using the so-called Pandora's Box approach introduced by Rao (1971 Rao , C. R. ( 1971 ). Unified theory of linear estimation . Sankhy?, Ser. A 33 : 371 – 394 . [Corrigendum (1972), 34, p. 194, 477.]  [Google Scholar], we give new necessary and sufficient conditions for the equality between the best linear unbiased estimators (BLUEs) of X ₁ β ₁ under 𝒜₁ and ?₁ as well as under 𝒜 and ?. In our considerations we will utilise the Frisch–Waugh–Lovell theorem which provides a connection between the full model 𝒜 and the reduced model 𝒜 _r  = {M ₂ y, M ₂ X ₁ β ₁, M ₂ V _𝒜 M ₂} with M ₂ being an appropriate orthogonal projector. Moreover, we consider the equality of the BLUEs under the full models assuming that they are equal under the submodels.     

Asymptotically optimal Berry-Esseen-type bounds for distributions with an absolutely continuous part

Recursive and closed form upper bounds are offered for the Kolmogorov and the total variation distance between the standard normal distribution and the distribution of a standardized sum of n independent and identically distributed random variables. The method employed is a modification of the method of compositions along with Zolotarev's ideal metric. The approximation error in the CLT obtained vanishes at a rate O(n^−k/2+1), provided that the common distribution of the summands possesses an absolutely continuous part, and shares the same k−1 (k?3) first moments with the standard normal distribution. Moreover, for the first time, these new uniform Berry-Esseen-type bounds are asymptotically optimal, that is, the ratio of the true distance to the respective bound converges to unity for a large class of distributions of the summands. Thus, apart from the correct rate, the proposed error estimates incorporate an optimal asymptotic constant (factor). Finally, three illustrative examples are presented along with numerical comparisons revealing that the new bounds are sharp enough even to be used in practical statistical applications.