Empirical Bayes approach to wavelet regression using ϵ-contaminated priors

We consider an empirical Bayes approach to standard nonparametric regression estimation using a nonlinear wavelet methodology. Instead of specifying a single prior distribution on the parameter space of wavelet coefficients, which is usually the case in the existing literature, we elicit the ?-contamination class of prior distributions that is particularly attractive to work with when one seeks robust priors in Bayesian analysis. The type II maximum likelihood approach to prior selection is used by maximizing the predictive distribution for the data in the wavelet domain over a suitable subclass of the ?-contamination class of prior distributions. For the prior selected, the posterior mean yields a thresholding procedure which depends on one free prior parameter and it is level- and amplitude-dependent, thus allowing better adaptation in function estimation. We consider an automatic choice of the free prior parameter, guided by considerations on an exact risk analysis and on the shape of the thresholding rule, enabling the resulting estimator to be fully automated in practice. We also compute pointwise Bayesian credible intervals for the resulting function estimate using a simulation-based approach. We use several simulated examples to illustrate the performance of the proposed empirical Bayes term-by-term wavelet scheme, and we make comparisons with other classical and empirical Bayes term-by-term wavelet schemes. As a practical illustration, we present an application to a real-life data set that was collected in an atomic force microscopy study.     

Near Optimal Prediction from Relevant Components

Abstract. The random x regression model is approached through the group of rotations of the eigenvectors for the x ‐covariance matrix together with scale transformations for each of the corresponding regression coefficients. The partial least squares model can be constructed from the orbits of this group. A generalization of Pitman's Theorem says that the best equivariant estimator under a group is given by the Bayes estimator with the group's invariant measure as the prior. A straightforward application of this theorem turns out to be impossible since the relevant invariant prior leads to a non‐defined posterior. Nevertheless we can devise an approximate scale group with a proper invariant prior leading to a well‐defined posterior distribution with a finite mean. This Bayes estimator is explored using Markov chain Monte Carlo technique. The estimator seems to require heavy computations, but can be argued to have several nice properties. It is also a valid estimator when p>n.     

Empirical Uncertain Bayes Methods in Area‐level Models

Random effects model can account for the lack of fitting a regression model and increase precision of estimating area‐level means. However, in case that the synthetic mean provides accurate estimates, the prior distribution may inflate an estimation error. Thus, it is desirable to consider the uncertain prior distribution, which is expressed as the mixture of a one‐point distribution and a proper prior distribution. In this paper, we develop an empirical Bayes approach for estimating area‐level means, using the uncertain prior distribution in the context of a natural exponential family, which we call the empirical uncertain Bayes (EUB) method. The regression model considered in this paper includes the Poisson‐gamma and the binomial‐beta, and the normal‐normal (Fay–Herriot) model, which are typically used in small area estimation. We obtain the estimators of hyperparameters based on the marginal likelihood by using a well‐known expectation‐maximization algorithm and propose the EUB estimators of area means. For risk evaluation of the EUB estimator, we derive a second‐order unbiased estimator of a conditional mean squared error by using some techniques of numerical calculation. Through simulation studies and real data applications, we evaluate a performance of the EUB estimator and compare it with the usual empirical Bayes estimator.     

Exponential-Bound Property of Estimators and Variable Selection in Generalized Additive Models

In this article, utilizing a scale mixture of skew-normal distribution in which mixing random variable is assumed to follow a mixture model with varying weights for each observation, we introduce a generalization of skew-normal linear regression model with the aim to provide resistant results. This model, which also includes the skew-slash distribution in a particular case, allows us to accommodate and detect outlying observations under the skew-normal linear regression model. Inferences about the model are carried out through the empirical Bayes approach. The conditions for propriety of the posterior and for existence of posterior moments are given under the standard noninformative priors for regression and scale parameters as well as proper prior for skewness parameter. Then, for Bayesian inference, a Markov chain Monte Carlo method is described. Since posterior results depend on the prior hyperparameters, we estimate them adopting the empirical Bayes method as well as using a Monte Carlo EM algorithm. Furthermore, to identify possible outliers, we also apply the Bayes factor obtained through the generalized Savage-Dickey density ratio. Examining the proposed approach on simulated instance and real data, it is found to provide not only satisfactory parameter estimates rather allow identifying outliers favorably.     

Bayesian analysis of vector-autoregressive models with noninformative priors

In this paper, we investigate the properties of Bayes estimators of vector autoregression (VAR) coefficients and the covariance matrix under two commonly employed loss functions. We point out that the posterior mean of the variances of the VAR errors under the Jeffreys prior is likely to have an over-estimation bias. Our Bayesian computation results indicate that estimates using the constant prior on the VAR regression coefficients and the reference prior of Yang and Berger (Ann. Statist. 22 (1994) 1195) on the covariance matrix dominate the constant-Jeffreys prior estimates commonly used in applications of VAR models in macroeconomics. We also estimate a VAR model of consumption growth using both constant-reference and constant-Jeffreys priors.     

BAYESIAN ANALYSIS OF THE LINEAR REGRESSION MODEL WITH NON-NORMAL DISTURBANCES

This paper considers the Bayesian analysis of a linear regression model with identically independently distributed non-normal disturbances. The distribution of disturbances is approximated by an Edgeworth series distribution with cumulants, of order higher than fourth, negligible. The posterior distribution of the regression coefficients vector is obtained under the assumption of a g-prior distribution for the parameters of the model. The Bayes estimator and its Bayes risk of the estimator are derived under a quadratic loss structure.     

Bayesian analysis of the linear regression model with an edgeworth series prior distribution

The purpose of the present investigation 1s to observe the effect of departure from normahty of the prior distribution of regresslon parameters on the Bayman analysis of a h e a r regresslon model Assuming an Edgeworth serles prior distribution for the regresslon coefficients and gamma prior for the disturbances precision, the expressions for the posterlor distribution, posterlor mean and Bayes risk under a quadratic loss function are obtalned The results of a numerical evaluation are also analyzed     

Robustness of Bayes Prediction Under Error-in-Variables Superpopulation Model

In this article, we consider Bayes prediction in a finite population under the simple location error-in-variables superpopulation model. Bayes predictor of the finite population mean under Zellner's balanced loss function and the corresponding relative losses and relative savings loss are derived. The prior distribution of the unknown location parameter of the model is assumed to have a non-normal distribution belonging to the class of Edgeworth series distributions. Effects of non normality of the “true” prior distribution and that of a possible misspecification of the loss function on the Bayes predictor are illustrated for a hypothetical population.     

Adaptive nonparametric empirical Bayes estimation via wavelet series: The minimax study

In the present paper, we derive lower bounds for the risk of the nonparametric empirical Bayes estimators. In order to attain the optimal convergence rate, we propose generalization of the linear empirical Bayes estimation method which takes advantage of the flexibility of the wavelet techniques. We present an empirical Bayes estimator as a wavelet series expansion and estimate coefficients by minimizing the prior risk of the estimator. As a result, estimation of wavelet coefficients requires solution of a well-posed low-dimensional sparse system of linear equations. The dimension of the system depends on the size of wavelet support and smoothness of the Bayes estimator. An adaptive choice of the resolution level is carried out using Lepski et al. (1997) method. The method is computationally efficient and provides asymptotically optimal adaptive EB estimators. The theory is supplemented by numerous examples.     

Learning from a lot: Empirical Bayes for high‐dimensional model‐based prediction

Empirical Bayes is a versatile approach to “learn from a lot” in two ways: first, from a large number of variables and, second, from a potentially large amount of prior information, for example, stored in public repositories. We review applications of a variety of empirical Bayes methods to several well‐known model‐based prediction methods, including penalized regression, linear discriminant analysis, and Bayesian models with sparse or dense priors. We discuss “formal” empirical Bayes methods that maximize the marginal likelihood but also more informal approaches based on other data summaries. We contrast empirical Bayes to cross‐validation and full Bayes and discuss hybrid approaches. To study the relation between the quality of an empirical Bayes estimator and p, the number of variables, we consider a simple empirical Bayes estimator in a linear model setting. We argue that empirical Bayes is particularly useful when the prior contains multiple parameters, which model a priori information on variables termed “co‐data”. In particular, we present two novel examples that allow for co‐data: first, a Bayesian spike‐and‐slab setting that facilitates inclusion of multiple co‐data sources and types and, second, a hybrid empirical Bayes–full Bayes ridge regression approach for estimation of the posterior predictive interval.     

Empirical bayes shrinkage estimation of reliability in the exponential distribution

In this paper we propose two empirical Bayes shrinkage estimators for the reliability of the exponential distribution and study their properties. Under the uniform prior distribution and the inverted gamma prior distribution these estimators are developed and compared with a preliminary test estimator and with a shrinkage testimator in terms of mean squared error. The proposed empirical Bayes shrinkage estimator under the inverted gamma prior distribution is shown to be preferable to the preliminary test estimator and the shrinkage testimator when the prior value of mean life is clsoe to the true mean life.     

An Empirical Bayes Inference for the von Mises Distribution

This paper develops an empirical Bayesian analysis for the von Mises distribution, which is the most useful distribution for statistical inference of angular data. A two-stage informative prior is proposed, in which the hyperparameter is obtained from the data in one of the stages. This empirical or approximate Bayes inference is justified on the basis of maximum entropy, and it eliminates the modified Bessel functions. An example with real data and a realistic prior distribution for the regression coefficients is considered via a Metropolis-within-Gibbs algorithm.     

Particle Learning for Fat-Tailed Distributions

It is well known that parameter estimates and forecasts are sensitive to assumptions about the tail behavior of the error distribution. In this article, we develop an approach to sequential inference that also simultaneously estimates the tail of the accompanying error distribution. Our simulation-based approach models errors with a t_ν-distribution and, as new data arrives, we sequentially compute the marginal posterior distribution of the tail thickness. Our method naturally incorporates fat-tailed error distributions and can be extended to other data features such as stochastic volatility. We show that the sequential Bayes factor provides an optimal test of fat-tails versus normality. We provide an empirical and theoretical analysis of the rate of learning of tail thickness under a default Jeffreys prior. We illustrate our sequential methodology on the British pound/U.S. dollar daily exchange rate data and on data from the 2008–2009 credit crisis using daily S&P500 returns. Our method naturally extends to multivariate and dynamic panel data.     

A Bayesian approach to non-parametric monotone function estimation

Summary.  The paper proposes two Bayesian approaches to non-parametric monotone function estimation. The first approach uses a hierarchical Bayes framework and a characterization of smooth monotone functions given by Ramsay that allows unconstrained estimation. The second approach uses a Bayesian regression spline model of Smith and Kohn with a mixture distribution of constrained normal distributions as the prior for the regression coefficients to ensure the monotonicity of the resulting function estimate. The small sample properties of the two function estimators across a range of functions are provided via simulation and compared with existing methods. Asymptotic results are also given that show that Bayesian methods provide consistent function estimators for a large class of smooth functions. An example is provided involving economic demand functions that illustrates the application of the constrained regression spline estimator in the context of a multiple-regression model where two functions are constrained to be monotone.     

Testing equality of regression coefficients in heteroscedastic normal regression models

This article addresses the problem of testing whether the vectors of regression coefficients are equal for two independent normal regression models when the error variances are unknown. This problem poses severe difficulties both to the frequentist and Bayesian approaches to statistical inference. In the former approach, normal hypothesis testing theory does not apply because of the unrelated variances. In the latter, the prior distributions typically used for the parameters are improper and hence the Bayes factor-based solution cannot be used.We propose a Bayesian solution to this problem in which no subjective input is considered. We first generate “objective” proper prior distributions (intrinsic priors) for which the Bayes factor and model posterior probabilities are well defined. The posterior probability of each model is used as a model selection tool. This consistent procedure of testing hypotheses is compared with some of the frequentist approximate tests proposed in the literature.     

Bayes prediction based on right censored data

This paper is concerned with a Bayes prediction problem in the exponential distribution under random censorship. Using censored samples, we work out a prediction interval for a sum of interest which consists of some future samples. Differing from the general Bayes approach, we do not specify the prior distribution of the parameter, and only a first moment condition on the prior is assumed. Simulation studies are conducted to exhibit the coverage probabilities of the prediction interval. Financial support from the IAP research network (#P5/24) of the Belgian Government (Belgian Science Policy) is gratefully acknowledged.     

Bayes and empirical Bayes shrinkage estimation of regression coefficients

For a moderate or large number of regression coefficients, shrinkage estimates towards an overall mean are obtained by Bayes and empirical Bayes methods. For a special case, the Bayes and empirical Bayes shrinking weights are shown to be asymptotically equivalent as the amount of shrinkage goes to zero. Based on comparisons between Bayes and empirical Bayes solutions, a modification of the empirical Bayes shrinking weights designed to guard against unreasonable overshrinking is suggested. A numerical example is given.     

Bayes Estimates for the Price and Income Elasticities of Alcoholic Beverages in Finland From 1955 to 1980

The Lindley–Smith theory of Bayes estimates for multiple regression equations with exchangeability between the regression coefficients of the individual equations is extended to the case in which a first-order autoregressive process generates the regression coefficients. The ensuing formulas are applied to study monthly Finnish consumption of alcohol. The point of this application is that exchangeability between the regression coefficients is less than we can assess beforehand when there is a natural ordering, in this case according to chronological time, of the equations. Still, the general task of the Lindley–Smith estimators, to consider the combined data when estimating individual regression coefficients, is a relevant one.     

A Mixed Model Approach for Geoadditive Hazard Regression

Abstract.  Mixed model based approaches for semiparametric regression have gained much interest in recent years, both in theory and application. They provide a unified and modular framework for penalized likelihood and closely related empirical Bayes inference. In this article, we develop mixed model methodology for a broad class of Cox-type hazard regression models where the usual linear predictor is generalized to a geoadditive predictor incorporating non-parametric terms for the (log-)baseline hazard rate, time-varying coefficients and non-linear effects of continuous covariates, a spatial component, and additional cluster-specific frailties. Non-linear and time-varying effects are modelled through penalized splines, while spatial components are treated as correlated random effects following either a Markov random field or a stationary Gaussian random field prior. Generalizing existing mixed model methodology, inference is derived using penalized likelihood for regression coefficients and (approximate) marginal likelihood for smoothing parameters. In a simulation we study the performance of the proposed method, in particular comparing it with its fully Bayesian counterpart using Markov chain Monte Carlo methodology, and complement the results by some asymptotic considerations. As an application, we analyse leukaemia survival data from northwest England.     

Bayesian estimation on semiparametric models with shape restricted B-splines

In this paper, we develop a Bayesian estimation procedure for semiparametric models under shape constrains. The approach uses a hierarchical Bayes framework and characterizations of shape-constrained B-splines. We employ Markov chain Monte Carlo methods for model fitting, using a truncated normal distribution as the prior for the coefficients of basis functions to ensure the desired shape constraints. The small sample properties of the function estimators are provided via simulation and compared with existing methods. A real data analysis is conducted to illustrate the application of the proposed method.