Higher-order Bayesian Approximations for Pseudo-posterior Distributions

The theory of higher-order asymptotics provides accurate approximations to posterior distributions for a scalar parameter of interest, and to the corresponding tail area, for practical use in Bayesian analysis. The aim of this article is to extend these approximations to pseudo-posterior distributions, e.g., posterior distributions based on a pseudo-likelihood function and a suitable prior, which are proved to be particularly useful when the full likelihood is analytically or computationally infeasible. In particular, from a theoretical point of view, we derive the Laplace approximation for a pseudo-posterior distribution, and for the corresponding tail area, for a scalar parameter of interest, also in the presence of nuisance parameters. From a computational point of view, starting from these higher-order approximations, we discuss the higher-order tail area (HOTA) algorithm useful to approximate marginal posterior distributions, and related quantities. Compared to standard Markov chain Monte Carlo methods, the main advantage of the HOTA algorithm is that it gives independent samples at a negligible computational cost. The relevant computations are illustrated by two examples.     

Modelling udder infection data using copula models for quadruples

We study copula models for correlated infection times in the four udder quarters of dairy cows. Both a semi-parametric and a nonparametric approach are considered to estimate the marginal survival functions, taking into account the effect of a binary udder quarter level covariate. We use a two-stage estimation approach and we briefly discuss the asymptotic behaviour of the estimators obtained in the first and the second stage of the estimation. A pseudo-likelihood ratio test is used to select an appropriate copula from the power variance copula family that describes the association between the outcomes in a cluster. We propose a new bootstrap algorithm to obtain the p-value for this test. This bootstrap algorithm also provides estimates for the standard errors of the estimated parameters in the copula. The proposed methods are applied to the udder infection data. A small simulation study for a setting similar to the setting of the udder infection data gives evidence that the proposed method provides a valid approach to select an appropriate copula within the power variance copula family.     

Application of an imputation method for variance estimation under pseudo-likelihood when missing data are NMAR

When data are outcome-dependent non response, pseudo-likelihood yields consistent regression coefficients without specifying the missing data mechanism. However, it is onerous to derive parameter estimators including their standard errors from the regression coefficients under pseudo-likelihood (PL). The present study applies an imputation method to compute the asymptotic standard errors of parameter estimators. The proposed method is simpler than Delta method and it showed similar effect size of the standard errors to bootstrapping in simulation and application studies.     

The multivariate alpha-power model

The main object of this paper is to propose a multivariate extension to the alpha-power model which is an alternative to the multivariate skew-normal model (Arellano-Valle and Azzalini, 2008). It also extends the power-normal model discussed in Gupta and Gupta (2008) by making it more flexible. Inference is dealt with by using the likelihood approach and a pseudo-likelihood approach based on conditional distributions which, although slightly less efficient, is simpler to implement. An application to a real data set is used to demonstrate the usefulness of the extension.     

Fisher information in randomly sampled sib pairs and extremely discordant sib pairs in genetic analysis for a quantitative trait locus

The conditional mixture likelihood method using the absolute difference of the trait values of a sib pair to estimate genetic parameters underlies commonly used method in linkage analysis. Here, the statistical properties of the model are examined. The marginal model with a pseudo-likelihood function based on a sample of the absolute difference of sib-traits is also studied. Both approaches are compared numerically. When genotyping is much more expensive than screening a quantitative trait, it is known that extremely discordant sib pairs provide more powerful linkage tests than randomly sampled sib pairs. The Fisher information about genetic parameters contained in extremely discordant sib pairs is calculated using the marginal mixture model. Our results supplement current research showing that extremely discordant sib pairs are powerful for the linkage detection by demonstrating they also contain more information about other genetic parameters.     

Pseudo MLE for semiparametric transformation model with doubly truncated data

In this article, we consider the efficient estimation of the semiparametric transformation model with doubly truncated data. We propose a two-step approach for obtaining the pseudo maximum likelihood estimators (PMLE) of regression parameters. In the first step, the truncation time distribution is estimated by the nonparametric maximum likelihood estimator (Shen, 2010a) when the distribution function $K$ of the truncation time is unspecified or by the conditional maximum likelihood estimator (Bilker and Wang, 1996) when $K$ is parameterized. In the second step, using the pseudo complete-data likelihood function with the estimated distribution of truncation time, we propose expectation–maximization algorithms for obtaining the PMLE. We establish the consistency of the PMLE. The simulation study indicates that the PMLE performs well in finite samples. The proposed method is illustrated using an AIDS data set.     

Variational Bayes for estimating the parameters of a hidden Potts model

Hidden Markov random field models provide an appealing representation of images and other spatial problems. The drawback is that inference is not straightforward for these models as the normalisation constant for the likelihood is generally intractable except for very small observation sets. Variational methods are an emerging tool for Bayesian inference and they have already been successfully applied in other contexts. Focusing on the particular case of a hidden Potts model with Gaussian noise, we show how variational Bayesian methods can be applied to hidden Markov random field inference. To tackle the obstacle of the intractable normalising constant for the likelihood, we explore alternative estimation approaches for incorporation into the variational Bayes algorithm. We consider a pseudo-likelihood approach as well as the more recent reduced dependence approximation of the normalisation constant. To illustrate the effectiveness of these approaches we present empirical results from the analysis of simulated datasets. We also analyse a real dataset and compare results with those of previous analyses as well as those obtained from the recently developed auxiliary variable MCMC method and the recursive MCMC method. Our results show that the variational Bayesian analyses can be carried out much faster than the MCMC analyses and produce good estimates of model parameters. We also found that the reduced dependence approximation of the normalisation constant outperformed the pseudo-likelihood approximation in our analysis of real and synthetic datasets.     

Pseudo-likelihood estimation for a class of spatial Markov chains

We consider a class of finite state, two-dimensional Markov chains which can produce a rich variety of patterns and whose simulation is very fast. A parameterization is chosen to make the process nearly spatially homogeneous. We use a form of pseudo-likelihood estimation which results in quick determination of estimate. Parameters associated with boundary cells are estimated separately. We derive the asymptotic distribution of the maximum pseudo-likelihood estimates and show that the usual form of the variance matrix has to be modified to take account of local dependence. Standard error calculations based on the modified asymptotic variance are supported by a simulation study. The procedure is applied to an eight-state permeability pattern from a section of hydrocarbon reservoir rock.     

Cross-validation Revisited

Data-based choice of the bandwidth is an important problem in kernel density estimation. The pseudo-likelihood and the least-squares cross-validation bandwidth selectors are well known, but widely criticized in the literature. For heavy-tailed distributions, the L₁ distance between the pseudo-likelihood-based estimator and the density does not seem to converge in probability to zero with increasing sample size. Even for normal-tailed densities, the rate of L₁ convergence is disappointingly slow. In this article, we report an interesting finding that with minor modifications both the cross-validation methods can be implemented effectively, even for heavy-tailed densities. For both these estimators, the L₁ distance (from the density) are shown to converge completely to zero irrespective of the tail of the density. The expected L₁ distance also goes to zero. These results hold even in the presence of a strongly mixing-type dependence. Monte Carlo simulations and analysis of the Old Faithful geyser data suggest that if implemented appropriately, contrary to the traditional belief, the cross-validation estimators compare well with the sophisticated plug-in and bootstrap-based estimators.     

MODELING AND ESTIMATING VARIANCES IN REGRESSION

ABSTRACT

It is well known that ignoring heteroscedasticity in regression analysis adversely affects the efficiency of estimation and renders the usual procedure for constructing prediction intervals inappropriate. In some applications, such as off-line quality control, knowledge of the variance function is also of considerable interest in its own right. Thus the modeling of variance constitutes an important part of regression analysis. A common practice in modeling variance is to assume that a certain function of the variance can be closely approximated by a function of a known parametric form. The logarithm link function is often used even if it does not fit the observed variation satisfactorily, as other alternatives may yield negative estimated variances. In this paper we propose a rich class of link functions for more flexible variance modeling which alleviates the major difficulty of negative variances. We suggest also an alternative analysis for heteroscedastic regression models that exploits the principle of “separation” discussed in Box (Signal-to-Noise Ratios, Performance Criteria and Transformation. Technometrics 1988, 30, 1–31). The proposed method does not require any distributional assumptions once an appropriate link function for modeling variance has been chosen. Unlike the analysis in Box (Signal-to-Noise Ratios, Performance Criteria and Transformation. Technometrics 1988, 30, 1–31), the estimated variances and their associated asymptotic variances are found in the original metric (although a transformation has been applied to achieve separation in a different scale), making interpretation of results considerably easier.