A simple way to obtain the spectral decomposition of variance components models for balanced data

By means of an example it is shown how eigenvalues and eigenvectors of variance components models can be obtained straightforwardly when balanced data are available. Simple asymptotically efficient estimators of the variance components are presented.     

Markov chain Monte Carlo estimation of a mixture item response theory model

Markov chain Monte Carlo (MCMC) algorithms have been shown to be useful for estimation of complex item response theory (IRT) models. Although an MCMC algorithm can be very useful, it also requires care in use and interpretation of results. In particular, MCMC algorithms generally make extensive use of priors on model parameters. In this paper, MCMC estimation is illustrated using a simple mixture IRT model, a mixture Rasch model (MRM), to demonstrate how the algorithm operates and how results may be affected by some commonly used priors. Priors on the probabilities of mixtures, label switching, model selection, metric anchoring, and implementation of the MCMC algorithm using WinBUGS are described, and their effects illustrated on parameter recovery in practical testing situations. In addition, an example is presented in which an MRM is fitted to a set of educational test data using the MCMC algorithm and a comparison is illustrated with results from three existing maximum likelihood estimation methods.     

Invariant co-ordinate selection

Summary.  A general method for exploring multivariate data by comparing different estimates of multivariate scatter is presented. The method is based on the eigenvalue–eigenvector decomposition of one scatter matrix relative to another. In particular, it is shown that the eigenvectors can be used to generate an affine invariant co-ordinate system for the multivariate data. Consequently, we view this method as a method for invariant co-ordinate selection . By plotting the data with respect to this new invariant co-ordinate system, various data structures can be revealed. For example, under certain independent components models, it is shown that the invariant co- ordinates correspond to the independent components. Another example pertains to mixtures of elliptical distributions. In this case, it is shown that a subset of the invariant co-ordinates corresponds to Fisher's linear discriminant subspace, even though the class identifications of the data points are unknown. Some illustrative examples are given.     

Bayesian Inference Using Artificial Augmenting Regressions

In this article, it is shown that many intractable problems of Bayesian inference can be cast in a form called “artificial augmenting regression” in which application of Markov Chain Monte Carlo techniques, especially Gibbs sampling with data augmentation, is rather convenient. The new techniques are illustrated using several challenging statistical problems and numerical results are presented.     

A note on finding critical values for a locally optimal multivariate test statistic

A previous paper provided a locally optimal test statistic for combining s independent test statistics of a common multivariate hypothesis. Difficulty in calculating the critical values limited the applicability of the test. In this paper, it is shown that approximate critical values can be easily calculated, with negligible loss of accuracy for most situations.     

On the Bayesian analysis of categorical data: the problem of nonresponse

It is demonstrated how a suitably chosen prior for the frequency parameters can streamline the Bayesian analysis of categorical data with missing entries due to nonresponse or other causes. The two cases where the data follow the Multinomial or the Hypergeometric model are treated separately. In the first case it is adequate to restrict the prior (for the cell probabilities) to the class of Dirichlet distributions. In the case of the Hypergeometric model it is convenient to select a prior from the class of Dirichlet-Multinomial (DM) distributions. The DM distributions are studied in some details.     

A Pitman measure of similarity in k-means for clustering heavy-tailed data

One of the most popular methods and algorithms to partition data to k clusters is k-means clustering algorithm. Since this method relies on some basic conditions such as, the existence of mean and finite variance, it is unsuitable for data that their variances are infinite such as data with heavy tailed distribution. Pitman Measure of Closeness (PMC) is a criterion to show how much an estimator is close to its parameter with respect to another estimator. In this article using PMC, based on k-means clustering, a new distance and clustering algorithm is developed for heavy tailed data.     

Influential Observations in Time Series

This article studies how to identify influential observations in univariate autoregressive integrated moving average time series models and how to measure their effects on the estimated parameters of the model. The sensitivity of the parameters to the presence of either additive or innovational outliers is analyzed, and influence statistics based on the Mahalanobis distance are presented. The statistic linked to additive outliers is shown to be very useful for indicating the robustness of the fitted model to the given data set. Its application is illustrated using a relevant set of historical data.     

Series evaluation of Tweedie exponential dispersion model densities

Exponential dispersion models, which are linear exponential families with a dispersion parameter, are the prototype response distributions for generalized linear models. The Tweedie family comprises those exponential dispersion models with power mean-variance relationships. The normal, Poisson, gamma and inverse Gaussian distributions belong to theTweedie family. Apart from these special cases, Tweedie distributions do not have density functions which can be written in closed form. Instead, the densities can be represented as infinite summations derived from series expansions. This article describes how the series expansions can be summed in an numerically efficient fashion. The usefulness of the approach is demonstrated, but full machine accuracy is shown not to be obtainable using the series expansion method for all parameter values. Derivatives of the density with respect to the dispersion parameter are also derived to facilitate maximum likelihood estimation. The methods are demonstrated on two data examples and compared with with Box-Cox transformations and extended quasi-likelihoood.     

Characterization of a mixture of gamma distributions via conditional finite moments

A necessary and sufficient condition that a continuous, positive random variable follow a gamma distribution is given in terms of any one of its conditional finite moments and an expression involving its failure rate. The results are then used to develop a characterization for a mixture of two gamma distributions. The general results about characterization of a mixture of gamma distributions yield several special cases that have appeared separately in recent literature, including characterization of a single exponential distribution, characterization of a single gamma distribution (in terms of either first or second moments) and a sufficient condition for a mixture of two exponential distributions (in terms of first moments). The condition in this last result is shown to be necessary also. Numerous other cases are possible, using different choices for distribution parameters along with a selection of the mixing parameter, for either individual or mixtures of distributions. Various characterizations can be expressed using higher order moments, too.     

Some nonparametric methods for changepoint problems

A general model for changepoint problems is discussed from a nonparametric viewpoint. The test statistics introduced are based on Cramér-von Mises functionals of certain processes and are shown to converge in distribution to corresponding Gaussian functionals (under the assumption of no change in distribution, H₀). We also demonstrate how the distribution of the limiting Gaussian functionals may be tabulated. Finally, properties of the tests under the alternative hypothesis of exactly one changepoint occurring are studied, and some examples are given.     

Extended Poisson process modelling of dilution series data

Summary.  Data comprising colony counts, or a binary variable representing fertile (or sterile) samples, as a dilution series of the containing medium are analysed by using extended Poisson process modelling. These models form a class of flexible probability distributions that are widely applicable to count and grouped binary data. Standard distributions such as Poisson and binomial, and those representing overdispersion and underdispersion relative to these distributions can be expressed within this class. For all the models in the class, likelihoods can be obtained. These models have not been widely used because of the perceived difficulty of performing the calculations and the lack of associated software. Exact calculation of the probabilities that are involved can be time consuming although accurate approximations that use considerably less computational time are available. Although dilution series data are the focus here, the models are applicable to any count or binary data. A benefit of the approach is the ability to draw likelihood-based inferences from the data.     

Incomplete categorical data analysis: a bayesian perspective

In this paper the Bayesian analysis of incomplete categorical data under informative general censoring proposed by Paulino and Pereira (1995) is revisited. That analysis is based on Dirichlet priors and can be applied to any missing data pattern. However, the known properties of the posterior distributions are scarce and therefore severe limitations to the posterior computations remain. Here is shown how a Monte Carlo simulation approach based on an alternative parameterisation can be used to overcome the former computational difficulties. The proposed simulation approach makes available the approximate estimation of general parametric functions and can be implemented in a very straightforward way.     

A minimally informative likelihood for decision analysis: Illustration and robustness

The authors discuss a class of likelihood functions involving weak assumptions on data generating mechanisms. These likelihoods may be appropriate when it is difficult to propose models for the data. The properties of these likelihoods are given and it is shown how they can be computed numerically by use of the Blahut-Arimoto algorithm. The authors then show how these likelihoods can give useful inferences using a data set for which no plausible physical model is apparent. The plausibility of the inferences is enhanced by the extensive robustness analysis these likelihoods permit.     

Bayesian analysis of the scatterometer wind retrieval inverse problem: some new approaches

Summary.  The retrieval of wind vectors from satellite scatterometer observations is a non-linear inverse problem. A common approach to solving inverse problems is to adopt a Bayesian framework and to infer the posterior distribution of the parameters of interest given the observations by using a likelihood model relating the observations to the parameters, and a prior distribution over the parameters. We show how Gaussian process priors can be used efficiently with a variety of likelihood models, using local forward (observation) models and direct inverse models for the scatterometer. We present an enhanced Markov chain Monte Carlo method to sample from the resulting multimodal posterior distribution. We go on to show how the computational complexity of the inference can be controlled by using a sparse, sequential Bayes algorithm for estimation with Gaussian processes. This helps to overcome the most serious barrier to the use of probabilistic, Gaussian process methods in remote sensing inverse problems, which is the prohibitively large size of the data sets. We contrast the sampling results with the approximations that are found by using the sparse, sequential Bayes algorithm.     

Fitting Distributions with the Polyhazard Model with Dependence

The polyhazard model with dependent causes, first introduced to fit lifetime data, generalized the traditional polyhazard model by allowing the latent causes of failure to be dependent by using copula functions. When modeling lifetime data, marginal distributions are supported on the positive reals. Dropping this restriction, the method generates a rich family of univariate distributions with asymmetries and multiple modes. We show that this new family of distributions is able to approximate other distributions proposed in the literature, such as the generalized beta-generated distributions. These distributions are fitted to three real data sets.     

Robust estimation of dropout models using hierarchical likelihood

It is very well known that analyses for missing data depend on untestable assumptions. As a consequence, in such settings, sensitivity analyses are often sensible. One such class of analyses assesses the dependence of conclusions on an explicit missing value mechanism. Inevitably, there is an association between such dependence and the actual (but unknown) distribution of the missing data. In a particular parametric framework for dropout in this paper, an approach is presented that reduces (but never removes) the impact of incorrect assumptions on the form of the association. It is shown how these models can be formulated and fitted relatively simply using hierarchical likelihood. These are applied directly to an example involving mastitis in dairy cattle, and an extensive simulation study is described to show the properties of the methods.     

The 3 F 2 with Complex Parameters as Generating Function of Discrete Distribution

A new discrete family of probability distributions that are generated by the ₃ F ₂ function with complex parameters is presented. Some of the properties of this new family are studied as well as methods of estimation for its parameters. It affords considerable flexibility of shape which turns the distribution into an appropriate candidate for modeling data that cannot be adequately fitted by classical families with fewer parameters. Finally, three examples in the fields of Agriculture and Education are included in order to show the versatility and utility of this distribution.     

Markov chain Monte Carlo with the Integrated Nested Laplace Approximation

The Integrated Nested Laplace Approximation (INLA) has established itself as a widely used method for approximate inference on Bayesian hierarchical models which can be represented as a latent Gaussian model (LGM). INLA is based on producing an accurate approximation to the posterior marginal distributions of the parameters in the model and some other quantities of interest by using repeated approximations to intermediate distributions and integrals that appear in the computation of the posterior marginals. INLA focuses on models whose latent effects are a Gaussian Markov random field. For this reason, we have explored alternative ways of expanding the number of possible models that can be fitted using the INLA methodology. In this paper, we present a novel approach that combines INLA and Markov chain Monte Carlo (MCMC). The aim is to consider a wider range of models that can be fitted with INLA only when some of the parameters of the model have been fixed. We show how new values of these parameters can be drawn from their posterior by using conditional models fitted with INLA and standard MCMC algorithms, such as Metropolis–Hastings. Hence, this will extend the use of INLA to fit models that can be expressed as a conditional LGM. Also, this new approach can be used to build simpler MCMC samplers for complex models as it allows sampling only on a limited number of parameters in the model. We will demonstrate how our approach can extend the class of models that could benefit from INLA, and how the R-INLA package will ease its implementation. We will go through simple examples of this new approach before we discuss more advanced applications with datasets taken from the relevant literature. In particular, INLA within MCMC will be used to fit models with Laplace priors in a Bayesian Lasso model, imputation of missing covariates in linear models, fitting spatial econometrics models with complex nonlinear terms in the linear predictor and classification of data with mixture models. Furthermore, in some of the examples we could exploit INLA within MCMC to make joint inference on an ensemble of model parameters.     

Identifiable state‐space models: A case study of the Bay of Fundy sea scallop fishery

State‐space models (SSMs) are now popular tools in fisheries science for providing management advice when faced with noisy survey and commercial fishery data. Such models are often fitted within a Bayesian framework requiring both the specification of prior distributions for model parameters and simulation‐based approaches for inference. Here we present a frequentist framework as a viable alternative and recommend using the Laplace approximation with automatic differentiation, as implemented in the R package Template Model Builder, for fast fitting and reliable inference. Additionally we highlight some identifiability issues associated with SSMs that fisheries scientists should be aware of and demonstrate how our modelling strategy surmounts these problems. Using the Bay of Fundy sea scallop fishery we show that our implementation yields more conservative advice than that of the reference model. The Canadian Journal of Statistics 47: 27–45; 2019 © 2018 Statistical Society of Canada