Distribution theory and inference for polynomial-normal densities

This paper considers a class of densities formed by taking the product of nonnegative polynomials and normal densities. These densities provide a rich class of distributions that can be used in modelling when faced with non-normal characteristics such as skewness and multimodality. In this paper we address inferential and computational issues arising in the practical implementation of this parametric family in the context of the linear model. Exact results are recorded for the conditional analysis of location-scale models and an importance sampling algorithm is developed for the implementation of a conditional analysis for the general linear model when using polynomial-normal distributions for the error.     

EM-Type algorithms for heavy-tailed logistic mixed models

This paper aims at evaluating different aspects of Monte Carlo expectation – maximization algorithm to estimate heavy-tailed mixed logistic regression (MLR) models. As a novelty it also proposes a multiple chain Gibbs sampler to generate of the latent variables distributions thus obtaining independent samples. In heavy-tailed MLR models, the analytical forms of the full conditional distributions for the random effects are unknown. Four different Metropolis–Hastings algorithms are assumed to generate from them. We also discuss stopping rules in order to obtain more efficient algorithms in heavy-tailed MLR models. The algorithms are compared through the analysis of simulated and Ascaris Suum data.     

A likelihood ratio like permutation test for one way designs

We consider permutation tests based on a likelihood ratio like statistic for the one way or k sample design used in an example in Kolassa and Robinson [(2011), ‘Saddlepoint Approximations for Likelihood Ratio Like Statistics with Applications to Permutation Tests’, Annals of Statistics, 39, 3357–3368]. We give explicitly the region in which the statistic exists, obtaining results which permit calculation of the statistic on the boundary of this region. Numerical examples are given to illustrate improvement in the power of the tests compared to the classical statistics for long-tailed error distributions and no loss of power for normal error distributions.     

Foreword (and Preface)

We present new algorithms for computing the exact distributions and p-values of quadratic t-sample distribution-free statistics of Kruskal–Wallis type. These algorithms are presented in terms of generating functions. We show that our algorithm also works for cases with ties and that it is much faster than existing algorithms. Moreover, we show how to use the results for the Kruskal–Wallis type statistics to compute the exact null distribution of the Chacko–Shorack statistic.     

On a goodness-of-fit test for normality with unknown parameters and type-II censored data

We propose a new goodness-of-fit test for normal and lognormal distributions with unknown parameters and type-II censored data. This test is a generalization of Michael's test for censored samples, which is based on the empirical distribution and a variance stabilizing transformation. We estimate the parameters of the model by using maximum likelihood and Gupta's methods. The quantiles of the distribution of the test statistic under the null hypothesis are obtained through Monte Carlo simulations. The power of the proposed test is estimated and compared to that of the Kolmogorov–Smirnov test also using simulations. The new test is more powerful than the Kolmogorov–Smirnov test in most of the studied cases. Acceptance regions for the PP, QQ and Michael's stabilized probability plots are derived, making it possible to visualize which data contribute to the decision of rejecting the null hypothesis. Finally, an illustrative example is presented.     

Likelihood ratio test for testing equality of location parameters of two exponential distributions from doubly censored samples

The Likelihood Ratio (LR) test for testing equality of two exponential distributions with common unknown scale parameter is obtained. Samples are assumed to be drawn under a type II doubly censored sampling scheme. Effects of left and right censoring on the power of the test are studied. Further, the performance of the LR test is compared with the Tiku(1981) test.     

Convergence of Heavy-tailed Monte Carlo Markov Chain Algorithms

Abstract.  In this paper, we use recent results of Jarner & Roberts ( Ann. Appl. Probab., 12, 2002, 224) to show polynomial convergence rates of Monte Carlo Markov Chain algorithms with polynomial target distributions, in particular random-walk Metropolis algorithms, Langevin algorithms and independence samplers. We also use similar methodology to consider polynomial convergence of the Gibbs sampler on a constrained state space. The main result for the random-walk Metropolis algorithm is that heavy-tailed proposal distributions lead to higher rates of convergence and thus to qualitatively better algorithms as measured, for instance, by the existence of central limit theorems for higher moments. Thus, the paper gives for the first time a theoretical justification for the common belief that heavy-tailed proposal distributions improve convergence in the context of random-walk Metropolis algorithms. Similar results are shown to hold for Langevin algorithms and the independence sampler, while results for the mixing of Gibbs samplers on uniform distributions on constrained spaces are rather different in character.     

The Generalized Birnbaum–Saunders Distribution and Its Theory,Methodology, and Application

In this paper, we discuss the class of generalized Birnbaum–Saunders distributions, which is a very flexible family suitable for modeling lifetime data as it allows for different degrees of kurtosis and asymmetry and unimodality as well as bimodality. We describe the theoretical developments on this model including properties, transformations and related distributions, lifetime analysis, and shape analysis. We also discuss methods of inference based on uncensored and censored data, diagnostics methods, goodness-of-fit tests, and random number generation algorithms for the generalized Birnbaum–Saunders model. Finally, we present some illustrative examples and show that this distribution fits the data better than the classical Birnbaum–Saunders model.     

A generalization of the adaptive rejection sampling algorithm

Rejection sampling is a well-known method to generate random samples from arbitrary target probability distributions. It demands the design of a suitable proposal probability density function (pdf) from which candidate samples can be drawn. These samples are either accepted or rejected depending on a test involving the ratio of the target and proposal densities. The adaptive rejection sampling method is an efficient algorithm to sample from a log-concave target density, that attains high acceptance rates by improving the proposal density whenever a sample is rejected. In this paper we introduce a generalized adaptive rejection sampling procedure that can be applied with a broad class of target probability distributions, possibly non-log-concave and exhibiting multiple modes. The proposed technique yields a sequence of proposal densities that converge toward the target pdf, thus achieving very high acceptance rates. We provide a simple numerical example to illustrate the basic use of the proposed technique, together with a more elaborate positioning application using real data.     

Sensitivity analysis of violations of the faithfulness assumption

We study the implication of violations of the faithfulness condition due to parameter cancellations on estimation of the directed acyclic graph (DAG) skeleton. Three settings are investigated: when (i) faithfulness is guaranteed (ii) faithfulness is not guaranteed and (iii) the parameter distributions are concentrated around unfaithfulness (near-unfaithfulness). In a simulation study, the effects of the different settings are compared using the parents and children (PC) and max–min parents and children (MMPC) algorithms. The results show that the performance in the faithful case is almost unchanged compared with the unrestricted case, whereas there is a general decrease in performance under the near-unfaithful case as compared with the unrestricted case. The response to near-unfaithful parameterizations is similar between the two algorithms, with the MMPC algorithm having higher true positive rates and the PC algorithm having lower false positive rates.     

Inference on cointegrating ranks using lr and lm tests based on pseudo-likelihoods

This paper considers Lagrange Multiplier (LM) and Likelihood Ratio (LR) tests for determining the cointegrating rank of a vector autoregressive system. n order to deal with outliers and possible fat-tailedness of the error process, non-Gaussian likelihoods are used to carry out the estimation. The limiting distributions of the tests based on these non-Gaussian pseudo-)likelihoods are derived. These distributions depend on nuisance parameters. An operational procedure is proposed to perform inference. It appears that the tests based on non-Gaussian pseudo-likelihoods are much more powerful than their Gaussian counterparts if the errors are fat-tailed. Moreover, the operational LM-type test has a better overall performance than the LR-type test. Copyright O 1998 by Marcel Dekker, Inc.     

Comments on specification of household expenditure functions and equivalence scales by nonparametric regression

This paper considers Lagrange Multiplier (LM) and Likelihood Ratio (LR) tests for determining the cointegrating rank of a vector autoregressive system. n order to deal with outliers and possible fat-tailedness of the error process, non-Gaussian likelihoods are used to carry out the estimation. The limiting distributions of the tests based on these non-Gaussian pseudo-)likelihoods are derived. These distributions depend on nuisance parameters. An operational procedure is proposed to perform inference. It appears that the tests based on non-Gaussian pseudo-likelihoods are much more powerful than their Gaussian counterparts if the errors are fat-tailed. Moreover, the operational LM-type test has a better overall performance than the LR-type test. Copyright O 1998 by Marcel Dekker, Inc.     

Moment matrices in conditional heteroskedastic models under elliptical distributions with applications in AR-ARCH models

It is well known that moment matrices play a very important rôle in econometrics and statistics. Liu and Heyde (Stat Pap 49:455–469, 2008) give exact expressions for two-moment matrices, including the Hessian for ARCH models under elliptical distributions. In this paper, we extend the theory by establishing two additional moment matrices for conditional heteroskedastic models under elliptical distributions. The moment matrices established in this paper implement the maximum likelihood estimation by some estimation algorithms like the scoring method. We illustrate the applicability of the additional moment matrices established in this paper by applying them to establish an AR-ARCH model under an elliptical distribution.     

Bayesian analysis of circular distributions based on non-negative trigonometric sums

ABSTRACT

Fernández-Durán [Circular distributions based on nonnegative trigonometric sums. Biometrics. 2004;60:499–503] developed a new family of circular distributions based on non-negative trigonometric sums that is suitable for modelling data sets that present skewness and/or multimodality. In this paper, a Bayesian approach to deriving estimates of the unknown parameters of this family of distributions is presented. Because the parameter space is the surface of a hypersphere and the dimension of the hypersphere is an unknown parameter of the distribution, the Bayesian inference must be based on transdimensional Markov Chain Monte Carlo (MCMC) algorithms to obtain samples from the high-dimensional posterior distribution. The MCMC algorithm explores the parameter space by moving along great circles on the surface of the hypersphere. The methodology is illustrated with real and simulated data sets.     

Optimal direction Gibbs sampler for truncated multivariate normal distributions

Generalized Gibbs samplers simulate from any direction, not necessarily limited to the coordinate directions of the parameters of the objective function. We study how to optimally choose such directions in a random scan Gibbs sampler setting. We consider that optimal directions will be those that minimize the Kullback–Leibler divergence of two Markov chain Monte Carlo steps. Two distributions over direction are proposed for the multivariate Normal objective function. The resulting algorithms are used to simulate from a truncated multivariate Normal distribution, and the performance of our algorithms is compared with the performance of two algorithms based on the Gibbs sampler.     

Collapsing of Non‐centred Parameterized MCMC Algorithms with Applications to Epidemic Models

Data augmentation is required for the implementation of many Markov chain Monte Carlo (MCMC) algorithms. The inclusion of augmented data can often lead to conditional distributions from well‐known probability distributions for some of the parameters in the model. In such cases, collapsing (integrating out parameters) has been shown to improve the performance of MCMC algorithms. We show how integrating out the infection rate parameter in epidemic models leads to efficient MCMC algorithms for two very different epidemic scenarios, final outcome data from a multitype SIR epidemic and longitudinal data from a spatial SI epidemic. The resulting MCMC algorithms give fresh insight into real‐life epidemic data sets.     

Mini-batch learning of exponential family finite mixture models

Mini-batch algorithms have become increasingly popular due to the requirement for solving optimization problems, based on large-scale data sets. Using an existing online expectation–maximization (EM) algorithm framework, we demonstrate how mini-batch (MB) algorithms may be constructed, and propose a scheme for the stochastic stabilization of the constructed mini-batch algorithms. Theoretical results regarding the convergence of the mini-batch EM algorithms are presented. We then demonstrate how the mini-batch framework may be applied to conduct maximum likelihood (ML) estimation of mixtures of exponential family distributions, with emphasis on ML estimation for mixtures of normal distributions. Via a simulation study, we demonstrate that the mini-batch algorithm for mixtures of normal distributions can outperform the standard EM algorithm. Further evidence of the performance of the mini-batch framework is provided via an application to the famous MNIST data set.     

Partially projected gradient algorithms for computing nonparametric maximum likelihood estimates of mixing distributions

There exist primarily three different types of algorithms for computing nonparametric maximum likelihood estimates (NPMLEs) of mixing distributions in the literature, which are the EM-type algorithms, the vertex direction algorithms such as VDM and VEM, and the algorithms based on general constrained optimization techniques such as the projected gradient method. It is known that the projected gradient algorithm may run into stagnation during iterations. When a stagnation occurs, VDM steps need to be added. We argue that the abrupt switch to VDM steps can significantly reduce the efficiency of the projected gradient algorithm, and is usually unnecessary. In this paper, we define a group of partially projected directions, which can be regarded as hybrids of ordinary projected gradient directions and VDM directions. Based on these directions, four new algorithms are proposed for computing NPMLEs of mixing distributions. The properties of the algorithms are discussed and their convergence is proved. Extensive numerical simulations show that the new algorithms outperform the existing methods, especially when a NPMLE has a large number of support points or when high accuracy is required.     

Adaptive optimal scaling of Metropolis–Hastings algorithms using the Robbins–Monro process

ABSTRACT

We present an adaptive method for the automatic scaling of random-walk Metropolis–Hastings algorithms, which quickly and robustly identifies the scaling factor that yields a specified overall sampler acceptance probability. Our method relies on the use of the Robbins–Monro search process, whose performance is determined by an unknown steplength constant. Based on theoretical considerations we give a simple estimator of this constant for Gaussian proposal distributions. The effectiveness of our method is demonstrated with both simulated and real data examples.     

A method to estimate power parameter in exponential power distribution via polynomial regression

The exponential power distribution (EPD), also known as generalized error distribution, is a flexible symmetrical unimodal family that belongs to the exponential one. The EPD becomes the density function of a range of symmetric distributions with different values of its power parameter β. A closed-form estimator for β does not exist, so the power parameter is usually estimated numerically. Unfortunately, the optimization algorithms do not always converge, especially when the true value of β is close to its parametric space frontier. In this paper, we present an alternative method to estimate β. Our proposal is based on the normal standardized Q–Q plot, and it exploits the relationship between β and the kurtosis. Furthermore, it is a direct method which does not require computational efforts nor the use of optimization algorithms.