An entropy test for the Rayleigh distribution and power comparison

In this paper, a goodness-of-fit test is proposed for the Rayleigh distribution. This test is based on the Kullback–Leibler discrimination methodology proposed by Song [2002, Goodness of fit tests based on Kullback–Leibler discrimination, IEEE Trans. Inf. Theory 48(5), pp. 1103–1117]. The critical values and powers for some alternatives are obtained by simulation. The proposed test is compared with other tests, namely Kolmogorov–Smirnov, Kuiper, Cramer–von Mises, Watson and Anderson–Darling. The use of the proposed test is shown in a real example.     

Influential observation in complex normal data for problems in allometry

Abstract

The aim of this paper is to investigate how some results related to the complex normal distribution are relevant in size and shape analysis. Our main focus is on the derivation of influential measures. In particular, Cook and Kullback–Leibler distances are combined with their respective asymptotic results as well as to an alternative process of defining cut-off points. Some numerical examples illustrate how these measures are used in practice. We perform an application to simulated and actual data. Results provide evidence that the methodology based on Kullback–Leibler distance outperforms one in terms of the Cook classic distance.     

An adjusted cumulative Kullback-Leibler information with application to test of exponentiality

Abstract

In order to discriminate between two probability distributions extensions of Kullback–Leibler (KL) information have been proposed in the literature. In recent years, an extension called cumulative Kullback–Leibler (CKL) information is considered by authors which is closely related to equilibrium distributions. In this paper, we propose an adjusted version of CKL based on equilibrium distributions. Some properties of the proposed measure of divergence are investigated. A test of exponentiality based on the adjusted measure, is proposed. The empirical power of the presented test is calculated and compared with some existing standard tests of exponentiality. The results show that our proposed test, for some important alternative distributions, has better performance than some of the existing tests.     

General cumulative Kullback–Leibler information

The cumulative residual Kullback–Leibler information is defined on the semi-infinite (non negative) interval. In this paper, we extend the cumulative residual Kullback–Leibler information to the whole real line and propose a general cumulative Kullback–Leibler information. We study its application to a test for normality in comparison with some competing test statistics based on the empirical distribution function including the well-known tests applied in practice like Kolmogorov–Smirnov, Cramer–von Mises, Anderson–Darling, and other existing tests.     

A general long-term aging model with different underlying activation mechanisms: Modeling,Bayesian estimation,and case influence diagnostics

In this paper we propose a general cure rate aging model. Our approach enables different underlying activation mechanisms which lead to the event of interest. The number of competing causes of the event of interest is assumed to follow a logarithmic distribution. The model is parameterized in terms of the cured fraction which is then linked to covariates. We explore the use of Markov chain Monte Carlo methods to develop a Bayesian analysis for the proposed model. Moreover, some discussions on the model selection to compare the fitted models are given, as well as case deletion influence diagnostics are developed for the joint posterior distribution based on the ψ-divergence, which has several divergence measures as particular cases, such as the Kullback–Leibler (K-L), J-distance, L₁ norm, and χ²-square divergence measures. Simulation studies are performed and experimental results are illustrated based on a real malignant melanoma data.     

Testing for Covariate Effect in the Cox Proportional Hazards Regression Model

This article presents methods for testing covariate effect in the Cox proportional hazards model based on Kullback–Leibler divergence and Renyi's information measure. Renyi's measure is referred to as the information divergence of order γ (γ ≠ 1) between two distributions. In the limiting case γ → 1, Renyi's measure becomes Kullback–Leibler divergence. In our case, the distributions correspond to the baseline and one possibly due to a covariate effect. Our proposed statistics are simple transformations of the parameter vector in the Cox proportional hazards model, and are compared with the Wald, likelihood ratio and score tests that are widely used in practice. Finally, the methods are illustrated using two real-life data sets.     

Bayesian inference and diagnostics in zero-inflated generalized power series regression model

ABSTRACT

The paper provides a Bayesian analysis for the zero-inflated regression models based on the generalized power series distribution. The approach is based on Markov chain Monte Carlo methods. The residual analysis is discussed and case-deletion influence diagnostics are developed for the joint posterior distribution, based on the ψ-divergence, which includes several divergence measures such as the Kullback–Leibler, J-distance, L₁ norm, and χ²-square in zero-inflated general power series models. The methodology is reflected in a data set collected by wildlife biologists in a state park in California.     

Testing exponentiality using transformed data

In this paper, we introduce a test for uniformity and use it as the second stage of an exact goodness-of-fit test of exponentiality. By simulation, the powers of the proposed test under various alternatives are compared with exponentiality test based on Kullback–Leibler information proposed by Ebrahimi et al. [N. Ebrahimi, M. Habibullah, and E.S. Soofi, Testing exponentiality based on Kullback–Leiber information, J. R. Statist. Soc. Ser. B 54 (1992), pp. 739–748]. The results are impressive, i.e. the proposed test has higher power than the test based on entropy.     

Quantile-based cumulative Kullback–Leibler divergence

The paper introduces a quantile-based cumulative Kullback–Leibler divergence and study its various properties. Unlike the distribution function approach, the quantile-based measure possesses some unique properties. The quantile functions used in many applied works do not have any tractable distribution functions where the proposed measure is a useful tool to compute the distance between two random variables. Some useful bounds are obtained for quantile-based residual cumulative Kullback–Leibler divergence and quantile-based reliability measures. Characterization results based on the functional forms of quantile-based residual Kullback–Leibler divergence are obtained for some well-known life distributions, namely exponential, Pareto II and beta.     

Discrimination Measures for Fractional Integrated Models Based on Wavelets

Discrimination measures have been well developed for stationary time series. However in a large number of phenomena, long-term dependencies are involved. In this article, we are dealing with discrimination of fractional integrated models. Kullback–Leibler and Chernoff's discrimination measures are approximated, using the discrete wavelet transform (DWT) for discrimination of these time series classes. The simulation study indicates low misclassification rate, related to the approximations of Kullback–Leibler and Chernoff discrimination measures. Application to problem of classifying seismic data showed that our procedure performs as well as other procedures.     

Information theoretic results for circular distributions

The broad class of generalized von Mises (GvM) circular distributions has certain optimal properties with respect to information theoretic quantities. It is shown that, under constraints on the trigonometric moments, and using the Kullback–Leibler information as the measure, the closest circular distribution to any other is of the GvM form. The lower bounds for the Kullback–Leibler information in this situation are also provided. The same problem is also considered using a modified version of the Kullback–Leibler information. Finally, series expansions are given for the entropy and the normalizing constants of the GvM distribution.     

Normalization of the origin-shifted exponential distribution for control chart construction

This study demonstrates that a location parameter of an exponential distribution significantly influences normalization of the exponential. The Kullback–Leibler information number is shown to be an appropriate index for measuring data normality using a location parameter. Control charts based on probability limits and transformation are compared for known and estimated location parameters. The probabilities of type II error (β-risks) and average run length (ARL) without a location parameter indicate an ability to detect an out-of-control signal of an individual chart using a power transformation similar to using probability limits. The β-risks and ARL of control charts with an estimated location parameter deviate significantly from their theoretical values when a small sample size of n≤50 is used. Therefore, without taking into account of the existence of a location parameter, the control charts result in inaccurate detection of an out-of-control signal regardless of whether a power or natural logarithmic transformation is used. The effects of a location parameter should be eliminated before transformation. Two examples are presented to illustrate these findings.     

General treatment of goodness-of-fit tests based on Kullback–Leibler information

This paper introduces a general goodness-of-fit test based on the estimated Kullback–Leibler information. The test uses the Vasicek entropy estimate. Two special cases of the test for location–scale and shape families are discussed. The results are used to introduce goodness-of-fit tests for the uniform, Laplace, Weibull and beta distributions. The critical values and powers for some alternatives are obtained by simulation.     

Model distances for vine copulas in high dimensions

Vine copulas are a flexible class of dependence models consisting of bivariate building blocks and have proven to be particularly useful in high dimensions. Classical model distance measures require multivariate integration and thus suffer from the curse of dimensionality. In this paper, we provide numerically tractable methods to measure the distance between two vine copulas even in high dimensions. For this purpose, we consecutively develop three new distance measures based on the Kullback–Leibler distance, using the result that it can be expressed as the sum over expectations of KL distances between univariate conditional densities, which can be easily obtained for vine copulas. To reduce numerical calculations, we approximate these expectations on adequately designed grids, outperforming Monte Carlo integration with respect to computational time. For the sake of interpretability, we provide a baseline calibration for the proposed distance measures. We further develop similar substitutes for the Jeffreys distance, a symmetrized version of the Kullback–Leibler distance. In numerous examples and applications, we illustrate the strengths and weaknesses of the developed distance measures.     

Fitting time series models for longitudinal survey data under informative sampling

The purpose of this paper is to account for informative sampling in fitting time series models, and in particular an autoregressive model of order one, for longitudinal survey data. The idea behind the proposed approach is to extract the model holding for the sample data as a function of the model in the population and the first-order inclusion probabilities, and then fit the sample model using maximum-likelihood, pseudo-maximum-likelihood and estimating equations methods. A new test for sampling ignorability is proposed based on the Kullback–Leibler information measure. Also, we investigate the issue of the sensitivity of the sample model to incorrect specification of the conditional expectations of the sample inclusion probabilities. The simulation study carried out shows that the sample-likelihood-based method produces better estimators than the pseudo-maximum-likelihood method, and that sensitivity to departures from the assumed model is low. Also, we find that both the conventional t-statistic and the Kullback–Leibler information statistic for testing of sampling ignorability perform well under both informative and noninformative sampling designs.     

Shannon Information Properties of the Endpoints of Record Coverage

This paper addresses the largest and the smallest observations, at the times when a new record of either kind (upper or lower) occurs, which are it called the current upper and lower record, respectively. We examine the entropy properties of these statistics, especially the difference between entropy of upper and lower bounds of record coverage. The results are presented for some common parametric families of distributions. Several upper and lower bounds, in terms of the entropy of parent distribution, for the entropy of current records are obtained. It is shown that mutual information, as well as Kullback–Leibler distance between the endpoints of record coverage, Kullback–Leibler distance between data distribution, and current records, are all distribution-free.     

Testing for Homogeneity in Mixture Using Weighted Relative Entropy

In this article, we propose a test for homogeneity based on Kullback–Leibler information (also known as relative entropy). Though widely used in hypothesis testing problems, Kullback–Leibler information is not desirable to many researchers in the context of mixture because of its complicated form. In this article, a weighted relative entropy test (WE test), which has closed form expression in terms of the parameter estimators, is proposed. Theoretical results show that this test is consistent. Some simulation results demonstrate that the WE test is better than some leading tests when the mixture components come from normal distribution, and is competitive with them in the Poisson case. The usage of the test is illustrated in an example with data about acidity index of lakes.     

Selection Strategy for Covariance Structure of Random Effects in Linear Mixed‐effects Models

Linear mixed‐effects models are a powerful tool for modelling longitudinal data and are widely used in practice. For a given set of covariates in a linear mixed‐effects model, selecting the covariance structure of random effects is an important problem. In this paper, we develop a joint likelihood‐based selection criterion. Our criterion is the approximately unbiased estimator of the expected Kullback–Leibler information. This criterion is also asymptotically optimal in the sense that for large samples, estimates based on the covariance matrix selected by the criterion minimize the approximate Kullback–Leibler information. Finite sample performance of the proposed method is assessed by simulation experiments. As an illustration, the criterion is applied to a data set from an AIDS clinical trial.     

Testing exponentiality based on Kullback—Leibler information for progressively Type II censored data

In many life-testing and reliability experiments, data are often censored in order to reduce the cost and time associated with testing and since the conventional Type-I and Type-II censoring schemes are not flexible enough, progressive censoring is developed by researchers. In this article, we develop a general goodness of fit test by using a new estimate of Kullback–Leibler information based on progressively Type-II censored data. Consistency and other properties of the proposed test are shown. Then, we use the proposed test statistic to test for exponentiality based on progressively Type-II censored data. The power values of the proposed test under different progressively Type-II censoring schemes are computed, through Monte Carlo simulations. It is observed that the proposed test is quite powerful in compared with the test proposed by Balakrishnan et al. (2007 Balakrishnan, N., Habibi Rad, A., and Arghami, N. R. (2007). Testing exponentiality based on Kullback–Leibler information with progressively type-II censored data. IEEE Transactions on Reliability 56:301–307. [Google Scholar]). Two real datasets from progressive censoring literature are finally presented for illustrative purpose.     

Estimation and influence diagnostics for zero-inflated hyper-Poisson regression model: full Bayesian analysis

The purpose of this paper is to develop a Bayesian analysis for the zero-inflated hyper-Poisson model. Markov chain Monte Carlo methods are used to develop a Bayesian procedure for the model and the Bayes estimators are compared by simulation with the maximum-likelihood estimators. Regression modeling and model selection are also discussed and case deletion influence diagnostics are developed for the joint posterior distribution based on the functional Bregman divergence, which includes ψ-divergence and several others, divergence measures, such as the Itakura–Saito, Kullback–Leibler, and χ² divergence measures. Performance of our approach is illustrated in artificial, real apple cultivation experiment data, related to apple cultivation.