Information theoretic results for circular distributions

The broad class of generalized von Mises (GvM) circular distributions has certain optimal properties with respect to information theoretic quantities. It is shown that, under constraints on the trigonometric moments, and using the Kullback–Leibler information as the measure, the closest circular distribution to any other is of the GvM form. The lower bounds for the Kullback–Leibler information in this situation are also provided. The same problem is also considered using a modified version of the Kullback–Leibler information. Finally, series expansions are given for the entropy and the normalizing constants of the GvM distribution.     

Quantile-based cumulative Kullback–Leibler divergence

The paper introduces a quantile-based cumulative Kullback–Leibler divergence and study its various properties. Unlike the distribution function approach, the quantile-based measure possesses some unique properties. The quantile functions used in many applied works do not have any tractable distribution functions where the proposed measure is a useful tool to compute the distance between two random variables. Some useful bounds are obtained for quantile-based residual cumulative Kullback–Leibler divergence and quantile-based reliability measures. Characterization results based on the functional forms of quantile-based residual Kullback–Leibler divergence are obtained for some well-known life distributions, namely exponential, Pareto II and beta.     

Statistical Evidence in Experiments and in Record Values

According to the law of likelihood, statistical evidence for one (simple) hypothesis against another is measured by their likelihood ratio. When the experimenter can choose between two or more experiments (of approximately the same cost) to obtain data, he would want to know which experiment provides (on average) stronger true evidence for one hypothesis against another. In this article, after defining a pre-experimental criterion for the potential strength of evidence provided by an experiment, based on entropy distance, we compare the potential statistical evidence in lower record values with that in the same number of iid observations from the same parent distribution. We also establish a relation between Fisher information and Kullback–Leibler distance.     

Distribution-free prediction intervals for the future current record statistics

Prediction of records plays an important role in many applications, such as, meteorology, hydrology, industrial stress testing and athletic events. In this paper, based on the observed current records of an iid sequence sample drawn from an arbitrary unknown distribution, we develop distribution-free prediction intervals as well as prediction upper and lower bounds for current records from another iid sequence. We also present sharp upper bounds for the expected lengths of the so obtained prediction intervals. Numerical computations of the coverage probabilities are presented for choosing the appropriate limits of the prediction intervals.      

Inequalities for Expected Current Record Statistics

In this article, we obtain sharp distribution-free bounds for the expected value of the gap between the current records and record values as well as upper sharp bounds for the spacings between any two upper current records. We also present two-sided bounds on the errors in approximating the means of current records by inverse hazard functions.     

Entropy properties of record statistics

Record values can be viewed as order statistics from a sample whose size is determined by the values and the order of occurrence of observations. They are closely connected with the occurrence times of a corresponding non-homogenous Poisson process and reliability theory. In this paper, the information properties of record values are presented based on Shannon information. Several upper and lower bounds for the entropy of record values are obtained. It is shown that, the mutual information between record values is distribution free and is computable using the distribution of the record values of the sequence from the uniform distribution.     

Influential observation in complex normal data for problems in allometry

Abstract

The aim of this paper is to investigate how some results related to the complex normal distribution are relevant in size and shape analysis. Our main focus is on the derivation of influential measures. In particular, Cook and Kullback–Leibler distances are combined with their respective asymptotic results as well as to an alternative process of defining cut-off points. Some numerical examples illustrate how these measures are used in practice. We perform an application to simulated and actual data. Results provide evidence that the methodology based on Kullback–Leibler distance outperforms one in terms of the Cook classic distance.     

Testing for Homogeneity in Mixture Using Weighted Relative Entropy

In this article, we propose a test for homogeneity based on Kullback–Leibler information (also known as relative entropy). Though widely used in hypothesis testing problems, Kullback–Leibler information is not desirable to many researchers in the context of mixture because of its complicated form. In this article, a weighted relative entropy test (WE test), which has closed form expression in terms of the parameter estimators, is proposed. Theoretical results show that this test is consistent. Some simulation results demonstrate that the WE test is better than some leading tests when the mixture components come from normal distribution, and is competitive with them in the Poisson case. The usage of the test is illustrated in an example with data about acidity index of lakes.     

Current records and record range with some applications

In a sequence of independent and identically distributed (iid) random variables, the upper (lower) current records and record range are studied. We derive general recurrence relations between the single and product moments for the upper and lower current records based on Weibull and positive Weibull distributions, as well as Pareto and negative Pareto distributions, respectively. Moreover, some asymptotic results for general current records are established. In addition, a recurrence relation and an explicit formula for the moments of record range based on the exponential distribution are given. Finally, numerical examples are presented to illustrate and corroborate theoretical results.     

General cumulative Kullback–Leibler information

The cumulative residual Kullback–Leibler information is defined on the semi-infinite (non negative) interval. In this paper, we extend the cumulative residual Kullback–Leibler information to the whole real line and propose a general cumulative Kullback–Leibler information. We study its application to a test for normality in comparison with some competing test statistics based on the empirical distribution function including the well-known tests applied in practice like Kolmogorov–Smirnov, Cramer–von Mises, Anderson–Darling, and other existing tests.     

On entropy of a Pareto distribution in the presence of outliers

ABSTRACT

The aim of this paper is obtaining the amount of information there exists in the Pareto distribution in the presence of outliers. For the sake of this purpose, Shannon entropy, ?-entropy, Fisher information, and Kullback–Leibler distance are computed. Furthermore, a section has been devoted to compare these quantities in these two cases of the Pareto distribution (with outliers and the homogenous case). At the end of this paper, two actual examples, which are related to insurance companies, are brought. A brief summary of which is done in this work is also reported.     

Model distances for vine copulas in high dimensions

Vine copulas are a flexible class of dependence models consisting of bivariate building blocks and have proven to be particularly useful in high dimensions. Classical model distance measures require multivariate integration and thus suffer from the curse of dimensionality. In this paper, we provide numerically tractable methods to measure the distance between two vine copulas even in high dimensions. For this purpose, we consecutively develop three new distance measures based on the Kullback–Leibler distance, using the result that it can be expressed as the sum over expectations of KL distances between univariate conditional densities, which can be easily obtained for vine copulas. To reduce numerical calculations, we approximate these expectations on adequately designed grids, outperforming Monte Carlo integration with respect to computational time. For the sake of interpretability, we provide a baseline calibration for the proposed distance measures. We further develop similar substitutes for the Jeffreys distance, a symmetrized version of the Kullback–Leibler distance. In numerous examples and applications, we illustrate the strengths and weaknesses of the developed distance measures.     

Estimation of the exponential Pareto II distribution parameters

In this article, we estimate the parameters of exponential Pareto II distribution by two new methods. The first one is based on the principle of maximum entropy (POME) and the second is by Kullback–Leibler divergence of survival function (KLS). Monte Carlo simulated data are used to evaluate these methods and compare them with the maximum likelihood method. Finally, we fit this distribution to a set of real data by estimation procedures.     

Some properties of Lin–Wong divergence on the past lifetime data

Measures of statistical divergence are used to assess mutual similarities between distributions of multiple variables through a variety of methodologies including Shannon entropy and Csiszar divergence. Modified measures of statistical divergence are introduced throughout the present article. Those modified measures are related to the Lin–Wong (LW) divergence applied on the past lifetime data. Accordingly, the relationship between Fisher information and the LW divergence measure was explored when applied on the past lifetime data. Throughout this study, a number of relations are proposed between various assessment methods which implement the Jensen–Shannon, Jeffreys, and Hellinger divergence measures. Also, relations between the LW measure and the Kullback–Leibler (KL) measures for past lifetime data were examined. Furthermore, the present study discusses the relationship between the proposed ordering scheme and the distance interval between LW and KL measures under certain conditions.     

Measures of dependence for the multivariate t distribution with applications to the stock market

Two measures of dependence for multivariate t and Cauchy random variables are developed based on Kullback–Leibler number. The mutual information number T(X) is obtained in a closed expression form, as well as its asymptotic distribution. A dependence coefficient ρ₁, is defined (based on the Kullback–Leibler number) with the properties of ρ₁=0 indicating independence and ρ₁=1indicating degeneracy. Two real life examples from the stock market are used to analyze the level of dependence and correlation among stocks.     

Goodness-of-fit tests based on Verma Kullback–Leibler information

In this article, a new consistent estimator of Veram’s entropy is introduced. We establish the entropy test based on the new information namely Verma Kullback–Leibler discrimination methodology. The results are used to introduce goodness-of-fit tests for normal and exponential distributions. The root of mean square errors, critical values, and powers for some alternatives are obtained by simulation. The proposed test is compared with other tests.     

An entropy test for the Rayleigh distribution and power comparison

In this paper, a goodness-of-fit test is proposed for the Rayleigh distribution. This test is based on the Kullback–Leibler discrimination methodology proposed by Song [2002, Goodness of fit tests based on Kullback–Leibler discrimination, IEEE Trans. Inf. Theory 48(5), pp. 1103–1117]. The critical values and powers for some alternatives are obtained by simulation. The proposed test is compared with other tests, namely Kolmogorov–Smirnov, Kuiper, Cramer–von Mises, Watson and Anderson–Darling. The use of the proposed test is shown in a real example.     

General treatment of goodness-of-fit tests based on Kullback–Leibler information

This paper introduces a general goodness-of-fit test based on the estimated Kullback–Leibler information. The test uses the Vasicek entropy estimate. Two special cases of the test for location–scale and shape families are discussed. The results are used to introduce goodness-of-fit tests for the uniform, Laplace, Weibull and beta distributions. The critical values and powers for some alternatives are obtained by simulation.     

Testing the validity of the exponential model for hybrid Type-I censored data

The exponential distribution has been used in life-testing and reliability studies. In this article, we first express the entropy of Type-I hybrid censoring scheme in terms of hazard function and provide an estimate of the entropy of Type-I hybrid censored data. Then, we construct a goodness-of-fit test statistic based on Kullback–Leibler information for Type-I hybrid censored data. The test statistic is used to test for exponentiality. A Monte Carlo simulation is conducted to obtain the power of the proposed test against various alternatives. Finally, a data example is presented for illustrative purpose.     

Testing exponentiality using transformed data

In this paper, we introduce a test for uniformity and use it as the second stage of an exact goodness-of-fit test of exponentiality. By simulation, the powers of the proposed test under various alternatives are compared with exponentiality test based on Kullback–Leibler information proposed by Ebrahimi et al. [N. Ebrahimi, M. Habibullah, and E.S. Soofi, Testing exponentiality based on Kullback–Leiber information, J. R. Statist. Soc. Ser. B 54 (1992), pp. 739–748]. The results are impressive, i.e. the proposed test has higher power than the test based on entropy.