General cumulative Kullback–Leibler information

The cumulative residual Kullback–Leibler information is defined on the semi-infinite (non negative) interval. In this paper, we extend the cumulative residual Kullback–Leibler information to the whole real line and propose a general cumulative Kullback–Leibler information. We study its application to a test for normality in comparison with some competing test statistics based on the empirical distribution function including the well-known tests applied in practice like Kolmogorov–Smirnov, Cramer–von Mises, Anderson–Darling, and other existing tests.     

Quantile-based cumulative Kullback–Leibler divergence

The paper introduces a quantile-based cumulative Kullback–Leibler divergence and study its various properties. Unlike the distribution function approach, the quantile-based measure possesses some unique properties. The quantile functions used in many applied works do not have any tractable distribution functions where the proposed measure is a useful tool to compute the distance between two random variables. Some useful bounds are obtained for quantile-based residual cumulative Kullback–Leibler divergence and quantile-based reliability measures. Characterization results based on the functional forms of quantile-based residual Kullback–Leibler divergence are obtained for some well-known life distributions, namely exponential, Pareto II and beta.     

Testing for Homogeneity in Mixture Using Weighted Relative Entropy

In this article, we propose a test for homogeneity based on Kullback–Leibler information (also known as relative entropy). Though widely used in hypothesis testing problems, Kullback–Leibler information is not desirable to many researchers in the context of mixture because of its complicated form. In this article, a weighted relative entropy test (WE test), which has closed form expression in terms of the parameter estimators, is proposed. Theoretical results show that this test is consistent. Some simulation results demonstrate that the WE test is better than some leading tests when the mixture components come from normal distribution, and is competitive with them in the Poisson case. The usage of the test is illustrated in an example with data about acidity index of lakes.     

Measures of dependence for the multivariate t distribution with applications to the stock market

Two measures of dependence for multivariate t and Cauchy random variables are developed based on Kullback–Leibler number. The mutual information number T(X) is obtained in a closed expression form, as well as its asymptotic distribution. A dependence coefficient ρ₁, is defined (based on the Kullback–Leibler number) with the properties of ρ₁=0 indicating independence and ρ₁=1indicating degeneracy. Two real life examples from the stock market are used to analyze the level of dependence and correlation among stocks.     

Shannon Information Properties of the Endpoints of Record Coverage

This paper addresses the largest and the smallest observations, at the times when a new record of either kind (upper or lower) occurs, which are it called the current upper and lower record, respectively. We examine the entropy properties of these statistics, especially the difference between entropy of upper and lower bounds of record coverage. The results are presented for some common parametric families of distributions. Several upper and lower bounds, in terms of the entropy of parent distribution, for the entropy of current records are obtained. It is shown that mutual information, as well as Kullback–Leibler distance between the endpoints of record coverage, Kullback–Leibler distance between data distribution, and current records, are all distribution-free.     

An entropy test for the Rayleigh distribution and power comparison

In this paper, a goodness-of-fit test is proposed for the Rayleigh distribution. This test is based on the Kullback–Leibler discrimination methodology proposed by Song [2002, Goodness of fit tests based on Kullback–Leibler discrimination, IEEE Trans. Inf. Theory 48(5), pp. 1103–1117]. The critical values and powers for some alternatives are obtained by simulation. The proposed test is compared with other tests, namely Kolmogorov–Smirnov, Kuiper, Cramer–von Mises, Watson and Anderson–Darling. The use of the proposed test is shown in a real example.     

Selection Strategy for Covariance Structure of Random Effects in Linear Mixed‐effects Models

Linear mixed‐effects models are a powerful tool for modelling longitudinal data and are widely used in practice. For a given set of covariates in a linear mixed‐effects model, selecting the covariance structure of random effects is an important problem. In this paper, we develop a joint likelihood‐based selection criterion. Our criterion is the approximately unbiased estimator of the expected Kullback–Leibler information. This criterion is also asymptotically optimal in the sense that for large samples, estimates based on the covariance matrix selected by the criterion minimize the approximate Kullback–Leibler information. Finite sample performance of the proposed method is assessed by simulation experiments. As an illustration, the criterion is applied to a data set from an AIDS clinical trial.     

Testing for Covariate Effect in the Cox Proportional Hazards Regression Model

This article presents methods for testing covariate effect in the Cox proportional hazards model based on Kullback–Leibler divergence and Renyi's information measure. Renyi's measure is referred to as the information divergence of order γ (γ ≠ 1) between two distributions. In the limiting case γ → 1, Renyi's measure becomes Kullback–Leibler divergence. In our case, the distributions correspond to the baseline and one possibly due to a covariate effect. Our proposed statistics are simple transformations of the parameter vector in the Cox proportional hazards model, and are compared with the Wald, likelihood ratio and score tests that are widely used in practice. Finally, the methods are illustrated using two real-life data sets.     

Influential observation in complex normal data for problems in allometry

Abstract

The aim of this paper is to investigate how some results related to the complex normal distribution are relevant in size and shape analysis. Our main focus is on the derivation of influential measures. In particular, Cook and Kullback–Leibler distances are combined with their respective asymptotic results as well as to an alternative process of defining cut-off points. Some numerical examples illustrate how these measures are used in practice. We perform an application to simulated and actual data. Results provide evidence that the methodology based on Kullback–Leibler distance outperforms one in terms of the Cook classic distance.     

Discrimination Measures for Fractional Integrated Models Based on Wavelets

Discrimination measures have been well developed for stationary time series. However in a large number of phenomena, long-term dependencies are involved. In this article, we are dealing with discrimination of fractional integrated models. Kullback–Leibler and Chernoff's discrimination measures are approximated, using the discrete wavelet transform (DWT) for discrimination of these time series classes. The simulation study indicates low misclassification rate, related to the approximations of Kullback–Leibler and Chernoff discrimination measures. Application to problem of classifying seismic data showed that our procedure performs as well as other procedures.     

An adjusted cumulative Kullback-Leibler information with application to test of exponentiality

Abstract

In order to discriminate between two probability distributions extensions of Kullback–Leibler (KL) information have been proposed in the literature. In recent years, an extension called cumulative Kullback–Leibler (CKL) information is considered by authors which is closely related to equilibrium distributions. In this paper, we propose an adjusted version of CKL based on equilibrium distributions. Some properties of the proposed measure of divergence are investigated. A test of exponentiality based on the adjusted measure, is proposed. The empirical power of the presented test is calculated and compared with some existing standard tests of exponentiality. The results show that our proposed test, for some important alternative distributions, has better performance than some of the existing tests.     

On entropy of a Pareto distribution in the presence of outliers

ABSTRACT

The aim of this paper is obtaining the amount of information there exists in the Pareto distribution in the presence of outliers. For the sake of this purpose, Shannon entropy, ?-entropy, Fisher information, and Kullback–Leibler distance are computed. Furthermore, a section has been devoted to compare these quantities in these two cases of the Pareto distribution (with outliers and the homogenous case). At the end of this paper, two actual examples, which are related to insurance companies, are brought. A brief summary of which is done in this work is also reported.     

Fitting time series models for longitudinal survey data under informative sampling

The purpose of this paper is to account for informative sampling in fitting time series models, and in particular an autoregressive model of order one, for longitudinal survey data. The idea behind the proposed approach is to extract the model holding for the sample data as a function of the model in the population and the first-order inclusion probabilities, and then fit the sample model using maximum-likelihood, pseudo-maximum-likelihood and estimating equations methods. A new test for sampling ignorability is proposed based on the Kullback–Leibler information measure. Also, we investigate the issue of the sensitivity of the sample model to incorrect specification of the conditional expectations of the sample inclusion probabilities. The simulation study carried out shows that the sample-likelihood-based method produces better estimators than the pseudo-maximum-likelihood method, and that sensitivity to departures from the assumed model is low. Also, we find that both the conventional t-statistic and the Kullback–Leibler information statistic for testing of sampling ignorability perform well under both informative and noninformative sampling designs.     

An efficient correction to the density-based empirical likelihood ratio goodness-of-fit test for the inverse Gaussian distribution

The inverse Gaussian (IG) distribution is widely used to model positively skewed data. An important issue is to develop a powerful goodness-of-fit test for the IG distribution. We propose and examine novel test statistics for testing the IG goodness of fit based on the density-based empirical likelihood (EL) ratio concept. To construct the test statistics, we use a new approach that employs a method of the minimization of the discrimination information loss estimator to minimize Kullback–Leibler type information. The proposed tests are shown to be consistent against wide classes of alternatives. We show that the density-based EL ratio tests are more powerful than the corresponding classical goodness-of-fit tests. The practical efficiency of the tests is illustrated by using real data examples.     

On the goodness-of-fit test based on the ratio of cumulative hazard functions with the Type II censored data

For comparing two cumulative hazard functions, we consider an extension of the Kullback–Leibler information to the cumulative hazard function, which is concerning the ratio of cumulative hazard functions. Then we consider its estimate as a goodness-of-fit test with the Type II censored data. For an exponential null distribution, the proposed test statistic is shown to outperform other test statistics based on the empirical distribution function in the heavy censoring case against the increasing hazard alternatives.     

General treatment of goodness-of-fit tests based on Kullback–Leibler information

This paper introduces a general goodness-of-fit test based on the estimated Kullback–Leibler information. The test uses the Vasicek entropy estimate. Two special cases of the test for location–scale and shape families are discussed. The results are used to introduce goodness-of-fit tests for the uniform, Laplace, Weibull and beta distributions. The critical values and powers for some alternatives are obtained by simulation.     

A new estimator of Kullback–Leibler information and its application in goodness of fit tests*

We propose here a general statistic for the goodness of fit test of statistical distributions. The proposed statistic is constructed based on an estimate of Kullback–Leibler information. The proposed test is consistent and the limiting distribution of the test statistic is derived. Then, the established results are used to introduce goodness of fit tests for the normal, exponential, Laplace and Weibull distributions. A simulation study is carried out for examining the power of the proposed test and to compare it with those of some existing procedures. Finally, some illustrative examples are presented and analysed, and concluding comments are made.     

Goodness-of-fit tests based on Verma Kullback–Leibler information

In this article, a new consistent estimator of Veram’s entropy is introduced. We establish the entropy test based on the new information namely Verma Kullback–Leibler discrimination methodology. The results are used to introduce goodness-of-fit tests for normal and exponential distributions. The root of mean square errors, critical values, and powers for some alternatives are obtained by simulation. The proposed test is compared with other tests.     

Testing Exponentiality Based on Type II Censored Data and a New cdf Estimator

In this article, we use a new cdf estimator to obtain a nanparametric entropy estimate and use it for testing exponentiality and normality. We also use the new cdf estimator to estimate the joint entropy of the Type II censored data which we use for some goodness-of-fit tests based on Kullback–Leibler information and show, by simulation, that it compares favorably with the leading competitor.     

Testing the validity of the exponential model for hybrid Type-I censored data

The exponential distribution has been used in life-testing and reliability studies. In this article, we first express the entropy of Type-I hybrid censoring scheme in terms of hazard function and provide an estimate of the entropy of Type-I hybrid censored data. Then, we construct a goodness-of-fit test statistic based on Kullback–Leibler information for Type-I hybrid censored data. The test statistic is used to test for exponentiality. A Monte Carlo simulation is conducted to obtain the power of the proposed test against various alternatives. Finally, a data example is presented for illustrative purpose.