Goodness-of-fit tests based on Verma Kullback–Leibler information

In this article, a new consistent estimator of Veram’s entropy is introduced. We establish the entropy test based on the new information namely Verma Kullback–Leibler discrimination methodology. The results are used to introduce goodness-of-fit tests for normal and exponential distributions. The root of mean square errors, critical values, and powers for some alternatives are obtained by simulation. The proposed test is compared with other tests.     

Information theoretic results for circular distributions

The broad class of generalized von Mises (GvM) circular distributions has certain optimal properties with respect to information theoretic quantities. It is shown that, under constraints on the trigonometric moments, and using the Kullback–Leibler information as the measure, the closest circular distribution to any other is of the GvM form. The lower bounds for the Kullback–Leibler information in this situation are also provided. The same problem is also considered using a modified version of the Kullback–Leibler information. Finally, series expansions are given for the entropy and the normalizing constants of the GvM distribution.     

On the Choice of Nonparametric Entropy Estimator in Entropy-Based Goodness-of-Fit Test Statistics

Entropy-based goodness-of-fit test statistics can be established by estimating the entropy difference or Kullback–Leibler information, and several entropy-based test statistics based on various entropy estimators have been proposed. In this article, we first give comments on some problems resulting from not satisfying the moment constraints. We then study the choice of the entropy estimator by noting the reason why a test based on a better entropy estimator does not necessarily provide better powers.     

Testing Exponentiality Based on Type II Censored Data and a New cdf Estimator

In this article, we use a new cdf estimator to obtain a nanparametric entropy estimate and use it for testing exponentiality and normality. We also use the new cdf estimator to estimate the joint entropy of the Type II censored data which we use for some goodness-of-fit tests based on Kullback–Leibler information and show, by simulation, that it compares favorably with the leading competitor.     

Testing the validity of the exponential model for hybrid Type-I censored data

The exponential distribution has been used in life-testing and reliability studies. In this article, we first express the entropy of Type-I hybrid censoring scheme in terms of hazard function and provide an estimate of the entropy of Type-I hybrid censored data. Then, we construct a goodness-of-fit test statistic based on Kullback–Leibler information for Type-I hybrid censored data. The test statistic is used to test for exponentiality. A Monte Carlo simulation is conducted to obtain the power of the proposed test against various alternatives. Finally, a data example is presented for illustrative purpose.     

General cumulative Kullback–Leibler information

The cumulative residual Kullback–Leibler information is defined on the semi-infinite (non negative) interval. In this paper, we extend the cumulative residual Kullback–Leibler information to the whole real line and propose a general cumulative Kullback–Leibler information. We study its application to a test for normality in comparison with some competing test statistics based on the empirical distribution function including the well-known tests applied in practice like Kolmogorov–Smirnov, Cramer–von Mises, Anderson–Darling, and other existing tests.     

Shannon Information Properties of the Endpoints of Record Coverage

This paper addresses the largest and the smallest observations, at the times when a new record of either kind (upper or lower) occurs, which are it called the current upper and lower record, respectively. We examine the entropy properties of these statistics, especially the difference between entropy of upper and lower bounds of record coverage. The results are presented for some common parametric families of distributions. Several upper and lower bounds, in terms of the entropy of parent distribution, for the entropy of current records are obtained. It is shown that mutual information, as well as Kullback–Leibler distance between the endpoints of record coverage, Kullback–Leibler distance between data distribution, and current records, are all distribution-free.     

An adjusted cumulative Kullback-Leibler information with application to test of exponentiality

Abstract

In order to discriminate between two probability distributions extensions of Kullback–Leibler (KL) information have been proposed in the literature. In recent years, an extension called cumulative Kullback–Leibler (CKL) information is considered by authors which is closely related to equilibrium distributions. In this paper, we propose an adjusted version of CKL based on equilibrium distributions. Some properties of the proposed measure of divergence are investigated. A test of exponentiality based on the adjusted measure, is proposed. The empirical power of the presented test is calculated and compared with some existing standard tests of exponentiality. The results show that our proposed test, for some important alternative distributions, has better performance than some of the existing tests.     

General treatment of goodness-of-fit tests based on Kullback–Leibler information

This paper introduces a general goodness-of-fit test based on the estimated Kullback–Leibler information. The test uses the Vasicek entropy estimate. Two special cases of the test for location–scale and shape families are discussed. The results are used to introduce goodness-of-fit tests for the uniform, Laplace, Weibull and beta distributions. The critical values and powers for some alternatives are obtained by simulation.     

An entropy test for the Rayleigh distribution and power comparison

In this paper, a goodness-of-fit test is proposed for the Rayleigh distribution. This test is based on the Kullback–Leibler discrimination methodology proposed by Song [2002, Goodness of fit tests based on Kullback–Leibler discrimination, IEEE Trans. Inf. Theory 48(5), pp. 1103–1117]. The critical values and powers for some alternatives are obtained by simulation. The proposed test is compared with other tests, namely Kolmogorov–Smirnov, Kuiper, Cramer–von Mises, Watson and Anderson–Darling. The use of the proposed test is shown in a real example.     

Testing exponentiality using transformed data

In this paper, we introduce a test for uniformity and use it as the second stage of an exact goodness-of-fit test of exponentiality. By simulation, the powers of the proposed test under various alternatives are compared with exponentiality test based on Kullback–Leibler information proposed by Ebrahimi et al. [N. Ebrahimi, M. Habibullah, and E.S. Soofi, Testing exponentiality based on Kullback–Leiber information, J. R. Statist. Soc. Ser. B 54 (1992), pp. 739–748]. The results are impressive, i.e. the proposed test has higher power than the test based on entropy.     

Estimation of the exponential Pareto II distribution parameters

In this article, we estimate the parameters of exponential Pareto II distribution by two new methods. The first one is based on the principle of maximum entropy (POME) and the second is by Kullback–Leibler divergence of survival function (KLS). Monte Carlo simulated data are used to evaluate these methods and compare them with the maximum likelihood method. Finally, we fit this distribution to a set of real data by estimation procedures.     

Testing for Covariate Effect in the Cox Proportional Hazards Regression Model

This article presents methods for testing covariate effect in the Cox proportional hazards model based on Kullback–Leibler divergence and Renyi's information measure. Renyi's measure is referred to as the information divergence of order γ (γ ≠ 1) between two distributions. In the limiting case γ → 1, Renyi's measure becomes Kullback–Leibler divergence. In our case, the distributions correspond to the baseline and one possibly due to a covariate effect. Our proposed statistics are simple transformations of the parameter vector in the Cox proportional hazards model, and are compared with the Wald, likelihood ratio and score tests that are widely used in practice. Finally, the methods are illustrated using two real-life data sets.     

Some properties of Lin–Wong divergence on the past lifetime data

Measures of statistical divergence are used to assess mutual similarities between distributions of multiple variables through a variety of methodologies including Shannon entropy and Csiszar divergence. Modified measures of statistical divergence are introduced throughout the present article. Those modified measures are related to the Lin–Wong (LW) divergence applied on the past lifetime data. Accordingly, the relationship between Fisher information and the LW divergence measure was explored when applied on the past lifetime data. Throughout this study, a number of relations are proposed between various assessment methods which implement the Jensen–Shannon, Jeffreys, and Hellinger divergence measures. Also, relations between the LW measure and the Kullback–Leibler (KL) measures for past lifetime data were examined. Furthermore, the present study discusses the relationship between the proposed ordering scheme and the distance interval between LW and KL measures under certain conditions.     

Discrimination Measures for Fractional Integrated Models Based on Wavelets

Discrimination measures have been well developed for stationary time series. However in a large number of phenomena, long-term dependencies are involved. In this article, we are dealing with discrimination of fractional integrated models. Kullback–Leibler and Chernoff's discrimination measures are approximated, using the discrete wavelet transform (DWT) for discrimination of these time series classes. The simulation study indicates low misclassification rate, related to the approximations of Kullback–Leibler and Chernoff discrimination measures. Application to problem of classifying seismic data showed that our procedure performs as well as other procedures.     

On the goodness-of-fit test based on the ratio of cumulative hazard functions with the Type II censored data

For comparing two cumulative hazard functions, we consider an extension of the Kullback–Leibler information to the cumulative hazard function, which is concerning the ratio of cumulative hazard functions. Then we consider its estimate as a goodness-of-fit test with the Type II censored data. For an exponential null distribution, the proposed test statistic is shown to outperform other test statistics based on the empirical distribution function in the heavy censoring case against the increasing hazard alternatives.     

Quantile-based cumulative Kullback–Leibler divergence

The paper introduces a quantile-based cumulative Kullback–Leibler divergence and study its various properties. Unlike the distribution function approach, the quantile-based measure possesses some unique properties. The quantile functions used in many applied works do not have any tractable distribution functions where the proposed measure is a useful tool to compute the distance between two random variables. Some useful bounds are obtained for quantile-based residual cumulative Kullback–Leibler divergence and quantile-based reliability measures. Characterization results based on the functional forms of quantile-based residual Kullback–Leibler divergence are obtained for some well-known life distributions, namely exponential, Pareto II and beta.     

On entropy of a Pareto distribution in the presence of outliers

ABSTRACT

The aim of this paper is obtaining the amount of information there exists in the Pareto distribution in the presence of outliers. For the sake of this purpose, Shannon entropy, ?-entropy, Fisher information, and Kullback–Leibler distance are computed. Furthermore, a section has been devoted to compare these quantities in these two cases of the Pareto distribution (with outliers and the homogenous case). At the end of this paper, two actual examples, which are related to insurance companies, are brought. A brief summary of which is done in this work is also reported.     

On the Kullback–Leibler information of hybrid censored data

ABSTRACT

A hybrid censoring is a mixture of Type I and Type II censoring where the experiment terminates when either rth failure or predetermined censoring time comes first or later. In this article, we consider order statistics of the Type I censored data and provide a simple expression for their Kullback–Leibler (KL) information. Then, we provide the expressions for the KL information of the Type I and Type II hybrid censored data.     

Nonparametric estimation of the entropy using a ranked set sample

This article is concerned with nonparametric estimation of the entropy in ranked set sampling. Theoretical properties of the proposed estimator are studied. The proposed estimator is compared with the rival estimator in simple random sampling. The applications of the proposed estimator to the mutual information estimation as well as estimation of the Kullback–Leibler divergence are provided. Several Monté-Carlo simulation studies are conducted to examine the performance of the estimator. The results are applied to the longleaf pine (Pinus palustris) trees and the body fat percentage datasets to illustrate applicability of theoretical results.