Some Characterizations Based on Entropy of Order Statistics and Record Values

In the literature of information theory, Shannon entropy plays an important role and in the context of reliability theory, order statistics and record values are used for statistical modeling. The aim of this article is characterizing the parent distributions based on Shannon entropy of order statistics and record values. It is shown that the equality of the Shannon information in order statistics or record values can determine uniquely the parent distribution. The exponential distribution is characterized through maximizing Shannon entropy of record values under some constraints. The results are useful in the modeling problems.     

An Estimator of Shannon Entropy of Beta-Generated Distributions and a Goodness-of-Fit Test

The aim of this article is twofold: on the one hand to introduce and study some of the statistical properties of an estimator for the Shannon entropy and on the other hand to develop a goodness-of-fit test for beta-generated distributions and the distribution of order statistics. Beta-generated distributions are a broad class of univariate distributions which has received great attention during the last 15 years, as it obeys nice properties and it extends the distribution of order statistics. The proposed estimator of Shannon entropy of beta-generated distributions is motivated by the respective Vasicek’s estimator, as the latter one is tailored to the class of the beta-generated distributions and the distribution of order statistics. The estimator of Shannon entropy is defined and its consistency is studied. It is, moreover, exploited to build a goodness-of-fit test for the beta-generated distribution and the distribution of order statistics. Simulations are performed to examine the small- and moderate-sample properties of the proposed estimator and to compare the power of the proposed test with the power of competitors under a variety of alternatives.     

Characterizations based on generalized cumulative residual entropy functions

The Shannon entropy and the cumulative residual entropy (CRE) of a random variable are useful tools in probability theory. Recently, a new concept called generalized cumulative residual entropy (GCRE) of order n was introduced and studied. It is related with the record values of a sequence of i.i.d. random variables and with the relevation transform. In this paper, we show that, under some assumptions, the GCRE function of a fixed order n uniquely determines the distribution function. Some characterizations of particular probability models are obtained from this general result.     

Exponentiated Pareto distributions

Most Pareto distributions are defined on one side of the real line. For wider applicability, we introduce five exponentiated Pareto distributions and derive several of their properties including the moment generating function, expectation, variance, skewness, kurtosis, Shannon entropy, and the Rényi entropy.     

On concave entropies of discrete systems

A new class of entropy functions of discrete systems, the class of concave entropies, is introduced. Each concave entropy function satisfies the fundamental axioms of general entropies. The Shannon and trigonometric entropies belong to the class of concave entropies. Each of the classes of Renyi and polynomial entropies contains a subclass of concave entropies. A sufficient condition is given, under which the total entropy tends to ∞, when the number N of probability components approaches ∞.     

Entropy-based goodness-of-fit tests for the Pareto I distribution

In this article, having observed the generalized order statistics in a sample, we construct a test for the hypothesis that the underlying distribution is the Pareto I distribution. The Shannon entropy of generalized order statistics is used to test the null hypothesis.     

Hypothesis testing with Rao's quadratic entropy and its application to Dinosaur biodiversity

Entropy indices, such as Shannon entropy and Gini-Simpson index, have been used for analysing biological diversities. However, these entropy indices are based on abundance of the species only and they do not take differences between the species into consideration. Rao's quadratic entropy has found many applications in different fields including ecology. Further, the quadratic entropy (QE) index is the only ecological diversity index that reflects both the differences and abundances of the species. The problem of testing of hypothesis of the equality of QEs is formulated as a problem of comparing practical equivalence intervals. Simulation experiments are used to compare various equivalence intervals. Previously analyzed dinosaur data are used to illustrate the methods for determining biodiversity.     

Some properties of Lin–Wong divergence on the past lifetime data

Measures of statistical divergence are used to assess mutual similarities between distributions of multiple variables through a variety of methodologies including Shannon entropy and Csiszar divergence. Modified measures of statistical divergence are introduced throughout the present article. Those modified measures are related to the Lin–Wong (LW) divergence applied on the past lifetime data. Accordingly, the relationship between Fisher information and the LW divergence measure was explored when applied on the past lifetime data. Throughout this study, a number of relations are proposed between various assessment methods which implement the Jensen–Shannon, Jeffreys, and Hellinger divergence measures. Also, relations between the LW measure and the Kullback–Leibler (KL) measures for past lifetime data were examined. Furthermore, the present study discusses the relationship between the proposed ordering scheme and the distance interval between LW and KL measures under certain conditions.     

The generalized maximum Tsallis entropy estimators and applications to the Portland cement dataset

Tsallis entropy is a generalized form of entropy and tends to be Shannon entropy when q → 1. Using Tsallis entropy, an alternative estimation methodology (generalized maximum Tsallis entropy) is introduced and used to estimate the parameters in a linear regression model when the basic data are ill-conditioned. We describe the generalized maximum Tsallis entropy and for q = 2 we call that GMET2 estimator. We apply the GMET2 estimator for estimating the linear regression model Y = Xβ + e where the design matrix X is subject to severe multicollinearity. We compared the GMET2, generalized maximum entropy (GME), ordinary least-square (OLS), and inequality restricted least-square (IRLS) estimators on the analyzed dataset on Portland cement.     

The McDonald arcsine distribution: a new model to proportional data

We propose a new three-parameter continuous model called the McDonald arcsine distribution, which is a very competitive model to the beta, beta type I and Kumaraswamy distributions for modelling rates and proportions. We provide a mathematical treatment of the new distribution including explicit expressions for the density function, moments, generating and quantile functions, mean deviations, two probability measures based on the Bonferroni and Lorenz curves, Shannon entropy, Rényi entropy and cumulative residual entropy. Maximum likelihood is used to estimate the model parameters and the expected information matrix is determined. An application of the proposed model to real data shows that it can give consistently a better fit than other important statistical models.     

Maximum entropy sampling and optimal Bayesian experimental design

When Shannon entropy is used as a criterion in the optimal design of experiments, advantage can be taken of the classical identity representing the joint entropy of parameters and observations as the sum of the marginal entropy of the observations and the preposterior conditional entropy of the parameters. Following previous work in which this idea was used in spatial sampling, the method is applied to standard parameterized Bayesian optimal experimental design. Under suitable conditions, which include non-linear as well as linear regression models, it is shown in a few steps that maximizing the marginal entropy of the sample is equivalent to minimizing the preposterior entropy, the usual Bayesian criterion, thus avoiding the use of conditional distributions. It is shown using this marginal formulation that under normality assumptions every standard model which has a two-point prior distribution on the parameters gives an optimal design supported on a single point. Other results include a new asymptotic formula which applies as the error variance is large and bounds on support size.     

A generalized normal distribution

Undoubtedly, the normal distribution is the most popular distribution in statistics. In this paper, we introduce a natural generalization of the normal distribution and provide a comprehensive treatment of its mathematical properties. We derive expressions for the nth moment, the nth central moment, variance, skewness, kurtosis, mean deviation about the mean, mean deviation about the median, Rényi entropy, Shannon entropy, and the asymptotic distribution of the extreme order statistics. We also discuss estimation by the methods of moments and maximum likelihood and provide an expression for the Fisher information matrix.     

q-Esscher transformed Laplace distribution

Pathway idea is a switching mechanism by which one can go from one functional form to another, and to yet another. In this paper, we introduce a q-Esscher transformed Laplace distribution, which is a stretched model for Esscher transformed Laplace distribution, obtained by introducing a new pathway parameter q, which facilitates a slow transition to the Esscher transformed Laplace distribution as q → 1. This pathway model can be obtained by optimizing Mathai’s generalized entropy with more general setup, which is a generalization of various entropy measures due to Shannon and others. The various properties of the q-Esscher transformed Laplace distribution are studied and its applications are discussed.     

Discrimination among bivariate beta-generated distributions

Some properties of the general families of bivariate distributions generated by beta dependent random variables are derived and discussed here. Some classic measures of dependence and information are derived, and their behaviours and properties are discussed as well. Finally, a discrimination procedure within this general family of bivariate distributions is proposed based on Shannon entropy. A real-life example is presented to illustrate the model as well as the inferential results developed here.     

Statistical estimation of quadratic Rényi entropy for a stationary m-dependent sequence

The Rényi entropy is a generalisation of the Shannon entropy and is widely used in mathematical statistics and applied sciences for quantifying the uncertainty in a probability distribution. We consider estimation of the quadratic Rényi entropy and related functionals for the marginal distribution of a stationary m-dependent sequence. The U-statistic estimators under study are based on the number of ε-close vector observations in the corresponding sample. A variety of asymptotic properties for these estimators are obtained (e.g. consistency, asymptotic normality, and Poisson convergence). The results can be used in diverse statistical and computer science problems whenever the conventional independence assumption is too strong (e.g. ε-keys in time series databases and distribution identification problems for dependent samples).     

On some characteristics of bivariate chi-square distribution

Product moments of bivariate chi-square distribution have been derived in closed forms. Finite expressions have been derived for product moments of integer orders. Marginal and conditional distributions, conditional moments, coefficient of skewness and kurtosis of conditional distribution have also been discussed. Shannon entropy of the distribution is also derived. We also discuss the Bayesian estimation of a parameter of the distribution. Results match with the independent case when the variables are uncorrelated.     

Maximum entropy estimation of income share function from generalized Gini index

Following Sir Anthony and Atkinson who started thinking about the insensitivity of the Gini index to income shares of the lower and the upper income groups, a generalization of the classical Gini index was introduced by Kakwani, Donaldson, Weymark and Yitzhaki which is sensitive to both high and low incomes. In this paper, the maximum entropy method is used to estimate the underlying true income share function based on the limited information of the generalized Gini index about the income shares of a population's percentiles. The income share function is estimated through maximizing both the Shannon entropy and the second-order entropy. In the end, through parametric bootstrap and analyzing a real dataset, the results are compared with the estimator of the share function, which is obtained based on the total information. In contrast to the classic Gini index, the derived share function based on the generalized Gini index provides more accurate approximations for income shares of the lower and the upper percentiles.     

On Some Properties of the Beta Normal Distribution

The beta normal distribution is a generalization of both the normal distribution and the normal order statistics. Some of its mathematical properties and a few applications have been studied in the literature. We provide a better foundation for some properties and an analytical study of its bimodality. The hazard rate function and the limiting behavior are examined. We derive explicit expressions for moments, generating function, mean deviations using a power series expansion for the quantile function, and Shannon entropy.     

The Gamma–Kumaraswamy distribution: An alternative to Gamma distribution

In this article, a new generalization of the Kumaraswamy distribution, namely the Gamma–Kumaraswamy distribution, is defined and studied. Various properties of the Gamma–Kumaraswamy are obtained. The structural analysis of the distribution in this article includes limiting behavior, mode, quantiles, moments, skewness, kurtosis, Shannon’s entropy, and order statistics. The method of maximum likelihood estimation is proposed for estimating the model parameters. For illustrative purposes, two real datasets are analyzed as application of the Gamma–Kumaraswamy distribution.