Modified entropy estimators for testing normality

In this paper, we present two new estimators for the entropy of absolutely continuous random variables and consider some of their properties. Consistency of the first estimator is shown by Monte Carlo method, and the consistency of the second estimator is proved theoretically. Using these estimators, two new tests for normality are presented and their powers are compared with the other entropy-based tests. Simulation results show that the proposed estimators and test statistics perform very well. Finally, a real example is presented and analysed.     

Test for normality based on two new estimators of entropy

In this article, two new consistent estimators are introduced of Shannon's entropy that compares root of mean-square error with other estimators. Then we define new tests for normality based on these new estimators. Finally, by simulation, the powers of the proposed tests are compared under different alternatives with other entropy tests for normality.     

A new estimator of entropy and its application in testing normality

In this paper, we introduce a new estimator of entropy of a continuous random variable. We compare the proposed estimator with the existing estimators, namely, Vasicek [A test for normality based on sample entropy, J. Roy. Statist. Soc. Ser. B 38 (1976), pp. 54–59], van Es [Estimating functionals related to a density by class of statistics based on spacings, Scand. J. Statist. 19 (1992), pp. 61–72], Correa [A new estimator of entropy, Commun. Statist. Theory and Methods 24 (1995), pp. 2439–2449] and Wieczorkowski-Grzegorewski [Entropy estimators improvements and comparisons, Commun. Statist. Simulation and Computation 28 (1999), pp. 541–567]. We next introduce a new test for normality. By simulation, the powers of the proposed test under various alternatives are compared with normality tests proposed by Vasicek (1976) and Esteban et al. [Monte Carlo comparison of four normality tests using different entropy estimates, Commun. Statist.–Simulation and Computation 30(4) (2001), pp. 761–785].     

Two new estimators of entropy for testing normality

ABSTRACT

We present two new estimators for estimating the entropy of absolutely continuous random variables. Some properties of them are considered, specifically consistency of the first is proved. The introduced estimators are compared with the existing entropy estimators. Also, we propose two new tests for normality based on the introduced entropy estimators and compare their powers with the powers of other tests for normality. The results show that the proposed estimators and test statistics perform very well in estimating entropy and testing normality. A real example is presented and analyzed.     

Testing uniformity based on new entropy estimators

In this paper, we first introduce new entropy estimators for distributions with known and bounded supports. Our estimators are obtained by using constrained maximum likelihood estimation of cumulative distribution function for absolutely continuous distributions with known and bounded supports. We prove the consistency of our estimators. Then, we propose uniformity tests based on the proposed entropy estimators and compare their powers with the powers of other tests of uniformity. Our simulation results show that the proposed entropy estimators perform well in estimating entropy and testing uniformity.     

On the entropy estimators

The paper introduces an estimator of the entropy of a continuous random variable. The estimator is obtained by modifying the estimator proposed by Ebrahimi et al. [Two measures of sample entropy, Statist. Probab. Lett. 20 (1994), pp. 225–234]. The consistency of the estimator is proved and comparisons are made with Vasicek's estimator [A test for normality based on sample entropy, J. R. Stat. Soc. Ser. B 38 (1976), pp. 54–59], van Es estimator [Estimating functionals related to a density by class of statistics based on spacings, Scand. J. Statist. 19 (1992), pp. 61–72], Ebrahimi et al. estimator and Correa estimator [A new estimator of entropy, Comm. Statist. Theory Methods 24 (1995), pp. 2439–2449]. The results indicate that the proposed estimator has smaller mean-squared error than above estimators. A real example is presented and analysed.     

Asymptotic normality for plug-in estimators of diversity indices on countable alphabets

The plug-in estimator is one of the most popular approaches to the estimation of diversity indices. In this paper, we study its asymptotic distribution for a large class of diversity indices on countable alphabets. In particular, we give conditions for the plug-in estimator to be asymptotically normal, and in the case of uniform distributions, where asymptotic normality fails, we give conditions for the asymptotic distribution to be chi-squared. Our results cover some of the most commonly used indices, including Simpson's index, Reńyi's entropy and Shannon's entropy.     

Improvement of goodness-of-fit test for normal distribution based on entropy and power comparison

Vasicek's entropy test for normality is based on sample entropy and a parametric entropy estimator. These estimators are known to have bias in small samples. The use of Vasicek's test could affect the capability of detecting non-normality to some extent. This paper presents an improved entropy test, which uses bias-corrected entropy estimators. A Monte Carlo simulation study is performed to compare the power of the proposed test under several alternative distributions with some other tests. The results report that as anticipated, the improved entropy test has consistently higher power than the ordinary entropy test in nearly all sample sizes and alternatives considered, and compares favorably with other tests.     

Tests of fit for the Laplace distribution based on correcting moments of entropy estimators

ABSTRACT

In this paper, we first consider the entropy estimators introduced by Vasicek [A test for normality based on sample entropy. J R Statist Soc, Ser B. 1976;38:54–59], Ebrahimi et al. [Two measures of sample entropy. Stat Probab Lett. 1994;20:225–234], Yousefzadeh and Arghami [Testing exponentiality based on type II censored data and a new cdf estimator. Commun Stat – Simul Comput. 2008;37:1479–1499], Alizadeh Noughabi and Arghami [A new estimator of entropy. J Iran Statist Soc. 2010;9:53–64], and Zamanzade and Arghami [Goodness-of-fit test based on correcting moments of modified entropy estimator. J Statist Comput Simul. 2011;81:2077–2093], and the nonparametric distribution functions corresponding to them. We next introduce goodness-of-fit test statistics for the Laplace distribution based on the moments of nonparametric distribution functions of the aforementioned estimators. We obtain power estimates of the proposed test statistics with Monte Carlo simulation and compare them with the competing test statistics against various alternatives. Performance of the proposed new test statistics is illustrated in real cases.     

Asymptotic normality of kernel estimators based upon incomplete data

In this paper, we are concerned with nonparametric estimation of the density and the failure rate functions of a random variable X which is at risk of being censored. First, we establish the asymptotic normality of a kernel density estimator in a general censoring setup. Then, we apply our result in order to derive the asymptotic normality of both the density and the failure rate estimators in the cases of right, twice and doubly censored data. Finally, the performance and the asymptotic Gaussian behaviour of the studied estimators, based on either doubly or twice censored data, are illustrated through a simulation study.     

Goodness-of-fit test based on correcting moments of modified entropy estimator

In this paper, we first propose a new estimator of entropy for continuous random variables. Our estimator is obtained by correcting the coefficients of Vasicek's [A test for normality based on sample entropy, J. R. Statist. Soc. Ser. B 38 (1976), pp. 54–59] entropy estimator. We prove the consistency of our estimator. Monte Carlo studies show that our estimator is better than the entropy estimators proposed by Vasicek, Ebrahimi et al. [Two measures of sample entropy, Stat. Probab. Lett. 20 (1994), pp. 225–234] and Correa [A new estimator of entropy, Commun. Stat. Theory Methods 24 (1995), pp. 2439–2449] in terms of root mean square error. We then derive the non-parametric distribution function corresponding to our proposed entropy estimator as a piece-wise uniform distribution. We also introduce goodness-of-fit tests for testing exponentiality and normality based on the said distribution and compare its performance with their leading competitors.     

On the asymptotic normality of the kernel estimators of the density function and its derivatives under censoring

In this paper, we study asymptotic normality of the kernel estimators of the density function and its derivatives as well as the mode in the randomly right censorship model. The mode estimator is defined as the random variable that maximizes the kernel density estimator. Our results are stated under some suitable conditions upon the kernel function, the smoothing parameter and both distributions functions that appear in this model. Here, the Kaplan–Meier estimator of the distribution function is used to build the estimates. We carry out a simulation study which shows how good the normality works.     

Testing high-dimensional normality based on classical skewness and Kurtosis with a possible small sample size

Abstract

By using the idea of principal component analysis, we propose an approach to applying the classical skewness and kurtosis statistics for detecting univariate normality to testing high-dimensional normality. High-dimensional sample data are projected to the principal component directions on which the classical skewness and kurtosis statistics can be constructed. The theory of spherical distributions is employed to derive the null distributions of the combined statistics constructed from the principal component directions. A Monte Carlo study is carried out to demonstrate the performance of the statistics on controlling type I error rates and a simple power comparison with some existing statistics. The effectiveness of the proposed statistics is illustrated by two real-data examples.     

Moments of nonparametric probability density functions of entropy estimators applied to testing the inverse Gaussian distribution

The inverse Gaussian (IG) distribution is widely used to model data and then it is important to develop efficient goodness of fit tests for this distribution. In this article, we introduce some new test statistics for examining the IG goodness of fit based on correcting moments of nonparametric probability density functions of entropy estimators. These tests are consistent against all alternatives. Critical points and power of the tests are explored by simulation. We show that the proposed tests are more powerful than competitor tests. Finally, the proposed tests are illustrated by real data examples.     

Testing exponentiality based on characterizations of the exponential distribution

In this paper, we first present two characterizations of the exponential distribution and next introduce three exact goodness-of-fit test for exponentiality. By simulation, the powers of the proposed tests under various alternatives are compared with the existing tests.     

Confidence regions based on inefficient estimators

We consider an unbiased estimator of a function of mean value parameters, which is not efficient. This inefficient estimator is correlated with a residual vector. Thus, if a unit dispersion is unknown, it is impossible to determine the correct confidence region for a function of mean value parameters via a standard estimator of an unknown dispersion with the exception of the case when the ordinary least squares (OLS) estimator is considered in a model with a special covariance structure such that the OLS and the generalized least squares (GLS) estimator are the same, that is the OLS estimator is efficient. Two different estimators of a unit dispersion independent of an inefficient estimator are derived in a singular linear statistical model. Their quality was verified by simulations for several types of experimental designs. Two new estimators of the unit dispersion were compared with the standard estimators based on the GLS and the OLS estimators of the function of the mean value parameters. The OLS estimator was considered in the incorrect model with a different covariance matrix such that the originally inefficient estimator became efficient. The numerical examples led to a slightly surprising result which seems to be due to data behaviour. An example from geodetic practice is presented in the paper.     

The generalized maximum Tsallis entropy estimators and applications to the Portland cement dataset

Tsallis entropy is a generalized form of entropy and tends to be Shannon entropy when q → 1. Using Tsallis entropy, an alternative estimation methodology (generalized maximum Tsallis entropy) is introduced and used to estimate the parameters in a linear regression model when the basic data are ill-conditioned. We describe the generalized maximum Tsallis entropy and for q = 2 we call that GMET2 estimator. We apply the GMET2 estimator for estimating the linear regression model Y = Xβ + e where the design matrix X is subject to severe multicollinearity. We compared the GMET2, generalized maximum entropy (GME), ordinary least-square (OLS), and inequality restricted least-square (IRLS) estimators on the analyzed dataset on Portland cement.     

Asymptotic normality of generalized maximum spacing estimators for multivariate observations

In this paper, the maximum spacing method is considered for multivariate observations. Nearest neighbor balls are used as a multidimensional analogue to univariate spacings. A class of information-type measures is used to generalize the concept of maximum spacing estimators of model parameters. Asymptotic normality of these generalized maximum spacing estimators is proved when the assigned model class is correct, that is, the true density is a member of the model class.     

A density based empirical likelihood approach for testing bivariate normality

Sample entropy based tests, methods of sieves and Grenander estimation type procedures are known to be very efficient tools for assessing normality of underlying data distributions, in one-dimensional nonparametric settings. Recently, it has been shown that the density based empirical likelihood (EL) concept extends and standardizes these methods, presenting a powerful approach for approximating optimal parametric likelihood ratio test statistics, in a distribution-free manner. In this paper, we discuss difficulties related to constructing density based EL ratio techniques for testing bivariate normality and propose a solution regarding this problem. Toward this end, a novel bivariate sample entropy expression is derived and shown to satisfy the known concept related to bivariate histogram density estimations. Monte Carlo results show that the new density based EL ratio tests for bivariate normality behave very well for finite sample sizes. To exemplify the excellent applicability of the proposed approach, we demonstrate a real data example.