Testing normality based on new entropy estimators

In this paper, we first introduce two new estimators for estimating the entropy of absolutely continuous random variables. We then compare the introduced estimators with the existing entropy estimators, including the first of such estimators proposed by Dimitriev and Tarasenko [On the estimation functions of the probability density and its derivatives, Theory Probab. Appl. 18 (1973), pp. 628–633]. We next propose goodness-of-fit tests for normality based on the introduced entropy estimators and compare their powers with the powers of other entropy-based tests for normality. Our simulation results show that the introduced estimators perform well in estimating entropy and testing normality.     

Two new estimators of entropy for testing normality

ABSTRACT

We present two new estimators for estimating the entropy of absolutely continuous random variables. Some properties of them are considered, specifically consistency of the first is proved. The introduced estimators are compared with the existing entropy estimators. Also, we propose two new tests for normality based on the introduced entropy estimators and compare their powers with the powers of other tests for normality. The results show that the proposed estimators and test statistics perform very well in estimating entropy and testing normality. A real example is presented and analyzed.     

A new estimator of entropy and its application in testing normality

In this paper, we introduce a new estimator of entropy of a continuous random variable. We compare the proposed estimator with the existing estimators, namely, Vasicek [A test for normality based on sample entropy, J. Roy. Statist. Soc. Ser. B 38 (1976), pp. 54–59], van Es [Estimating functionals related to a density by class of statistics based on spacings, Scand. J. Statist. 19 (1992), pp. 61–72], Correa [A new estimator of entropy, Commun. Statist. Theory and Methods 24 (1995), pp. 2439–2449] and Wieczorkowski-Grzegorewski [Entropy estimators improvements and comparisons, Commun. Statist. Simulation and Computation 28 (1999), pp. 541–567]. We next introduce a new test for normality. By simulation, the powers of the proposed test under various alternatives are compared with normality tests proposed by Vasicek (1976) and Esteban et al. [Monte Carlo comparison of four normality tests using different entropy estimates, Commun. Statist.–Simulation and Computation 30(4) (2001), pp. 761–785].     

Test for normality based on two new estimators of entropy

In this article, two new consistent estimators are introduced of Shannon's entropy that compares root of mean-square error with other estimators. Then we define new tests for normality based on these new estimators. Finally, by simulation, the powers of the proposed tests are compared under different alternatives with other entropy tests for normality.     

On the entropy estimators

The paper introduces an estimator of the entropy of a continuous random variable. The estimator is obtained by modifying the estimator proposed by Ebrahimi et al. [Two measures of sample entropy, Statist. Probab. Lett. 20 (1994), pp. 225–234]. The consistency of the estimator is proved and comparisons are made with Vasicek's estimator [A test for normality based on sample entropy, J. R. Stat. Soc. Ser. B 38 (1976), pp. 54–59], van Es estimator [Estimating functionals related to a density by class of statistics based on spacings, Scand. J. Statist. 19 (1992), pp. 61–72], Ebrahimi et al. estimator and Correa estimator [A new estimator of entropy, Comm. Statist. Theory Methods 24 (1995), pp. 2439–2449]. The results indicate that the proposed estimator has smaller mean-squared error than above estimators. A real example is presented and analysed.     

Asymptotic normality for plug-in estimators of diversity indices on countable alphabets

The plug-in estimator is one of the most popular approaches to the estimation of diversity indices. In this paper, we study its asymptotic distribution for a large class of diversity indices on countable alphabets. In particular, we give conditions for the plug-in estimator to be asymptotically normal, and in the case of uniform distributions, where asymptotic normality fails, we give conditions for the asymptotic distribution to be chi-squared. Our results cover some of the most commonly used indices, including Simpson's index, Reńyi's entropy and Shannon's entropy.     

Moments of nonparametric probability density functions of entropy estimators applied to testing the inverse Gaussian distribution

The inverse Gaussian (IG) distribution is widely used to model data and then it is important to develop efficient goodness of fit tests for this distribution. In this article, we introduce some new test statistics for examining the IG goodness of fit based on correcting moments of nonparametric probability density functions of entropy estimators. These tests are consistent against all alternatives. Critical points and power of the tests are explored by simulation. We show that the proposed tests are more powerful than competitor tests. Finally, the proposed tests are illustrated by real data examples.     

Testing uniformity based on new entropy estimators

In this paper, we first introduce new entropy estimators for distributions with known and bounded supports. Our estimators are obtained by using constrained maximum likelihood estimation of cumulative distribution function for absolutely continuous distributions with known and bounded supports. We prove the consistency of our estimators. Then, we propose uniformity tests based on the proposed entropy estimators and compare their powers with the powers of other tests of uniformity. Our simulation results show that the proposed entropy estimators perform well in estimating entropy and testing uniformity.     

Improvement of goodness-of-fit test for normal distribution based on entropy and power comparison

Vasicek's entropy test for normality is based on sample entropy and a parametric entropy estimator. These estimators are known to have bias in small samples. The use of Vasicek's test could affect the capability of detecting non-normality to some extent. This paper presents an improved entropy test, which uses bias-corrected entropy estimators. A Monte Carlo simulation study is performed to compare the power of the proposed test under several alternative distributions with some other tests. The results report that as anticipated, the improved entropy test has consistently higher power than the ordinary entropy test in nearly all sample sizes and alternatives considered, and compares favorably with other tests.     

A density based empirical likelihood approach for testing bivariate normality

Sample entropy based tests, methods of sieves and Grenander estimation type procedures are known to be very efficient tools for assessing normality of underlying data distributions, in one-dimensional nonparametric settings. Recently, it has been shown that the density based empirical likelihood (EL) concept extends and standardizes these methods, presenting a powerful approach for approximating optimal parametric likelihood ratio test statistics, in a distribution-free manner. In this paper, we discuss difficulties related to constructing density based EL ratio techniques for testing bivariate normality and propose a solution regarding this problem. Toward this end, a novel bivariate sample entropy expression is derived and shown to satisfy the known concept related to bivariate histogram density estimations. Monte Carlo results show that the new density based EL ratio tests for bivariate normality behave very well for finite sample sizes. To exemplify the excellent applicability of the proposed approach, we demonstrate a real data example.     

Tests of fit for the Laplace distribution based on correcting moments of entropy estimators

ABSTRACT

In this paper, we first consider the entropy estimators introduced by Vasicek [A test for normality based on sample entropy. J R Statist Soc, Ser B. 1976;38:54–59], Ebrahimi et al. [Two measures of sample entropy. Stat Probab Lett. 1994;20:225–234], Yousefzadeh and Arghami [Testing exponentiality based on type II censored data and a new cdf estimator. Commun Stat – Simul Comput. 2008;37:1479–1499], Alizadeh Noughabi and Arghami [A new estimator of entropy. J Iran Statist Soc. 2010;9:53–64], and Zamanzade and Arghami [Goodness-of-fit test based on correcting moments of modified entropy estimator. J Statist Comput Simul. 2011;81:2077–2093], and the nonparametric distribution functions corresponding to them. We next introduce goodness-of-fit test statistics for the Laplace distribution based on the moments of nonparametric distribution functions of the aforementioned estimators. We obtain power estimates of the proposed test statistics with Monte Carlo simulation and compare them with the competing test statistics against various alternatives. Performance of the proposed new test statistics is illustrated in real cases.     

On the asymptotic normality of the kernel estimators of the density function and its derivatives under censoring

In this paper, we study asymptotic normality of the kernel estimators of the density function and its derivatives as well as the mode in the randomly right censorship model. The mode estimator is defined as the random variable that maximizes the kernel density estimator. Our results are stated under some suitable conditions upon the kernel function, the smoothing parameter and both distributions functions that appear in this model. Here, the Kaplan–Meier estimator of the distribution function is used to build the estimates. We carry out a simulation study which shows how good the normality works.     

Asymptotic normality of generalized maximum spacing estimators for multivariate observations

In this paper, the maximum spacing method is considered for multivariate observations. Nearest neighbor balls are used as a multidimensional analogue to univariate spacings. A class of information-type measures is used to generalize the concept of maximum spacing estimators of model parameters. Asymptotic normality of these generalized maximum spacing estimators is proved when the assigned model class is correct, that is, the true density is a member of the model class.     

Asymptotic normality of kernel estimators based upon incomplete data

In this paper, we are concerned with nonparametric estimation of the density and the failure rate functions of a random variable X which is at risk of being censored. First, we establish the asymptotic normality of a kernel density estimator in a general censoring setup. Then, we apply our result in order to derive the asymptotic normality of both the density and the failure rate estimators in the cases of right, twice and doubly censored data. Finally, the performance and the asymptotic Gaussian behaviour of the studied estimators, based on either doubly or twice censored data, are illustrated through a simulation study.     

Asymptotic normality of estimators for parameters of a multivariate skew-normal distribution

In this paper, asymptotic normality is established for the parameters of the multivariate skew-normal distribution under two parametrizations. Also, an analytic expression and an asymptotic normal law are derived for the skewness vector of the skew-normal distribution. The estimates are derived using the method of moments. Convergence to the asymptotic distributions is examined both computationally and in a simulation experiment.     

MSE performance of the weighted average estimators consisting of shrinkage estimators

In this paper, we consider a regression model and propose estimators which are the weighted averages of two estimators among three estimators; the Stein-rule (SR), the minimum mean squared error (MMSE), and the adjusted minimum mean-squared error (AMMSE) estimators. It is shown that one of the proposed estimators has smaller mean-squared error (MSE) than the positive-part Stein-rule (PSR) estimator over a moderate region of parameter space when the number of the regression coefficients is small (i.e., 3), and its MSE performance is comparable to the PSR estimator even when the number of the regression coefficients is not so small.     

Asymptotic posterior normality for multiparameter problems

For asymptotic posterior normality in the one-parameter cases, Weng [2003. On Stein's identity for posterior normality. Statist. Sinica 13, 495–506] proposed to use a version of Stein's Identity to write the posterior expectations for functions of a normalized quantity in a form that is more transparent and can be easily analyzed. In the present paper we extend this approach to the multi-parameter cases and compare our conditions with earlier work. Three examples are used to illustrate the application of this method.     

The generalized maximum Tsallis entropy estimators and applications to the Portland cement dataset

Tsallis entropy is a generalized form of entropy and tends to be Shannon entropy when q → 1. Using Tsallis entropy, an alternative estimation methodology (generalized maximum Tsallis entropy) is introduced and used to estimate the parameters in a linear regression model when the basic data are ill-conditioned. We describe the generalized maximum Tsallis entropy and for q = 2 we call that GMET2 estimator. We apply the GMET2 estimator for estimating the linear regression model Y = Xβ + e where the design matrix X is subject to severe multicollinearity. We compared the GMET2, generalized maximum entropy (GME), ordinary least-square (OLS), and inequality restricted least-square (IRLS) estimators on the analyzed dataset on Portland cement.     

More on testing the normality assumptionin the Tobit Model

In a recent volume of this journal, Holden [Testing the normality assumption in the Tobit Model, J. Appl. Stat. 31 (2004) pp. 521–532] presents Monte Carlo evidence comparing several tests for departures from normality in the Tobit Model. This study adds to the work of Holden by considering another test, and several information criteria, for detecting departures from normality in the Tobit Model. The test given here is a modified likelihood ratio statistic based on a partially adaptive estimator of the Censored Regression Model using the approach of Caudill [A partially adaptive estimator for the Censored Regression Model based on a mixture of normal distributions, Working Paper, Department of Economics, Auburn University, 2007]. The information criteria examined include the Akaike’s Information Criterion (AIC), the Consistent AIC (CAIC), the Bayesian information criterion (BIC), and the Akaike’s BIC (ABIC). In terms of fewest ‘rejections’ of a true null, the best performance is exhibited by the CAIC and the BIC, although, like some of the statistics examined by Holden, there are computational difficulties with each.