Testing normality based on new entropy estimators

In this paper, we first introduce two new estimators for estimating the entropy of absolutely continuous random variables. We then compare the introduced estimators with the existing entropy estimators, including the first of such estimators proposed by Dimitriev and Tarasenko [On the estimation functions of the probability density and its derivatives, Theory Probab. Appl. 18 (1973), pp. 628–633]. We next propose goodness-of-fit tests for normality based on the introduced entropy estimators and compare their powers with the powers of other entropy-based tests for normality. Our simulation results show that the introduced estimators perform well in estimating entropy and testing normality.     

Two new estimators of entropy for testing normality

ABSTRACT

We present two new estimators for estimating the entropy of absolutely continuous random variables. Some properties of them are considered, specifically consistency of the first is proved. The introduced estimators are compared with the existing entropy estimators. Also, we propose two new tests for normality based on the introduced entropy estimators and compare their powers with the powers of other tests for normality. The results show that the proposed estimators and test statistics perform very well in estimating entropy and testing normality. A real example is presented and analyzed.     

A new estimator of entropy and its application in testing normality

In this paper, we introduce a new estimator of entropy of a continuous random variable. We compare the proposed estimator with the existing estimators, namely, Vasicek [A test for normality based on sample entropy, J. Roy. Statist. Soc. Ser. B 38 (1976), pp. 54–59], van Es [Estimating functionals related to a density by class of statistics based on spacings, Scand. J. Statist. 19 (1992), pp. 61–72], Correa [A new estimator of entropy, Commun. Statist. Theory and Methods 24 (1995), pp. 2439–2449] and Wieczorkowski-Grzegorewski [Entropy estimators improvements and comparisons, Commun. Statist. Simulation and Computation 28 (1999), pp. 541–567]. We next introduce a new test for normality. By simulation, the powers of the proposed test under various alternatives are compared with normality tests proposed by Vasicek (1976) and Esteban et al. [Monte Carlo comparison of four normality tests using different entropy estimates, Commun. Statist.–Simulation and Computation 30(4) (2001), pp. 761–785].     

On the entropy estimators

The paper introduces an estimator of the entropy of a continuous random variable. The estimator is obtained by modifying the estimator proposed by Ebrahimi et al. [Two measures of sample entropy, Statist. Probab. Lett. 20 (1994), pp. 225–234]. The consistency of the estimator is proved and comparisons are made with Vasicek's estimator [A test for normality based on sample entropy, J. R. Stat. Soc. Ser. B 38 (1976), pp. 54–59], van Es estimator [Estimating functionals related to a density by class of statistics based on spacings, Scand. J. Statist. 19 (1992), pp. 61–72], Ebrahimi et al. estimator and Correa estimator [A new estimator of entropy, Comm. Statist. Theory Methods 24 (1995), pp. 2439–2449]. The results indicate that the proposed estimator has smaller mean-squared error than above estimators. A real example is presented and analysed.     

Moments of nonparametric probability density functions of entropy estimators applied to testing the inverse Gaussian distribution

The inverse Gaussian (IG) distribution is widely used to model data and then it is important to develop efficient goodness of fit tests for this distribution. In this article, we introduce some new test statistics for examining the IG goodness of fit based on correcting moments of nonparametric probability density functions of entropy estimators. These tests are consistent against all alternatives. Critical points and power of the tests are explored by simulation. We show that the proposed tests are more powerful than competitor tests. Finally, the proposed tests are illustrated by real data examples.     

Testing uniformity based on new entropy estimators

In this paper, we first introduce new entropy estimators for distributions with known and bounded supports. Our estimators are obtained by using constrained maximum likelihood estimation of cumulative distribution function for absolutely continuous distributions with known and bounded supports. We prove the consistency of our estimators. Then, we propose uniformity tests based on the proposed entropy estimators and compare their powers with the powers of other tests of uniformity. Our simulation results show that the proposed entropy estimators perform well in estimating entropy and testing uniformity.     

Improvement of goodness-of-fit test for normal distribution based on entropy and power comparison

Vasicek's entropy test for normality is based on sample entropy and a parametric entropy estimator. These estimators are known to have bias in small samples. The use of Vasicek's test could affect the capability of detecting non-normality to some extent. This paper presents an improved entropy test, which uses bias-corrected entropy estimators. A Monte Carlo simulation study is performed to compare the power of the proposed test under several alternative distributions with some other tests. The results report that as anticipated, the improved entropy test has consistently higher power than the ordinary entropy test in nearly all sample sizes and alternatives considered, and compares favorably with other tests.     

A density based empirical likelihood approach for testing bivariate normality

Sample entropy based tests, methods of sieves and Grenander estimation type procedures are known to be very efficient tools for assessing normality of underlying data distributions, in one-dimensional nonparametric settings. Recently, it has been shown that the density based empirical likelihood (EL) concept extends and standardizes these methods, presenting a powerful approach for approximating optimal parametric likelihood ratio test statistics, in a distribution-free manner. In this paper, we discuss difficulties related to constructing density based EL ratio techniques for testing bivariate normality and propose a solution regarding this problem. Toward this end, a novel bivariate sample entropy expression is derived and shown to satisfy the known concept related to bivariate histogram density estimations. Monte Carlo results show that the new density based EL ratio tests for bivariate normality behave very well for finite sample sizes. To exemplify the excellent applicability of the proposed approach, we demonstrate a real data example.     

Tests of fit for the Laplace distribution based on correcting moments of entropy estimators

ABSTRACT

In this paper, we first consider the entropy estimators introduced by Vasicek [A test for normality based on sample entropy. J R Statist Soc, Ser B. 1976;38:54–59], Ebrahimi et al. [Two measures of sample entropy. Stat Probab Lett. 1994;20:225–234], Yousefzadeh and Arghami [Testing exponentiality based on type II censored data and a new cdf estimator. Commun Stat – Simul Comput. 2008;37:1479–1499], Alizadeh Noughabi and Arghami [A new estimator of entropy. J Iran Statist Soc. 2010;9:53–64], and Zamanzade and Arghami [Goodness-of-fit test based on correcting moments of modified entropy estimator. J Statist Comput Simul. 2011;81:2077–2093], and the nonparametric distribution functions corresponding to them. We next introduce goodness-of-fit test statistics for the Laplace distribution based on the moments of nonparametric distribution functions of the aforementioned estimators. We obtain power estimates of the proposed test statistics with Monte Carlo simulation and compare them with the competing test statistics against various alternatives. Performance of the proposed new test statistics is illustrated in real cases.     

On the asymptotic normality of the kernel estimators of the density function and its derivatives under censoring

In this paper, we study asymptotic normality of the kernel estimators of the density function and its derivatives as well as the mode in the randomly right censorship model. The mode estimator is defined as the random variable that maximizes the kernel density estimator. Our results are stated under some suitable conditions upon the kernel function, the smoothing parameter and both distributions functions that appear in this model. Here, the Kaplan–Meier estimator of the distribution function is used to build the estimates. We carry out a simulation study which shows how good the normality works.     

Asymptotic normality of estimators for parameters of a multivariate skew-normal distribution

In this paper, asymptotic normality is established for the parameters of the multivariate skew-normal distribution under two parametrizations. Also, an analytic expression and an asymptotic normal law are derived for the skewness vector of the skew-normal distribution. The estimates are derived using the method of moments. Convergence to the asymptotic distributions is examined both computationally and in a simulation experiment.     

Asymptotic posterior normality for multiparameter problems

For asymptotic posterior normality in the one-parameter cases, Weng [2003. On Stein's identity for posterior normality. Statist. Sinica 13, 495–506] proposed to use a version of Stein's Identity to write the posterior expectations for functions of a normalized quantity in a form that is more transparent and can be easily analyzed. In the present paper we extend this approach to the multi-parameter cases and compare our conditions with earlier work. Three examples are used to illustrate the application of this method.     

The generalized maximum Tsallis entropy estimators and applications to the Portland cement dataset

Tsallis entropy is a generalized form of entropy and tends to be Shannon entropy when q → 1. Using Tsallis entropy, an alternative estimation methodology (generalized maximum Tsallis entropy) is introduced and used to estimate the parameters in a linear regression model when the basic data are ill-conditioned. We describe the generalized maximum Tsallis entropy and for q = 2 we call that GMET2 estimator. We apply the GMET2 estimator for estimating the linear regression model Y = Xβ + e where the design matrix X is subject to severe multicollinearity. We compared the GMET2, generalized maximum entropy (GME), ordinary least-square (OLS), and inequality restricted least-square (IRLS) estimators on the analyzed dataset on Portland cement.     

More on testing the normality assumptionin the Tobit Model

In a recent volume of this journal, Holden [Testing the normality assumption in the Tobit Model, J. Appl. Stat. 31 (2004) pp. 521–532] presents Monte Carlo evidence comparing several tests for departures from normality in the Tobit Model. This study adds to the work of Holden by considering another test, and several information criteria, for detecting departures from normality in the Tobit Model. The test given here is a modified likelihood ratio statistic based on a partially adaptive estimator of the Censored Regression Model using the approach of Caudill [A partially adaptive estimator for the Censored Regression Model based on a mixture of normal distributions, Working Paper, Department of Economics, Auburn University, 2007]. The information criteria examined include the Akaike’s Information Criterion (AIC), the Consistent AIC (CAIC), the Bayesian information criterion (BIC), and the Akaike’s BIC (ABIC). In terms of fewest ‘rejections’ of a true null, the best performance is exhibited by the CAIC and the BIC, although, like some of the statistics examined by Holden, there are computational difficulties with each.     

Asymptotic normality of kernel type regression estimators for random fields

The asymptotic normality of the Nadaraya–Watson regression estimator is studied for α-mixing$\alpha \text{-}\mathrm{mixing}$ random fields. The infill-increasing setting is considered, that is when the locations of observations become dense in an increasing sequence of domains. This setting fills the gap between continuous and discrete models. In the infill-increasing case the asymptotic normality of the Nadaraya–Watson estimator holds, but with an unusual asymptotic covariance structure. It turns out that this covariance structure is a combination of the covariance structures that we observe in the discrete and in the continuous case.     

Asymptotic properties for the moment estimators in Rayleigh distribution with two parameters

ABSTRACT

We consider the asymptotic properties for the moment estimators in Rayleigh distribution with two parameters. The law of the iterated logarithm for the estimators can be obtained. Moreover, we can give a simple proof of the asymptotic normality which has been obtained by Li and Li (2012) Li, Y.W., Li, M.H. (2012). Moment estimation of the parameters in Rayleigh distribution with two parameters. Commun. Stat.-Theor. Methods 41:2643–2660.[Taylor & Francis Online], [Web of Science ®] , [Google Scholar].     

Goodness-of-fit test based on correcting moments of modified entropy estimator

In this paper, we first propose a new estimator of entropy for continuous random variables. Our estimator is obtained by correcting the coefficients of Vasicek's [A test for normality based on sample entropy, J. R. Statist. Soc. Ser. B 38 (1976), pp. 54–59] entropy estimator. We prove the consistency of our estimator. Monte Carlo studies show that our estimator is better than the entropy estimators proposed by Vasicek, Ebrahimi et al. [Two measures of sample entropy, Stat. Probab. Lett. 20 (1994), pp. 225–234] and Correa [A new estimator of entropy, Commun. Stat. Theory Methods 24 (1995), pp. 2439–2449] in terms of root mean square error. We then derive the non-parametric distribution function corresponding to our proposed entropy estimator as a piece-wise uniform distribution. We also introduce goodness-of-fit tests for testing exponentiality and normality based on the said distribution and compare its performance with their leading competitors.     

Asymptotic normality of the posterior given a statistic

The authors establish the asymptotic normality and determine the limiting variance of the posterior density for a multivariate parameter, given the value of a consistent and asymptotically Gaussian statistic satisfying a uniform local central limit theorem. Their proof is given in the continuous case but generalizes to lattice‐valued random variables. It hinges on a uniform Edgeworth expansion used to control the behaviour of the conditioning statistic. They provide examples and show how their result can help in identifying reference priors.     

Asymptotic normality for chi-bar-square distributions

Chi-bar-square distributions, which are mixtures of chi-square distributions, mixed over their degrees of freedom, often occur when testing hypotheses that involve inequality constraints. Here, necessary and sufficient conditions on the mixing or weighting distribution are found to ensure asymptotic normality of the corresponding chi-bar-square distribution. Essentially, asymptotic normality occurs for the chi-bar-square distribution if either the ratio of the mean to the variance of the mixing distribution goes to infinity, or the weighting distribution itself is asymptotically normal. Other than a combination of these two phenomena, this is also the only way for asymptotic normality to hold. Several examples of pertinent chi-bar-square distributions are shown to be asymptotically normal by the results in this paper.