Using principal components to test normality of high-dimensional data

Many multivariate statistical procedures are based on the assumption of normality and different approaches have been proposed for testing this assumption. The vast majority of these tests, however, are exclusively designed for cases when the sample size n is larger than the dimension of the variable p, and the null distributions of their test statistics are usually derived under the asymptotic case when p is fixed and n increases. In this article, a test that utilizes principal components to test for nonnormality is proposed for cases when p/n → c. The power and size of the test are examined through Monte Carlo simulations, and it is argued that the test remains well behaved and consistent against most nonnormal distributions under this type of asymptotics.     

Simulation-assisted saddlepoint approximation

A general saddlepoint/Monte Carlo method to approximate (conditional) multivariate probabilities is presented. This method requires a tractable joint moment generating function (m.g.f.), but does not require a tractable distribution or density. The method is easy to program and has a third-order accuracy with respect to increasing sample size in contrast to standard asymptotic approximations which are typically only accurate to the first order.

The method is most easily described in the context of a continuous regular exponential family. Here, inferences can be formulated as probabilities with respect to the joint density of the sufficient statistics or the conditional density of some sufficient statistics given the others. Analytical expressions for these densities are not generally available, and it is often not possible to simulate exactly from the conditional distributions to obtain a direct Monte Carlo approximation of the required integral. A solution to the first of these problems is to replace the intractable density by a highly accurate saddlepoint approximation. The second problem can be addressed via importance sampling, that is, an indirect Monte Carlo approximation involving simulation from a crude approximation to the true density. Asymptotic normality of the sufficient statistics suggests an obvious candidate for an importance distribution.

The more general problem considers the computation of a joint probability for a subvector of random T, given its complementary subvector, when its distribution is intractable, but its joint m.g.f. is computable. For such settings, the distribution may be tilted, maintaining T as the sufficient statistic. Within this tilted family, the computation of such multivariate probabilities proceeds as described for the exponential family setting.     

Nonparametric estimation of regression models with mixed discrete and continuous covariates by the K-nn method

In this article we consider the problem of estimating a nonparametric conditional mean function with mixed discrete and continuous covariates by the nonparametric k-nearest-neighbor (k-nn) method. We derive the asymptotic normality result of the proposed estimator and use Monte Carlo simulations to demonstrate its finite sample performance. We also provide an illustrative empirical example of our method.     

A new estimator of entropy and its application in testing normality

In this paper, we introduce a new estimator of entropy of a continuous random variable. We compare the proposed estimator with the existing estimators, namely, Vasicek [A test for normality based on sample entropy, J. Roy. Statist. Soc. Ser. B 38 (1976), pp. 54–59], van Es [Estimating functionals related to a density by class of statistics based on spacings, Scand. J. Statist. 19 (1992), pp. 61–72], Correa [A new estimator of entropy, Commun. Statist. Theory and Methods 24 (1995), pp. 2439–2449] and Wieczorkowski-Grzegorewski [Entropy estimators improvements and comparisons, Commun. Statist. Simulation and Computation 28 (1999), pp. 541–567]. We next introduce a new test for normality. By simulation, the powers of the proposed test under various alternatives are compared with normality tests proposed by Vasicek (1976) and Esteban et al. [Monte Carlo comparison of four normality tests using different entropy estimates, Commun. Statist.–Simulation and Computation 30(4) (2001), pp. 761–785].     

Inferences on the Coefficients of Variation in a Multivariate Normal Population

In this article, the problem of testing the equality of coefficients of variation in a multivariate normal population is considered, and an asymptotic approach and a generalized p-value approach based on the concepts of generalized test variable are proposed. Monte Carlo simulation studies show that the proposed generalized p-value test has good empirical sizes, and it is better than the asymptotic approach. In addition, the problem of hypothesis testing and confidence interval for the common coefficient variation of a multivariate normal population are considered, and a generalized p-value and a generalized confidence interval are proposed. Using Monte Carlo simulation, we find that the coverage probabilities and expected lengths of this generalized confidence interval are satisfactory, and the empirical sizes of the generalized p-value are close to nominal level. We illustrate our approaches using a real data.     

A simple bera-jarque normality test for nonparametric residuals

This paper demonstrates that the Bera-Jarque normality test using residuals from nonparametric series estimates has the usual X ²(2) asymptotic distribution. Monte Carlo results shows that the test has good properties for reasonably sized samples.     

The Multivariate Randomized Complete Block Design: A Novel Permutation Solution in Case of Ordered Categorical Variables

In this article, multivariate extensions of the combination-based test statistics for the comparison of several treatments in the multivariate Randomized Complete Block designs are introduced in case of categorical response variables. Several tests for the multivariate Randomized Complete Block designs, including MANOVA procedure, are compared with the method proposed via a Monte Carlo simulation study. The method has also been applied to a real case study in the field of sensorial testing studies. Results suggest that in each experimental situation where normality of the supposed underlying continuous model is hard to justify and especially when errors have heavy-tailed distributions, the proposed nonparametric procedure can be considered as a valid solution.     

Bounded influence nonlinear signed‐rank regression

In this paper we consider weighted generalized‐signed‐rank estimators of nonlinear regression coefficients. The generalization allows us to include popular estimators such as the least squares and least absolute deviations estimators but by itself does not give bounded influence estimators. Adding weights results in estimators with bounded influence function. We establish conditions needed for the consistency and asymptotic normality of the proposed estimator and discuss how weight functions can be chosen to achieve bounded influence function of the estimator. Real life examples and Monte Carlo simulation experiments demonstrate the robustness and efficiency of the proposed estimator. An example shows that the weighted signed‐rank estimator can be useful to detect outliers in nonlinear regression. The Canadian Journal of Statistics 40: 172–189; 2012 © 2012 Statistical Society of Canada     

A Monte Carlo comparison of Jarque–Bera type tests and Henze–Zirkler test of multivariate normality

In the paper, tests for multivariate normality (MVN) of Jarque-Bera type, based on skewness and kurtosis, have been considered. Tests proposed by Mardia and Srivastava, and the combined tests based on skewness and kurtosis defined by Jarque and Bera have been taken into account. In the Monte Carlo simulations, for each combination of p = 2, 3, 4, 5 number of traits and n = 10(5)50(10)100 sample sizes 10,000 runs have been done to calculate empirical Type I errors of tests under consideration, and empirical power against different alternative distributions. Simulation results have been compared to the Henze–Zirkler’s test. It should be stressed that no test yet proposed is uniformly better than all the others in every combination of conditions examined.     

A test of normality using nonparametrlic residuals

In this paper, we develop a test of the normality assumption of the errors using the residuals from a nonparametric kernel regression. Contrary to the existing tests based on the residuals from a parametric regression, our test is thus robust to misspecification of the regression function. The test statistic proposed here is a Bera-Jarque type test of skewness and kurtosis. We show that the test statistic has the usual x ²(2) limit distribution under the null hypothesis. In contrast to the results of Rilstone (1992), we provide a set of primitive assumptions that allow weakly dependent observations and data dependent bandwidth parameters. We also establish consistency property of the test. Monte Carlo experiments show that our test has reasonably good size and power performance in small samples and perfornu better than some of the alternative tests in various situations.     

Likelihood ratio tests for multivariate normality

This paper presents some powerful omnibus tests for multivariate normality based on the likelihood ratio and the characterizations of the multivariate normal distribution. The power of the proposed tests is studied against various alternatives via Monte Carlo simulations. Simulation studies show our tests compare well with other powerful tests including multivariate versions of the Shapiro–Wilk test and the Anderson–Darling test.     

On population-based simulation for static inference

In this paper we present a review of population-based simulation for static inference problems. Such methods can be described as generating a collection of random variables {X _n } _n=1,…,N in parallel in order to simulate from some target density π (or potentially sequence of target densities). Population-based simulation is important as many challenging sampling problems in applied statistics cannot be dealt with successfully by conventional Markov chain Monte Carlo (MCMC) methods. We summarize population-based MCMC (Geyer, Computing Science and Statistics: The 23rd Symposium on the Interface, pp. 156–163, 1991; Liang and Wong, J. Am. Stat. Assoc. 96, 653–666, 2001) and sequential Monte Carlo samplers (SMC) (Del Moral, Doucet and Jasra, J. Roy. Stat. Soc. Ser. B 68, 411–436, 2006a), providing a comparison of the approaches. We give numerical examples from Bayesian mixture modelling (Richardson and Green, J. Roy. Stat. Soc. Ser. B 59, 731–792, 1997).     

Maximum Likelihood Estimation and Inference in Multivariate Conditionally Heteroscedastic Dynamic Regression Models With Student t Innovations

We provide numerically reliable analytical expressions for the score, Hessian, and information matrix of conditionally heteroscedastic dynamic regression models when the conditional distribution is multivariatet. We also derive one-sided and two-sided Lagrange multiplier tests for multivariate normality versus multivariate t based on the first two moments of the squared norm of the standardized innovations evaluated at the Gaussian pseudo-maximum likelihood estimators of the conditional mean and variance parameters. Finally, we illustrate our techniques through both Monte Carlo simulations and an empirical application to 26 U.K. sectorial stock returns that confirms that their conditional distribution has fat tails.     

Goodness-of-fit test based on correcting moments of modified entropy estimator

In this paper, we first propose a new estimator of entropy for continuous random variables. Our estimator is obtained by correcting the coefficients of Vasicek's [A test for normality based on sample entropy, J. R. Statist. Soc. Ser. B 38 (1976), pp. 54–59] entropy estimator. We prove the consistency of our estimator. Monte Carlo studies show that our estimator is better than the entropy estimators proposed by Vasicek, Ebrahimi et al. [Two measures of sample entropy, Stat. Probab. Lett. 20 (1994), pp. 225–234] and Correa [A new estimator of entropy, Commun. Stat. Theory Methods 24 (1995), pp. 2439–2449] in terms of root mean square error. We then derive the non-parametric distribution function corresponding to our proposed entropy estimator as a piece-wise uniform distribution. We also introduce goodness-of-fit tests for testing exponentiality and normality based on the said distribution and compare its performance with their leading competitors.     

A Generalization of Shapiro–Wilk's Test for Multivariate Normality

A goodness-of-fit test for multivariate normality is proposed which is based on Shapiro–Wilk's statistic for univariate normality and on an empirical standardization of the observations. The critical values can be approximated by using a transformation of the univariate standard normal distribution. A Monte Carlo study reveals that this test has a better power performance than some of the best known tests for multinormality against a wide range of alternatives.     

A computational framework for empirical Bayes inference

In empirical Bayes inference one is typically interested in sampling from the posterior distribution of a parameter with a hyper-parameter set to its maximum likelihood estimate. This is often problematic particularly when the likelihood function of the hyper-parameter is not available in closed form and the posterior distribution is intractable. Previous works have dealt with this problem using a multi-step approach based on the EM algorithm and Markov Chain Monte Carlo (MCMC). We propose a framework based on recent developments in adaptive MCMC, where this problem is addressed more efficiently using a single Monte Carlo run. We discuss the convergence of the algorithm and its connection with the EM algorithm. We apply our algorithm to the Bayesian Lasso of Park and Casella (J. Am. Stat. Assoc. 103:681–686, 2008) and on the empirical Bayes variable selection of George and Foster (J. Am. Stat. Assoc. 87:731–747, 2000).     

Bias correction for the regression-based LM fractional integration test

This paper examines the finite-sample behavior of the Lagrange Multiplier (LM) test for fractional integration proposed by Breitung and Hassler (J. Econom. 110:167–185, 2002). We find by extensive Monte Carlo simulations that size distortions can be quite large in small samples. These are caused by a finite-sample bias towards the alternative. Analytic expressions for this bias are derived, based on which the test can easily be corrected.     

A density based empirical likelihood approach for testing bivariate normality

Sample entropy based tests, methods of sieves and Grenander estimation type procedures are known to be very efficient tools for assessing normality of underlying data distributions, in one-dimensional nonparametric settings. Recently, it has been shown that the density based empirical likelihood (EL) concept extends and standardizes these methods, presenting a powerful approach for approximating optimal parametric likelihood ratio test statistics, in a distribution-free manner. In this paper, we discuss difficulties related to constructing density based EL ratio techniques for testing bivariate normality and propose a solution regarding this problem. Toward this end, a novel bivariate sample entropy expression is derived and shown to satisfy the known concept related to bivariate histogram density estimations. Monte Carlo results show that the new density based EL ratio tests for bivariate normality behave very well for finite sample sizes. To exemplify the excellent applicability of the proposed approach, we demonstrate a real data example.     

Parameter and reliability estimation for an exponentiated half-logistic distribution under progressive type II censoring

In this article, we deal with a two-parameter exponentiated half-logistic distribution. We consider the estimation of unknown parameters, the associated reliability function and the hazard rate function under progressive Type II censoring. Maximum likelihood estimates (M LEs) are proposed for unknown quantities. Bayes estimates are derived with respect to squared error, linex and entropy loss functions. Approximate explicit expressions for all Bayes estimates are obtained using the Lindley method. We also use importance sampling scheme to compute the Bayes estimates. Markov Chain Monte Carlo samples are further used to produce credible intervals for the unknown parameters. Asymptotic confidence intervals are constructed using the normality property of the MLEs. For comparison purposes, bootstrap-p and bootstrap-t confidence intervals are also constructed. A comprehensive numerical study is performed to compare the proposed estimates. Finally, a real-life data set is analysed to illustrate the proposed methods of estimation.