Using point optimal test of a simple null hypothesis for testing a composite null hypothesis via maximized Monte Carlo approach

King’s Point Optimal (PO) test of a simple null hypothesis is useful in a number of ways, for example it can be used to trace the power envelope against which existing tests can be compared. However, this test cannot always be constructed when testing a composite null hypothesis. It is suggested in the literature that approximate PO (APO) tests can overcome this problem, but they also have some drawbacks. This paper investigates if King’s PO test can be used for testing a composite null in the presence of nuisance parameters via a maximized Monte Carlo (MMC) approach, with encouraging results.     

Comparing the performance of normality tests with ROC analysis and confidence intervals

There are several statistical hypothesis tests available for assessing normality assumptions, which is an a priori requirement for most parametric statistical procedures. The usual method for comparing the performances of normality tests is to use Monte Carlo simulations to obtain point estimates for the corresponding powers. The aim of this work is to improve the assessment of 9 normality hypothesis tests. For that purpose, random samples were drawn from several symmetric and asymmetric nonnormal distributions and Monte Carlo simulations were carried out to compute confidence intervals for the power achieved, for each distribution, by two of the most usual normality tests, Kolmogorov–Smirnov with Lilliefors correction and Shapiro–Wilk. In addition, the specificity was computed for each test, again resorting to Monte Carlo simulations, taking samples from standard normal distributions. The analysis was then additionally extended to the Anderson–Darling, Cramer-Von Mises, Pearson chi-square Shapiro–Francia, Jarque–Bera, D'Agostino and uncorrected Kolmogorov–Smirnov tests by determining confidence intervals for the areas under the receiver operating characteristic curves. Simulations were performed to this end, wherein for each sample from a nonnormal distribution an equal-sized sample was taken from a normal distribution. The Shapiro–Wilk test was seen to have the best global performance overall, though in some circumstances the Shapiro–Francia or the D'Agostino tests offered better results. The differences between the tests were not as clear for smaller sample sizes. Also to be noted, the SW and KS tests performed generally quite poorly in distinguishing between samples drawn from normal distributions and t Student distributions.     

Blending Bayesian and Classical Tools to Define Optimal Sample-Size-Dependent Significance Levels

ABSTRACT

This article argues that researchers do not need to completely abandon the p-value, the best-known significance index, but should instead stop using significance levels that do not depend on sample sizes. A testing procedure is developed using a mixture of frequentist and Bayesian tools, with a significance level that is a function of sample size, obtained from a generalized form of the Neyman–Pearson Lemma that minimizes a linear combination of α, the probability of rejecting a true null hypothesis, and β, the probability of failing to reject a false null, instead of fixing α and minimizing β. The resulting hypothesis tests do not violate the Likelihood Principle and do not require any constraints on the dimensionalities of the sample space and parameter space. The procedure includes an ordering of the entire sample space and uses predictive probability (density) functions, allowing for testing of both simple and compound hypotheses. Accessible examples are presented to highlight specific characteristics of the new tests.     

Likelihood ratio tests for multivariate normality

This paper presents some powerful omnibus tests for multivariate normality based on the likelihood ratio and the characterizations of the multivariate normal distribution. The power of the proposed tests is studied against various alternatives via Monte Carlo simulations. Simulation studies show our tests compare well with other powerful tests including multivariate versions of the Shapiro–Wilk test and the Anderson–Darling test.     

Bayesian (mean) most powerful tests

A fundamental theorem in hypothesis testing is the Neyman‐Pearson (N‐P) lemma, which creates the most powerful test of simple hypotheses. In this article, we establish Bayesian framework of hypothesis testing, and extend the Neyman‐Pearson lemma to create the Bayesian most powerful test of general hypotheses, thus providing optimality theory to determine thresholds of Bayes factors. Unlike conventional Bayes tests, the proposed Bayesian test is able to control the type I error.     

Approximating the risk score for disease diagnosis using MARS

In disease screening and diagnosis, often multiple markers are measured and combined to improve the accuracy of diagnosis. McIntosh and Pepe [Combining several screening tests: optimality of the risk score, Biometrics 58 (2002), pp. 657–664] showed that the risk score, defined as the probability of disease conditional on multiple markers, is the optimal function for classification based on the Neyman–Pearson lemma. They proposed a two-step procedure to approximate the risk score. However, the resulting receiver operating characteristic (ROC) curve is only defined in a subrange (L, h) of false-positive rates in (0,1) and the determination of the lower limit L needs extra prior information. In practice, most diagnostic tests are not perfect, and it is usually rare that a single marker is uniformly better than the other tests. Using simulation, I show that multivariate adaptive regression spline is a useful tool to approximate the risk score when combining multiple markers, especially when ROC curves from multiple tests cross. The resulting ROC is defined in the whole range of (0,1) and is easy to implement and has intuitive interpretation. The sample code of the application is shown in the appendix.     

Bridging the gap: A likelihood function approach for the analysis of ranking data

ABSTRACT

In the parametric setting, the notion of a likelihood function forms the basis for the development of tests of hypotheses and estimation of parameters. Tests in connection with the analysis of variance stem entirely from considerations of the likelihood function. On the other hand, non parametric procedures have generally been derived without any formal mechanism and are often the result of clever intuition. In the present article, we propose a more formal approach for deriving tests involving the use of ranks. Specifically, we define a likelihood function motivated by characteristics of the ranks of the data and demonstrate that this leads to well-known tests of hypotheses. We also point to various areas of further exploration such as how co-variates may be incorporated.     

A tool for systematically comparing the power of tests for normality

In this article, we describe a new approach to compare the power of different tests for normality. This approach provides the researcher with a practical tool for evaluating which test at their disposal is the most appropriate for their sampling problem. Using the Johnson systems of distribution, we estimate the power of a test for normality for any mean, variance, skewness, and kurtosis. Using this characterization and an innovative graphical representation, we validate our method by comparing three well-known tests for normality: the Pearson χ² test, the Kolmogorov–Smirnov test, and the D'Agostino–Pearson K ² test. We obtain such comparison for a broad range of skewness, kurtosis, and sample sizes. We demonstrate that the D'Agostino–Pearson test gives greater power than the others against most of the alternative distributions and at most sample sizes. We also find that the Pearson χ² test gives greater power than Kolmogorov–Smirnov against most of the alternative distributions for sample sizes between 18 and 330.     

A Comparative Study of Some Modified Chi-Squared Tests

Some recent results in the theory and applications of modified chi-squared goodness-of-fit tests are briefly discussed. It seems that for the first time power of modified chi-squared type tests for the logistic and three-parameter Weibull distributions based on moment type estimators is studied. Power of different modified tests against some alternatives for equiprobable fixed or random grouping intervals, and for Neyman–Pearson classes is investigated. It is shown that power of test statistic essentially depends on the quantity of Fisher's sample information this statistic uses. Some recommendations on implementing modified chi-squared type tests are given.     

Ratio tests under limiting normality

We propose a class of ratio tests that is applicable whenever a cumulation (of transformed) data is asymptotically normal upon appropriate normalization. The Karhunen–Loève theorem is employed to compute weighted averages. The test statistics are ratios of quadratic forms of these averages and hence scale-invariant, also called self-normalizing: The scaling parameter cancels asymptotically. Limiting distributions are obtained. Critical values and asymptotic local power functions can be calculated by standard numerical means. The ratio tests are directed against local alternatives and turn out to be almost as powerful as optimal competitors, without being plagued by nuisance parameters at the same time. Also in finite samples they perform well relative to self-normalizing competitors.     

A re-appraisal of tests for normality

Tests for normality can be divided into two groups - those based upon a function of the empirical distribution function and those based upon a function of the original observations. The latter group of statistics test spherical symmetry and not necessarily normality. If the distribution is completely specified then the first group can be used to test for ‘spherical’ normality. However, if the distribution is incompletely specified and F‘‘x_i - x’/s’ is used these test statistics also test sphericity rather than normality. A Monte Carlo study was conducted for the completely specified case, to investigate the sensitivity of the distance tests to departures from normality when the alternative distributions are non-normal spherically symmetric laws. A “new” test statistic is proposed for testing a completely specified normal distribution     

Re-examining two tests for bivariate normality

Many goodness of fit tests for bivariate normality are not rigorous procedures because the distributions of the proposed statistics are unknown or too difficult to manipulate. Two familiar examples are the ring test and the line test. In both tests the statistic utilized generally is approximated by a chi-square distribution rather than compared to its known beta distribution. These two procedures are re-examined and re-evaluated in this paper. It is shown that the chi-square approximation can be too conservative and can lead to unnecessary

rejection of normality.     

Comparisons of various types of normality tests

Normality tests can be classified into tests based on chi-squared, moments, empirical distribution, spacings, regression and correlation and other special tests. This paper studies and compares the power of eight selected normality tests: the Shapiro–Wilk test, the Kolmogorov–Smirnov test, the Lilliefors test, the Cramer–von Mises test, the Anderson–Darling test, the D'Agostino–Pearson test, the Jarque–Bera test and chi-squared test. Power comparisons of these eight tests were obtained via the Monte Carlo simulation of sample data generated from alternative distributions that follow symmetric short-tailed, symmetric long-tailed and asymmetric distributions. Our simulation results show that for symmetric short-tailed distributions, D'Agostino and Shapiro–Wilk tests have better power. For symmetric long-tailed distributions, the power of Jarque–Bera and D'Agostino tests is quite comparable with the Shapiro–Wilk test. As for asymmetric distributions, the Shapiro–Wilk test is the most powerful test followed by the Anderson–Darling test.     

Empirical behavior of some tests for normality

The behavior of a range of tests assessing the normality of a sequence of independent and identically distributed random variables is investigated.An examination of the empirical significance level of the tests is undertaken for different sample sizes. The empirical power associated with these tests is also calculated under some alternative distributions.     

A Decomposition of Pearson–Fisher and Dzhaparidze–Nikulin Statistics and Some Ideas for a More Powerful Test Construction

An explicit decomposition on asymptotically independent distributed as chi-squared with one degree of freedom components of the Pearson–Fisher and Dzhaparidze–Nikulin tests is presented. The decomposition is formally the same for both tests and is valid for any partitioning of a sample space. Vector-valued tests, components of which can be not only different scalar tests based on the same sample, but also scalar tests based on components or groups of components of the same statistic are considered. Numerical examples illustrating the idea are presented.     

An Empirical Analysis of Some Nonparametric Goodness-of-Fit Tests for Censored Data

In this article, we consider some nonparametric goodness-of-fit tests for right censored samples, viz., the modified Kolmogorov, Cramer–von Mises–Smirnov, Anderson–Darling, and Nikulin–Rao–Robson χ² tests. We also consider an approach based on a transformation of the original censored sample to a complete one and the subsequent application of classical goodness-of-fit tests to the pseudo-complete sample. We then compare these tests in terms of power in the case of Type II censored data along with the power of the Neyman–Pearson test, and draw some conclusions. Finally, we present an illustrative example.     

New invariant and consistent chi-squared type goodness-of-fit tests for multivariate normality and a related comparative simulation study

ABSTRACT

New invariant and consistent goodness-of-fit tests for multivariate normality are introduced. Tests are based on the Karhunen–Loève transformation of a multidimensional sample from a population. A comparison of simulated powers of tests and other well-known tests with respect to some alternatives is given. The simulation study demonstrates that power of the proposed McCull test almost does not depend on the number of grouping cells. The test shows an advantage over other chi-squared type tests. However, averaged over all of the simulated conditions examined in this article, the Anderson–Darling type and the Cramer–von Mises type tests seem to be the best.     

Asymptotic power of tests of normality under local alternatives

Seven tests of univariate normality are studied in view of their asymptotic power under local alternatives. The procedures under consideration are either based on the empirical skewness and/or kurtosis, including the popular Jarque-Bera statistic, as well as Cramér-von Mises, Anderson-Darling and Kolmogorov-Smirnov functionals of an empirical process with estimated parameters. The large-sample behavior of these test statistics under contiguous sequences is obtained; this allows for the computation of their associated local power curves and of their asymptotic relative efficiency in the light of a measure proposed by Berg and Quessy (2009). Comparisons are made under four classes of local alternatives, including those used by Thadewald and Büning (2007) in a recent Monte-Carlo power study. These theoretical results are related to empirical ones and many recommendations are formulated.     

The widespread misinterpretation of p-values as error probabilities

The anonymous mixing of Fisherian (p-values) and Neyman–Pearsonian (α levels) ideas about testing, distilled in the customary but misleading p < α criterion of statistical significance, has led researchers in the social and management sciences (and elsewhere) to commonly misinterpret the p-value as a ‘data-adjusted’ Type I error rate. Evidence substantiating this claim is provided from a number of fronts, including comments by statisticians, articles judging the value of significance testing, textbooks, surveys of scholars, and the statistical reporting behaviours of applied researchers. That many investigators do not know the difference between p’s and α’s indicates much bewilderment over what those most ardently sought research outcomes—statistically significant results—means. Statisticians can play a leading role in clearing this confusion. A good starting point would be to abolish the p < α criterion of statistical significance.     

Graphical comparison of normality tests for unimodal distribution data

A methodology is proposed to compare the power of normality tests with a wide variety of alternative unimodal distributions. It is based on the representation of a distribution mosaic in which kurtosis varies vertically and skewness horizontally. The mosaic includes distributions such as exponential, Laplace or uniform, with normal occupying the centre. Simulation is used to determine the probability of a sample from each distribution in the mosaic being accepted as normal. We demonstrate our proposal by applying it to the analysis and comparison of some of the most well-known tests.