A Cautionary Note on the Analysis of Extreme Data with Cox Regression

This article analyzes a small censored data set to demonstrate the potential dangers of using statistical computing packages without understanding the details of statistical methods. The data, consisting of censored response times with heavy ties in one time point, were analyzed with a Cox regression model utilizing SAS PHREG and BMDP2L procedures. The p values, reported from both SAS PHREG and BMDP2L procedures, for testing the equality of two treatments vary considerably. This article illustrates that (1) the Breslow likelihood used in both BMDP2L and SAS PHREG procedures is too conservative and can have a critical effect on an extreme data set, (2) Wald's test in the SAS PHREG procedure may yield absurd results from most likelihood models, and (3) BMDP2L needs to include more than just the Breslow likelihood in future development.     

Overview of Friedman’s Test and Post-hoc Analysis

When the null hypothesis of Friedman’s test is rejected, there is a wide variety of multiple comparisons that can be used to determine which treatments differ from each other. We will discuss the contexts where different multiple comparisons should be applied, when the population follows some discrete distributions commonly used to model count data in biological and ecological fields. Our simulation study shows that sign test is very conservative. Fisher’s LSD and Tukey’s HSD tests computed with ranks are the most liberal. Theoretical considerations are illustrated with data of the Azores Buzzard (Buteo buteo rothschildi) population from Azores, Portugal.     

Testing for monotonic trend in time series based on resampling methods

In this paper, we propose several tests for monotonic trend based on the Brillinger's test statistic (1989, Biometrika, 76, 23–30). When there are highly correlated residuals or short record lengths, Brillinger's test procedure tends to have significance level much higher than the nominal level. It is found that this could be related to the discrepancy between the empirical distribution of the test statistic and the asymptotic normal distribution. Hence, in this paper, we propose three bootstrap-based procedures based on the Brillinger's test statistic to test for monotonic trend. The performance of the proposed test procedures is evaluated through an extensive Monte Carlo simulation study, and is compared to other trend test procedures in the literature. It is shown that the proposed bootstrap-based Brillinger test procedures can well control the significance levels and provide satisfactory power performance in testing the monotonic trend under different scenarios.     

Bootstrap-assisted tests of symmetry for dependent data*

The paper considers the problem of testing for symmetry (about an unknown centre) of the marginal distribution of a strictly stationary and weakly dependent stochastic process. The possibility of using the autoregressive sieve bootstrap and stationary bootstrap procedures to obtain critical values and P-values for symmetry tests is explored. Bootstrap-assisted tests for symmetry are straightforward to implement and require no prior estimation of asymptotic variances. The small-sample properties of a wide variety of tests are investigated using Monte Carlo experiments. A bootstrap-assisted version of the triples test is found to have the best overall performance.     

R 2 Measures Based on Wald and Likelihood Ratio Joint Significance Tests

Two methods are suggested for generating R ² measures for a wide class of models. These measures are linked to the R ² of the standard linear regression model through Wald and likelihood ratio statistics for testing the joint significance of the explanatory variables. Some currently used R ²'s are shown to be special cases of these methods.     

Estimation of the percentage of a normal distribution lying outside a specified interval

Point and interval estimation for the percentage of a normal population lying outside a specified Interval Is investigated. Resnikoff's (1955) confidence Interval procedure is shown to perform well when the percentage outside is small, and to be conservative otherwise.When the percentage to be estimated is large, an alternative interval is recommended.Monte Carlo studies show the sensitivity of these estimation procedures to violations of the normality assumption.Attempts to transform the sample to approximate normality prior to estimation were moderately successful in achieving robust estimation procedures.     

Testing for Zero Intercept in Multivariate Normal Regression Using the Univariate Multiple-Regression F Test

A likelihood ratio test is derived for comparing the performance potential of a subset of a population of financial assets to the performance potential of the entire population. The test is shown to be equivalent to a test for zero intercept in a multivariate normal regression model. Rao's F approximation to Wilks' Lamda is shown to be equivalent in this case to the conventional F test used to test the significance of a subset of regressors in a univariate multiple-regression model. The test is illustrated using a sample of returns from ten stocks from the New York Stock Exchange.     

Closed testing procedures for all pairwise comparisons in a randomized block design

We consider multiple comparison test procedures among treatment effects in a randomized block design. We propose closed testing procedures based on maximum values of some two-sample t test statistics and based on F test statistics. It is shown that the proposed procedures are more powerful than single-step procedures and the REGW (Ryan/Einot–Gabriel/Welsch)-type tests. Next, we consider the randomized block design under simple ordered restrictions of treatment effects. We propose closed testing procedures based on maximum values of two-sample one-sided t test statistics and based on Batholomew’s statistics for all pairwise comparisons of treatment effects. Although single-step multiple comparison procedures are utilized in general, the power of these procedures is low for a large number of groups. The closed testing procedures stated in the present article are more powerful than the single-step procedures. Simulation studies are performed under the null hypothesis and some alternative hypotheses. In this studies, the proposed procedures show a good performance.     

On optimal tests for general interval-hypotheses

Optimal statistical tests, using the normality assumptions for general interval hypotheses including equivalence testing and testing for nonzero difference (or for non-unit) are presented. These tests are based on the decision theory for Polya Type distributions and are compared with usual confidence tests and with ’two one-sided tests’- procedures. A formal relationship between some optimal tests and the Anderson and Hauck procedure as well as a procedure recommended by Patel and Gupta is given. A new procedure for a generalisation of Student's test as well as for equivalence testing for thet-statistics is shown.     

Notes on Test Homogeneity of the Odds Ratio in Matched Pairs Under Stratified Sampling: A Monte Carlo Evaluation

Using Monte Carlo simulation, we compare the performance of five asymptotic test procedures and a randomized permutation test procedure for testing the homogeneity of odds ratio under the stratified matched-pair design. We note that the weighted-least-square test procedure is liberal, while Pearson's goodness-of-fit (PGF) test procedure with the continuity correction is conservative. We note that PGF without the continuity correction, the conditional likelihood ratio test procedure, and the randomized permutation test procedure can generally perform well with respect to Type I error. We use the data taken from a case–control study regarding the endometrial cancer incidence published elsewhere to illustrate the use of these test procedures.     

Normality testing: two new tests using L-moments

Establishing that there is no compelling evidence that some population is not normally distributed is fundamental to many statistical inferences, and numerous approaches to testing the null hypothesis of normality have been proposed. Fundamentally, the power of a test depends on which specific deviation from normality may be presented in a distribution. Knowledge of the potential nature of deviation from normality should reasonably guide the researcher's selection of testing for non-normality. In most settings, little is known aside from the data available for analysis, so that selection of a test based on general applicability is typically necessary. This research proposes and reports the power of two new tests of normality. One of the new tests is a version of the R-test that uses the L-moments, respectively, L-skewness and L-kurtosis and the other test is based on normalizing transformations of L-skewness and L-kurtosis. Both tests have high power relative to alternatives. The test based on normalized transformations, in particular, shows consistently high power and outperforms other normality tests against a variety of distributions.     

Tests for Differences in Dispersion Based on Quantiles

It is commonly known that the validity of the F test for testing differences in variability is highly sensitive to the assumption that the population distributions are normal. Hence there is a need for nonparametric tests that do not rely on the assumption of normal population distributions. Several nonparametric tests for testing differences in dispersion have been developed in the past 40 years. These include Mood's test, Klotz's test, and the Siegel-Tukey test. Unfortunately, many of these tests do not have a natural or easily calculated measure of dispersion associated with them. This article introduces a test for differences in dispersion based on quantiles that is easy to compute and readily comprehended by the casual user of statistics.     

Comparison of nonparametric analysis of variance methods: A vote for van der Waerden

ABSTRACT

For two-way layouts in a between-subjects analysis of variance design, the parametric F-test is compared with seven nonparametric methods: rank transform (RT), inverse normal transform (INT), aligned rank transform (ART), a combination of ART and INT, Puri & Sen's L statistic, Van der Waerden, and Akritas and Brunners ANOVA-type statistics (ATS). The type I error rates and the power are computed for 16 normal and nonnormal distributions, with and without homogeneity of variances, for balanced and unbalanced designs as well as for several models including the null and the full model. The aim of this study is to identify a method that is applicable without too much testing for all the attributes of the plot. The Van der Waerden test shows the overall best performance though there are some situations in which it is disappointing. The Puri & Sen's and the ATS tests show generally very low power. These two and the other methods cannot keep the type I error rate under control in too many situations. Especially in the case of lognormal distributions, the use of any of the rank-based procedures can be dangerous for cell sizes above 10. As already shown by many other authors, nonnormal distributions do not violate the parametric F-test, but unequal variances do, and heterogeneity of variances leads to an inflated error rate more or less also for the nonparametric methods. Finally, it should be noted that some procedures show rising error rates with increasing cell sizes, the ART, especially for discrete variables, and the RT, Puri & Sen, and the ATS in the cases of heteroscedasticity.     

A more powerful unconditional exact test of homogeneity for 2 × c contingency table analysis

The classical unconditional exact p-value test can be used to compare two multinomial distributions with small samples. This general hypothesis requires parameter estimation under the null which makes the test severely conservative. Similar property has been observed for Fisher's exact test with Barnard and Boschloo providing distinct adjustments that produce more powerful testing approaches. In this study, we develop a novel adjustment for the conservativeness of the unconditional multinomial exact p-value test that produces nominal type I error rate and increased power in comparison to all alternative approaches. We used a large simulation study to empirically estimate the 5th percentiles of the distributions of the p-values of the exact test over a range of scenarios and implemented a regression model to predict the values for two-sample multinomial settings. Our results show that the new test is uniformly more powerful than Fisher's, Barnard's, and Boschloo's tests with gains in power as large as several hundred percent in certain scenarios. Lastly, we provide a real-life data example where the unadjusted unconditional exact test wrongly fails to reject the null hypothesis and the corrected unconditional exact test rejects the null appropriately.     

Testing multivariate uniformity: The distance‐to‐boundary method

Given a random sample taken on a compact domain S ? ^d, the authors propose a new method for testing the hypothesis of uniformity of the underlying distribution. The test statistic is based on the distance of every observation to the boundary of S. The proposed test has a number of interesting properties. In particular, it is feasible and particularly suitable for high dimensional data; it is distribution free for a wide range of choices of 5; it can be adapted to the case that the support of S is unknown; and it also allows for one‐sided versions. Moreover, the results suggest that, in some cases, this procedure does not suffer from the well‐known curse of dimensionality. The authors study the properties of this test from both a theoretical and practical point of view. In particular, an extensive Monte Carlo simulation study allows them to compare their methods with some alternative procedures. They conclude that the proposed test provides quite a satisfactory balance between power, computational simplicity, and adaptability to different dimensions and supports.     

An empirical power study of multi-sample tests for exponentiality

Results are given of an empirical power study of three statistical procedures for testing for exponentiality of several independent samples. The test procedures are the Tiku (1974) test, a multi-sample Durbin (1975) test, and a multi-sample Shapiro–Wilk (1972) test. The alternative distributions considered in the study were selected from the gamma, Weibull, Lomax, lognormal, inverse Gaussian, and Burr families of positively skewed distributions. The general behavior of the conditional mean exceedance function is used to classify each alternative distribution. It is shown that Tiku's test generally exhibits overall greater power than either of the other two test procedures. For certain alternative distributions, Shapiro–Wilk's test is superior when the sample sizes are small.     

Application of boundary calculation methodologies to group sequential testing in general parametric models

For clinical trials with interim analyses, there have been methodologies and software to calculate boundaries for comparing binomial, normal, and survival data from two treatment groups. Jermison & Turnbull (1991) extended Pocock (1977) and O' Brien- Fleming (1979) boundaries to t-tests, x²-tests and F-tests for comparing normal data from several treatment groups. This paper demonstrates that the above boundaries can be applied to a wide variety of test statistics based on general parametric settings. We show that asymptotically the x² boundaries as well as the corresponding nominal significance levels calculated by Jennison & Turnbull can be applied to interim analyses based on the score test, the Wald test, and the likelihood ratio test for general parametric models. Based on the results of this paper, currently available software in group sequential testing can be used to calculate. the nominal significance levels (or boundaries) for group sequential testing based on logistic regression, A NOVA, and other parametric methods.     

TESTING THE SIGNIFICANCE OF GENOTYPE-ENVIRONMENT INTERACTION

The AMMI (additive main effects-multiplicative interaction) model is often used to investigate interactions in two-way tables, in particular for genotype-environment interactions. Both Gollob (1968) and Mandel (1969, 1971) proposed methods for testing the significance of such interactions. These methods are compared using simulated data. Our results support Mandel's conclusions, but his method is conservative and we recommend a test proposed by Johnson & Graybill (1972).     

Multiple Comparison Procedures: The Practical Solution

A practicing statistician looks at the multiple comparison controversy and related issues through the eyes of the users. The concept of consistency is introduced and discussed in relation to five of the more common multiple comparison procedures. All of the procedures are found to be inconsistent except the simplest procedure, the unrestricted least significant difference (LSD) procedure (or multiple t test). For this and other reasons the unrestricted LSD procedure is recommended for general use, with the proviso that it should be viewed as a hypothesis generator rather than as a method for simultaneous hypothesis generation and testing. The implications for Scheffé's test for general contrasts are also discussed, and a new recommendation is made.     

A Comparison of Significance Testing Procedures for the Intraclass Correlation from Family Data

Different procedures for testing problems concerning intraclass correlation from familial data are considered in the case of varying number of siblings per family. Under the assumption of multivariate normality, the hypotheses that the intraclass correlation is equal to a specified value are tested. To assess the performance of the tests, Monte Carlo simulations are designed to compare their powers. The Neyman's (1959) C(α) test and the test based on the modified ANOVA F statistic are shown to be consistently more powerful than other procedures.