Robustness of the t and U tests under combined assumption violations

SUMMARY When the assumptions of parametric statistical tests for the difference between two means are violated, it is commonly advised that non-parametric tests are a more robust substitute. The history of the investigation of this issue is summarized. The robustness of the t -test was evaluated, by repeated computer testing for differences between samples from two populations of equal means but non-normal distributions and with different variances and sample sizes. Two common alternatives to t -Welch's approximate t and the Mann-Whitney U -test-were evaluated in the same way. The t -test is sufficiently robust for use in all likely cases, except when skew is severe or when population variances and sample sizes both differ. The Welch test satisfactorily addressed the latter problem, but was itself sensitive to departures from normality. Contrary to its popular reputation, the U -test showed a dramatic 'lack of robustness' in many cases-largely because it is sensitive to population differences other than between means, so it is not properly a 'non-parametric analogue' of the t -test, as it is too often described.     

A Nonparametric Test for the Parallelism of Two First-Order Autoregressive Processes

A nonparametric testing procedure for the parallelism of two first-order autoregressive processes is presented. This paper discuss the Mann–Whitney statistic, its natural competitor two-sample t -test, and the bootstrap method. It studies the asymptotic efficacies of the studentized Mann–Whitney statistic and the t -test statistic with their relative efficiency. Simulation results for comparing the powers of these test statistics are also presented.     

On wasps and club dinners

Francis Ysidro Edgeworth could be termed a Victorian gentleman polymath: born on the family's estate in Edgeworthstown, Ireland, and a trained classicist and barrister, he was capable of both working as a Lecturer in Greek and deriving in 1883 (using what we now call a Bayesian argument) "what may the earliest appearance in any form of the Student's t distribution"¹. His unusual second name arose from his Catalan refugee mother, whom his father is reputed to have met on the steps of the British Museum.     

To be or not to be valid in testing the significance of the slope in simple quantitative linear models with autocorrelated errors

In this article, the validity of procedures for testing the significance of the slope in quantitative linear models with one explanatory variable and first-order autoregressive [AR(1)] errors is analyzed in a Monte Carlo study conducted in the time domain. Two cases are considered for the regressor: fixed and trended versus random and AR(1). In addition to the classical t -test using the Ordinary Least Squares (OLS) estimator of the slope and its standard error, we consider seven t -tests with n-2\,\hbox{df} built on the Generalized Least Squares (GLS) estimator or an estimated GLS estimator, three variants of the classical t -test with different variances of the OLS estimator, two asymptotic tests built on the Maximum Likelihood (ML) estimator, the F -test for fixed effects based on the Restricted Maximum Likelihood (REML) estimator in the mixed-model approach, two t -tests with n - 2 df based on first differences (FD) and first-difference ratios (FDR), and four modified t -tests using various corrections of the number of degrees of freedom. The FDR t -test, the REML F -test and the modified t -test using Dutilleul's effective sample size are the most valid among the testing procedures that do not assume the complete knowledge of the covariance matrix of the errors. However, modified t -tests are not applicable and the FDR t -test suffers from a lack of power when the regressor is fixed and trended ( i.e. , FDR is the same as FD in this case when observations are equally spaced), whereas the REML algorithm fails to converge at small sample sizes. The classical t -test is valid when the regressor is fixed and trended and autocorrelation among errors is predominantly negative, and when the regressor is random and AR(1), like the errors, and autocorrelation is moderately negative or positive. We discuss the results graphically, in terms of the circularity condition defined in repeated measures ANOVA and of the effective sample size used in correlation analysis with autocorrelated sample data. An example with environmental data is presented.     

On Non-parametric Testing, the Uniform Behaviour of the t-test, and Related Problems

Abstract.  In this article, we revisit some problems in non-parametric hypothesis testing. First, we extend the classical result of Bahadur & Savage [ Ann. Math. Statist . 25 (1956) 1115] to other testing problems, and we answer a conjecture of theirs. Other examples considered are testing whether or not the mean is rational, testing goodness-of-fit, and equivalence testing. Next, we discuss the uniform behaviour of the classical t -test. For most non-parametric models, the Bahadur–Savage result yields that the size of the t -test is one for every sample size. Even if we restrict attention to the family of symmetric distributions supported on a fixed compact set, the t -test is not even uniformly asymptotically level α . However, the convergence of the rejection probability is established uniformly over a large family with a very weak uniform integrability type of condition. Furthermore, under such a restriction, the t -test possesses an asymptotic maximin optimality property.     

Approximating the Shapiro-Wilk W-test for non-normality

A new approximation for the coefficients required to calculate the Shapiro-WilkW-test is derived. It is easy to calculate and applies for any sample size greater than 3. A normalizing transformation for theW statistic is given, enabling itsP-value to be computed simply. The distribution of the new approximation toW agrees well with published critical points which use exact coefficients.     

The ubiquitous angle

Summary.  Previously we used the geometry of n -dimensional space to derive the paired samples t -test and its p -value. In the present paper we describe the 'ubiquitous' application of these results to single degree of freedom linear model hypothesis tests. As examples, we derive the p - and t -values for the independent samples t -test, for testing a contrast in an analysis of variance and for testing the slope in a simple linear regression analysis. An angle θ in n -dimensional space is again pivotal in the development of the ideas. The relationships between p , t , θ , F and the correlation coefficient are also described by using a 'statistical triangle'.     

Weak identification in probit models with endogenous covariates

Weak identification is a well-known issue in the context of linear structural models. However, for probit models with endogenous explanatory variables, this problem has been little explored. In this paper, we study by simulating the behavior of the usual z-test and the LR test in the presence of weak identification. We find that the usual asymptotic z-test exhibits large level distortions (over-rejections under the null hypothesis). The magnitude of the level distortions depends heavily on the parameter value tested. In contrast, asymptotic LR tests do not over-reject and appear to be robust to weak identification.     

“十一五”期间辽宁省城镇劳动力供求趋势分析

关于城镇劳动力供求趋势的研究大多针对全国,而且预测方法各异。而就业形势严峻的辽宁省失业率位居全国前列,城镇劳动力供求矛盾十分突出。特别是随着产业结构的调整和国有企业改革的深化以及资源枯竭型城市的转化,“十一五”期间辽宁省城镇劳动力供求趋势将会发生何种变化?这是关系振兴东北老工业基地与结构调整以及国民经济持续较快增长的不可忽视的重要因素。因此,突破以往不区分城镇劳动力供给和全部劳动力供给以及仅来自城镇供给压力的预测方法,将政策因子引入模型,并采用加权最小二乘法有效地修正了模型,其拟合效果理想。     

Effect on probabilities and quantiles of adding a quantity with small variance

This paper gives simple approximations for the distribution function and quantiles of the sum X + Y when X is a continuous variable and Y is an independent variable with variance small compared to that of X . The approximations are based around the distribution function or quantiles of X and require only the first two or three moments of Y to be known. Example evaluations with X having a normal, Student's t or chi-squared distribution suggest that the approximations are good in unbounded tail regions when the ratio of variances is less than 0.2.     

A DISTRIBUTION-FREE TEST FOR TWO-WAY LAYOUT1

Many nonparametric tests have been proposed for the hypothesis of no row (treatment) effect in a one-way layout design. Examples of such tests are Kruskal-Wallis H-test, Bhapkar's (1961) V-test and Deshpande's (1965) L-test. However not many tests are available for testing the same hypothesis in a two-way layout design without interaction. Perhaps the only “established” test is the one due to Friedman (1937). However, it applies to the case of one observation per cell only. In this paper, a new distribution-free test is proposed for the hypothesis of row effect in a two-way layout design. It applies to the case of several observations per cell, not necessarily equal. The asymptotic efficiency of the proposed test relative to other tests is studied.     

Defying symmetry

John Venn, Cambridge don, is best known for the "Venn diagram", which he introduced in the course of his lectures on logic and published in 1880. But he had other strings to his bow. He wrote The Logic of Chance, which greatly influenced the development of the frequentist viewpoint and included the first drawing of a random walk in the plane (accompanied by the realization that in the limit the figure was fractal, as we should now say). He gave the first British lecture course on Theory of Statistics, in the Moral Science Tripos in 1890, and among historians he is famous for that branch of local history which concentrates on colleges, universities and their members.     

Stepwise Regression in Mixed Quantitative Linear Models with Autocorrelated Errors

ABSTRACT

In the stepwise procedure of selection of a fixed or a random explanatory variable in a mixed quantitative linear model with errors following a Gaussian stationary autocorrelated process, we have studied the efficiency of five estimators relative to Generalized Least Squares (GLS): Ordinary Least Squares (OLS), Maximum Likelihood (ML), Restricted Maximum Likelihood (REML), First Differences (FD), and First-Difference Ratios (FDR). We have also studied the validity and power of seven derived testing procedures, to assess the significance of the slope of the candidate explanatory variable x ₂ to enter the model in which there is already one regressor x ₁. In addition to five testing procedures of the literature, we considered the FDR t-test with n ? 3 df and the modified t-test with n? ? 3 df for partial correlations, where n? is Dutilleul's effective sample size. Efficiency, validity, and power were analyzed by Monte Carlo simulations, as functions of the nature, fixed vs. random (purely random or autocorrelated), of x ₁ and x ₂, the sample size and the autocorrelation of random terms in the regression model. We report extensive results for the autocorrelation structure of first-order autoregressive [AR(1)] type, and discuss results we obtained for other autocorrelation structures, such as spherical semivariogram, first-order moving average [MA(1)] and ARMA(1,1), but we could not present because of space constraints. Overall, we found that:

1. the efficiency of slope estimators and the validity of testing procedures depend primarily on the nature of x ₂, but not on that of x ₁;

2. FDR is the most inefficient slope estimator, regardless of the nature of x ₁ and x ₂;

3. REML is the most efficient of the slope estimators compared relative to GLS, provided the specified autocorrelation structure is correct and the sample size is large enough to ensure the convergence of its optimization algorithm;

4. the FDR t-test, the modified t-test and the REML t-test are the most valid of the testing procedures compared, despite the inefficiency of the FDR and OLS slope estimators for the former two;

5. the FDR t-test, however, suffers from a lack of power that varies with the nature of x ₁ and x ₂; and

6. the modified t-test for partial correlations, which does not require the specification of an autocorrelation structure, can be recommended when x ₁ is fixed or random and x ₂ is random, whether purely random or autocorrelated. Our results are illustrated by the environmental data that motivated our work.

     

Using pilot study information to increase efficiency in clinical trials

It is often necessary to conduct a pilot study to determine the sample size required for a clinical trial. Due to differences in sampling environments, the pilot data are usually discarded after sample size calculation. This paper tries to use the pilot information to modify the subsequent testing procedure when a two-sided t$t$-test or a regression model is used to compare two treatments. The new test maintains the required significance level regardless of the dissimilarity between the pilot and the target populations, but increases the power when the two are similar. The test is constructed based on the posterior distribution of the parameters given the pilot study information, but its properties are investigated from a frequentist's viewpoint. Due to the small likelihood of an irrelevant pilot population, the new approach is a viable alternative to the current practice.     

A cautionary tale

Anthony Edwards wrote this cautionary tale for genetics students at Stanford University whom he was teaching in 1965. It has not previously been published. "Its appearance now is due to my having been asked whether a copy from the papers of the Nobel Laureate Joshua Lederberg might be put on the web by the US National Library of Medicine", he says. "Lederberg was Professor of Genetics at Stanford at the time and I must have given him a copy. More remarkably, he thought it worth keeping." It concerns what is known as Simpson's paradox.     

On the Superiority of a Variable Sampling Interval Control Chart

The paper establishes the analytical grounds of the uniform superiority of a variable sampling interval (VSI) Shewhart control chart over the conventional fixed sampling interval (FSI) control chart, with respect to the zero-time performance, for a wide class of process distributions. We provide a sufficient condition on the distribution of a control chart statistic, and propose a criterion to determine the control limits and the regions in the in-control area of the VSI chart, corresponding to the different sampling intervals used by it. The condition and the criterion together ensure the uniform zero-time superiority of the VSI chart over the matched FSI chart, in detecting a process shift of any magnitude. It is shown that normal, Student's t and Laplace distributions satisfy the sufficient condition. In addition, chi-square, F and beta distributions satisfy it, provided that these are not extremely skewed. Further, it is illustrated that the superiority of the VSI feature is not trivial and cannot be assured if the sufficient condition is not satisfied or the control limits and the regions are not determined according to the proposed criterion. An application of the result to confirm the superiority of the VSI feature is demonstrated for the control chart for individual observations used to monitor a milk-pouch filling process.     

Modified d-optimal sequential procedure in allocation problem

When an experimenter wishes to compare t treatments with M experimental units, one of the first problems he faces is how to allocate N experimental units into t treatments. When no pre treat merit information about the experimental units is available, "randomization" is the widely accepted guiding principle to deal with the allocation problem But pre treat merit information usually is available, although it is seldom fully used for allocation purposes. Recently, Harville considered the allocation problem under a covariance model. He suggested a D-optimal sequential procedure that may be used to construct nearly D-optimal allocations. However, Harville's sequential procedure requires constructing a D-optimal initial allocation at the first stages and that may be computationally unfeasible in some real situations, Such construction is not needed for a new sequential.     

A closer examination on some parametric alternatives to the ANOVA F-test

In experiments, the classical (ANOVA) F-test is often used to test the omnibus null-hypothesis μ₁ = μ₂ ... = μ _j = ... = μ _n (all n population means are equal) in a one-way ANOVA design, even when one or more basic assumptions are being violated. In the first part of this article, we will briefly discuss the consequences of the different types of violations of the basic assumptions (dependent measurements, non-normality, heteroscedasticity) on the validity of the F-test. Secondly, we will present a simulation experiment, designed to compare the type I-error and power properties of both the F-test and some of its parametric adaptations: the Brown & Forsythe F^*-test and Welch’s V_w-test. It is concluded that the Welch V_w-test offers acceptable control over the type I-error rate in combination with (very) high power in most of the experimental conditions. Therefore, its use is highly recommended when one or more basic assumptions are being violated. In general, the use of the Brown & Forsythe F^*-test cannot be recommended on power considerations unless the design is balanced and the homoscedasticity assumption holds.     

A note on tolerance regions for random vectors and best linear predictors

When a (p+q)-variate column vector (x′,y′)′ has a (p+q)-variate normal density with mean vector (μ₁,μ₂) and covariance matrix Ω, unknown, Schervish (1980) obtains prediction intervals for the linear functions of a future y, given x. He bases the prediction interval on the F-distribution. However, for a specified linear function the statistic to be used is Student's t, since the prediction intervals based on t are shorter than those based on F. Similar results hold for the multivariate linear regression model.     

Robust projection indices

Loosely speaking a robust projection index is one that prefers projections involving true clusters over projections consisting of a cluster and an outlier. We introduce a mathematical definition of one-dimensional index robustness and describe a numerical experiment to measure it. We design five new indices based on measuring divergence from Student's t -distribution which are intended to be especially robust: the experiment shows that they are more robust than several established indices. The experiment also reveals more generally that the robustness of moment indices depends on the number of approximation terms, providing additional practical guidance for existing projection pursuit implementations. We investigate the theoretical properties of one new Student t -index and Hall's index and show that the new index automatically adapts its robustness to the degree of outlier contamination. We conclude by outlining the possibilities for extending our experiments to both higher dimensions and other new indices.