On testing random effects in the nested-error regression model

We derive an exact F-test for random effects in the nested-error regression model. The derivation utilizes a matrix decomposition that offers a transformation of the response vector into two independent subvectors. When the random effects are absent, the test statistic reduces to a ratio of two independent residual sums of squares that are computed by fitting a regression model using each subvector. A small simulation study compares the power of the F-test with various recent tests and shows that the proposed test has a competitive performance under small as well as large number of clusters.     

On the power of some tests for examining the stability of regression coefficients

The main objective of the study is to compare four different procedures to test for the stability of regression coefficients. The comparisons are based on a numerical study and are with respect to their abilities to detect various simple forms of parameter instabilities. Besides the power comparisons a special interest is directed towards the choice of “window length” in the tests based on moving sums of squared recursive and ordinary least-squares residuals.     

Nearly exact tests in factorial experiments using the aligned rank transform

A procedure is studied that uses rank-transformed data to perform exact and estimated exact tests, which is an alternative to the commonly used F-ratio test procedure. First, a common parametric test statistic is computed using rank-transformed data, where two methods of ranking-ranks taken for the original observations and ranks taken after aligning the observations-are studied. Significance is then determined using either the exact permutation distribution of the statistic or an estimate of this distribution based on a random sample of all possible permutations. Simulation studies compare the performance of this method with the normal theory parametric F-test and the traditional rank transform procedure. Power and nominal type I error rates are compared under conditions when normal theory assumptions are satisfied, as well as when these assumptions are violated. The method is studied for a two-factor factorial arrangement of treatments in a completely randomized design and for a split-unit experiment. The power of the tests rivals the parametric F-test when normal theory assumptions are satisfied, and is usually superior when normal theory assumptions are not satisfied. Based on the evidence of this study, the exact aligned rank procedure appears to be the overall best choice for performing tests in a general factorial experiment.     

Multiple hypothesis test for parameter constancy based on recursive residuals

This article presents a multiple hypothesis test procedure that combines two well known tests for structural change in the linear regression model, the CUSUM test and the recursive t test. The CUSUM test is run through the sequence of recursive residuals as usual; if the CUSUM plot does not violate the critical lines, one more step is taken to perform the t test for hypothesis of zero mean based on all recursive residuals. The asymptotic size of this multiple hypothesis test is derived; power simulation results suggest that it outperforms the traditional CUSUM test and complements other tests that are currently stressed in econometrics.     

Multiple hypothesis test for parameter constancy based on recursive residuals

This article presents a multiple hypothesis test procedure that combines two well known tests for structural change in the linear regression model, the CUSUM test and the recursive t test. The CUSUM test is run through the sequence of recursive residuals as usual; if the CUSUM plot does not violate the critical lines, one more step is taken to perform the t test for hypothesis of zero mean based on all recursive residuals. The asymptotic size of this multiple hypothesis test is derived; power simulation results suggest that it outperforms the traditional CUSUM test and complements other tests that are currently stressed in econometrics.     

The equivalence between likelihood ratio test and F-test for testing variance component in a balanced one-way random effects model

In mixed linear models, it is frequently of interest to test hypotheses on the variance components. F-test and likelihood ratio test (LRT) are commonly used for such purposes. Current LRTs available in literature are based on limiting distribution theory. With the development of finite sample distribution theory, it becomes possible to derive the exact test for likelihood ratio statistic. In this paper, we consider the problem of testing null hypotheses on the variance component in a one-way balanced random effects model. We use the exact test for the likelihood ratio statistic and compare the performance of F-test and LRT. Simulations provide strong support of the equivalence between these two tests. Furthermore, we prove the equivalence between these two tests mathematically.     

Local Fourier tests for structural change based on residuals

We propose a structural change test based on the recursive residuals with the local Fourier series estimators. The statistical properties of the proposed test are derived and the empirical properties are shown via simulation. We also consider other structural change tests based on CUSUM, MOSUM, moving estimates (ME), and empirical distribution functions with the recursive residuals and the ordinary residuals. Empirical powers are calculated in various structural change models for the comparison of those tests. These structural change tests are applied to South Korea's gross domestic product (GDP), South Korean Won to US Dollar currency exchange rates, and South Korea's Okun's law.     

Testing the equality of the variances of two linear models

Testing the equality of variances of two linear models with common β-parameter is considered. A test based on least squares residuals (ASR test) is proposed, and it is shown that this test is invariant under the group of scale and translation changes. For some special cases, it is also proved that this test has a monotone power function. Finding the exact critical values of this test is not easy; an approximation is given to facilitate the computation of these. The powers of the BLUS test, the F-test and the new test are computed for various alternatives and compared in a particular case. The proposed test seems to be locally more powerful than the alternative tests.     

A Monte Carlo study of rank tests for block designs

Tests based on ranks and the F-test are compared for block designs with n observations per block-treatment combination. Com-parisons are made on level of significance and on power. Rank tests examined include the Friedman as well as those using aligned ranks, weighted ranks, and the rank transformation. It is seen that the performance of these tests in relationship to each other depends on sample size, distribution of the random error term, and the severity of the block effects.     

Distribution of variance ratio in unbalanced random model under heterogeneity of error variances

The effects of heteroscedasticity have been studied on the mean and variance of F ratio and on the power of F-test in unbalanced one-way random model, numerically. The computed results reveal that the heteroscedasticity and unbalanoedness have combined effects. The mean and variance of F as well as the power of F-test increase with inequality of error variances under balanced and those unbalanced situations where more variable groups have larger size. The effects are of serious nature when more variable groups have smaller size.     

Significance tests and spurious correlation in regression models with autocorrelated errors

In this paper, we report some results on the exact significance level when the usual F-statistic is used in a linear regression model with autocorrelated disturbances. The exact tail area probabilities sometimes differ substantially from the nominal size used in an ‘F-test’ and from upper-bound probabilities derived by Kiviet (1979) which do not depend on the values of the regressors. A similar conclusion is also reached for the exact size of the significance tests for the spurious regressions considered by Granger and Newbold (1974, 1977). The results indicate once more that one has to be careful when using an algebraic F-test in the presence of autoregressive errors. However then too, the Durbin-Watson test is expected to indicate the presence of autocorrelation.     

Schwarz information criterion based tests for a change-point in regression models

Standard Schwarz information criterion for testing a change-point in regression models is considered and two new test procedures are evolved. The case of small sample size is investigated. Numerical approximations to the power against various alternatives are given and compared with powers of tests based on r-ahead recursive residuals and of the CUSUM of squares test. Application of these procedures to some real data is also provided.     

Size performance of some tests in one-way anova

Tsui and Weerahandi (1989) introduced the notion of generalized p-values and since then this idea is used to solve many statistical testing problems. Heteroskedasticity is one of the major practical problems encountered in ANOVA problems. To compare the means of several groups under heteroskedasticity approximate tests are used in the literature. Weerahandi (1995a) introduced a test using the notion of generalized p-values for comparing the means of several populations when the variances are not equal. This test is referred to as a generalized F-test.

In this paper we compare the size performance of the Generalized F-test and four other widely used procedures: the Classical F-test for ANOVA, the F-test obtained by the weighted least-squares to adjust for heteroskedasticity, the Brown-Forsythe-test, and the Welch-test. The comparison is based on a simulation study of size performance of tests applied to the balanced one-way model. The intended level of the tests is set at 0.05. While the Generalized F-test was found to have size not exceeding the intended level, as heteroskedasticity becomes severe the other tests were found to have poor size performance. With mild heteroskedasticity the Welch-test and the classical ANOVA F-test have the intended levels, and the Welch-test was found to perform better than the latter. Widely used (due to computational convenience) weighted F-test was found to have very serious size problems. The size advantage of the generalized F-test was also found to be robust even under severe deviations from the assumption of normality.     

A comparison of algorithms for exact goodness-of-fit tests for multinomial data

Multinomial goodness-of-fit tests arise in a diversity of milieu. The long history of the problem has spawned a multitude of asymptotic tests. If the sample size relative to the number of categories is small, the accuracy of these tests is compromised. In that case, an exact test is a prudent option. But such tests are computationally intensive and need efficient algorithms. This paper gives a conceptual overview, and empirical comparisons of two avenues, namely the network and fast Fourier transform (FFT) algorithms, for an exact goodness-of-fit test on a multinomial. We show that a recursive execution of a polynomial product forms the basis of both these approaches. Specific details to implement the network method, and techniques to enhance the efficiency of the FFT algorithm are given. Our empirical comparisons show that for exact analysis with the chi-square and likelihood ratio statistics, the network-cum-polynomial multiplication algorithm is the more efficient and accurate of the two.     

Sensitivity analysis for two control groups

In many experiments where data have been collected at two points in time (pre-treatment and post-treatment), investigators wish to determine if there is a difference between two treatment groups. In recent years it has been proposed that an appropriate statistical analysis to determine if treatment differences exist is to use the post-treatment values as the primary comparison variables and the pre-treatment values as covariates. When there are several outcome variables, we propose new tests based on residuals as alternatives to existing methods and investigate how the powers of the new and existing tests are affected by various choices of covariates. The limiting distribution of the test statistic of the new test based on residuals is given. Monte Carlo simulations are employed in the power comparisons.     

Power comparisons between the F-test,the studentised range test,and an optimal test of the equality of several normal means

The power assessment of tests of the equality of k normal means such as the k treatment means in a one-way fixed effects analysis of variance model is addressed. Power assessment is considered in terms of a constraint on the range of the treatment means. The power properties of the standard F-test and Studentised range test are compared with those of an optimal (minimax) test procedure, which is known to maximise power levels under this constraint. It is shown that the standard test procedures compare well with the optimal test procedure, and in particular, the Studentised range test is shown to be practically as good as optimal in this setting.     

Negative exponential disparity based family of goodness-of-fit tests for multinomial models

This paper investigates a new family of goodness-of-fit tests based on the negative exponential disparities. This family includes the popular Pearson's chi-square as a member and is a subclass of the general class of disparity tests (Basu and Sarkar, 1994) which also contains the family of power divergence statistics. Pitman efficiency and finite sample power comparisons between different members of this new family are made. Three asymptotic approximations of the exact null distributions of the negative exponential disparity famiiy of tests are discussed. Some numerical results on the small sample perfomance of this family of tests are presented for the symmetric null hypothesis. It is shown that the negative exponential disparity famiiy, Like the power divergence family, produces a new goodness-of-fit test statistic that can be a very attractive alternative to the Pearson's chi-square. Some numerical results suggest that, application of this test statistic, as an alternative to Pearson's chi-square, could be preferable to the I ^2/3 statistic of Cressie and Read (1984) under the use of chi-square critical values.     

Assessing and Improving the Performance of Nearly Efficient Unit Root Tests in Small Samples

Abstract

The development of unit root tests continues unabated, with many recent contributions using techniques such as generalized least squares (GLS) detrending and recursive detrending to improve the power of the test. In this article, the relation between the seemingly disparate tests is demonstrated by algebraically nesting all of them as ratios of quadratic forms in normal variables. By doing so, and using the exact sampling distribution of the ratio, it is straightforward to compute, examine, and compare the test' critical values and power functions. It is shown that use of GLS detrending parameters other than those recommended in the literature can lead to substantial power improvements. The open and important question regarding the nature of the first observation is addressed. Tests with high power are proposed irrespective of the distribution of the initial observation, which should be of great use in practical applications.     

Avoiding 'data snooping' in multilevel and mixed effects models

Summary.  Multilevel or mixed effects models are commonly applied to hierarchical data. The level 2 residuals, which are otherwise known as random effects, are often of both substantive and diagnostic interest. Substantively, they are frequently used for institutional comparisons or rankings. Diagnostically, they are used to assess the model assumptions at the group level. Inference on the level 2 residuals, however, typically does not account for 'data snooping', i.e. for the harmful effects of carrying out a multitude of hypothesis tests at the same time. We provide a very general framework that encompasses both of the following inference problems: inference on the 'absolute' level 2 residuals to determine which are significantly different from 0, and inference on any prespecified number of pairwise comparisons. Thus, the user has the choice of testing the comparisons of interest. As our methods are flexible with respect to the estimation method that is invoked, the user may choose the desired estimation method accordingly. We demonstrate the methods with the London education authority data, the wafer data and the National Educational Longitudinal Study data.     

Jarque–Bera Test and its Competitors for Testing Normality – A Power Comparison

For testing normality we investigate the power of several tests, first of all, the well-known test of Jarque & Bera (1980) and furthermore the tests of Kuiper (1960) and Shapiro & Wilk (1965) as well as tests of Kolmogorov–Smirnov and Cramér-von Mises type. The tests on normality are based, first, on independent random variables (model I) and, second, on the residuals in the classical linear regression (model II). We investigate the exact critical values of the Jarque–Bera test and the Kolmogorov–Smirnov and Cramér-von Mises tests, in the latter case for the original and standardized observations where the unknown parameters μ and σ have to be estimated. The power comparison is carried out via Monte Carlo simulation assuming the model of contaminated normal distributions with varying parameters μ and σ and different proportions of contamination. It turns out that for the Jarque–Bera test the approximation of critical values by the chi-square distribution does not work very well. The test is superior in power to its competitors for symmetric distributions with medium up to long tails and for slightly skewed distributions with long tails. The power of the Jarque–Bera test is poor for distributions with short tails, especially if the shape is bimodal – sometimes the test is even biased. In this case a modification of the Cramér-von Mises test or the Shapiro–Wilk test may be recommended.