Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models

ABSTRACT

In panel data models and other regressions with unobserved effects, fixed effects estimation is often paired with cluster-robust variance estimation (CRVE) to account for heteroscedasticity and un-modeled dependence among the errors. Although asymptotically consistent, CRVE can be biased downward when the number of clusters is small, leading to hypothesis tests with rejection rates that are too high. More accurate tests can be constructed using bias-reduced linearization (BRL), which corrects the CRVE based on a working model, in conjunction with a Satterthwaite approximation for t-tests. We propose a generalization of BRL that can be applied in models with arbitrary sets of fixed effects, where the original BRL method is undefined, and describe how to apply the method when the regression is estimated after absorbing the fixed effects. We also propose a small-sample test for multiple-parameter hypotheses, which generalizes the Satterthwaite approximation for t-tests. In simulations covering a wide range of scenarios, we find that the conventional cluster-robust Wald test can severely over-reject while the proposed small-sample test maintains Type I error close to nominal levels. The proposed methods are implemented in an R package called clubSandwich. This article has online supplementary materials.     

The split sample permutation t-tests

Without the exchangeability assumption, permutation tests for comparing two population means do not provide exact control of the probability of making a Type I error. Another drawback of permutation tests is that it cannot be used to test hypothesis about one population. In this paper, we propose a new type of permutation tests for testing the difference between two population means: the split sample permutation t-tests. We show that the split sample permutation t-tests do not require the exchangeability assumption, are asymptotically exact and can be easily extended to testing hypothesis about one population. Extensive simulations were carried out to evaluate the performance of two specific split sample permutation t-tests: the split in the middle permutation t-test and the split in the end permutation t-test. The simulation results show that the split in the middle permutation t-test has comparable performance to the permutation test if the population distributions are symmetric and satisfy the exchangeability assumption. Otherwise, the split in the end permutation t-test has significantly more accurate control of level of significance than the split in the middle permutation t-test and other existing permutation tests.     

An adaptive test for the one-way layout

An adaptive test is proposed for the one-way layout. This test procedure uses the order statistics of the combined data to obtain estimates of percentiles, which are used to select an appropriate set of rank scores for the one-way test statistic. This test is designed to have reasonably high power over a range of distributions. The adaptive procedure proposed for a one-way layout is a generalization of an existing two-sample adaptive test procedure. In this Monte Carlo study, the power and significance level of the F-test, the Kruskal-Wallis test, the normal scores test, and the adaptive test were evaluated for the one-way layout. All tests maintained their significance level for data sets having at least 24 observations. The simulation results show that the adaptive test is more powerful than the other tests for skewed distributions if the total number of observations equals or exceeds 24. For data sets having at least 60 observations the adaptive test is also more powerful than the F-test for some symmetric distributions.     

Asymptotic test statistics for coefficients of variation

A one-sample asymptotically normal test statistic Is derived for testing the hypothesis that the coefficient of variation of a normal population is equal to a specified value. Based on this derivation, an asymptotically noraml two-sample test statistic and an asymptotically chi-square k-sample test statistic are derived for testing the hypothesis that the coefficients of variation of k ≥2 normal populations are equal. The two and k-sample test statistics allow for unequal sample sizes. Results of a simulation study which evaluate the size and power of the test statistics and compare the test statistics to earlier ones developed by McKay (1932) and Bennett (1976) are presented.     

Linear hypothesis testing for weighted functional data with applications

In socioeconomic areas, functional observations may be collected with weights, called weighted functional data. In this paper, we deal with a general linear hypothesis testing (GLHT) problem in the framework of functional analysis of variance with weighted functional data. With weights taken into account, we obtain unbiased and consistent estimators of the group mean and covariance functions. For the GLHT problem, we obtain a pointwise F-test statistic and build two global tests, respectively, via integrating the pointwise F-test statistic or taking its supremum over an interval of interest. The asymptotic distributions of test statistics under the null and some local alternatives are derived. Methods for approximating their null distributions are discussed. An application of the proposed methods to density function data is also presented. Intensive simulation studies and two real data examples show that the proposed tests outperform the existing competitors substantially in terms of size control and power.     

A comparison of several adaptive tests for paired data

In the last few years, two adaptive tests for paired data have been proposed. One test proposed by Freidlin et al. [On the use of the Shapiro–Wilk test in two-stage adaptive inference for paired data from moderate to very heavy tailed distributions, Biom. J. 45 (2003), pp. 887–900] is a two-stage procedure that uses a selection statistic to determine which of three rank scores to use in the computation of the test statistic. Another statistic, proposed by O'Gorman [Applied Adaptive Statistical Methods: Tests of Significance and Confidence Intervals, Society for Industrial and Applied Mathematics, Philadelphia, 2004], uses a weighted t-test with the weights determined by the data. These two methods, and an earlier rank-based adaptive test proposed by Randles and Hogg [Adaptive Distribution-free Tests, Commun. Stat. 2 (1973), pp. 337–356], are compared with the t-test and to Wilcoxon's signed-rank test. For sample sizes between 15 and 50, the results show that the adaptive test proposed by Freidlin et al. and the adaptive test proposed by O'Gorman have higher power than the other tests over a range of moderate to long-tailed symmetric distributions. The results also show that the test proposed by O'Gorman has greater power than the other tests for short-tailed distributions. For sample sizes greater than 50 and for small sample sizes the adaptive test proposed by O'Gorman has the highest power for most distributions.     

Some asymptotic results for hypothesis testing in multivariate failure time data

This paper presents a class of generalized Wald, generalized score and generalized likelihood ratio statistics for hypothesis testing and model selection for multivariate failure time data. These statistics are based on a marginal hazard model with a common baseline hazard function. The large sample distributions of these statistics are examined. It is shown that the proposed test statistics follow asymptotically a weighted sum of independent χ₁² distributions.     

A class of k-sample distribution-free tests for location against ordered alternatives

This paper introduces a new class of distribution-free tests for testing the homogeneity of several location parameters against ordered alternatives. The proposed class of test statistics is based on a linear combination of two-sample U-statistics based on subsample extremes. The mean and variance of the test statistic are obtained under the null hypothesis as well as under the sequence of local alternatives. The optimal weights are also determined. It is shown via Pitman ARE comparisons that the proposed class of test statistics performs better than its competitor tests in case of heavy-tailed and long-tailed distributions     

On a multiple endpoints testing problem

In a clinical trial comparing drug with placebo, where there are multiple primary endpoints, we consider testing problems where an efficacious drug effect can be claimed only if statistical significance is demonstrated at the nominal level for all endpoints. Under the assumption that the data are multivariate normal, the multiple endpoint-testing problem is formulated. The usual testing procedure involves testing each endpoint separately at the same significance level using two-sample t-tests, and claiming drug efficacy only if each t-statistic is significant. In this paper we investigate properties of this procedure. We show that it is identical to both an intersection union test and the likelihood ratio test. A simple expression for the p-value is given. The level and power function are studied; it is shown that the test may be conservative and that it is biased. Computable bounds for the power function are established.     

A Nonparametric Comparison of Conditional Distributions with Nonnegligible Cure Fractions

Survival data with nonnegligible cure fractions are commonly encountered in clinical cancer clinical research. Recently, several authors (e.g. Kuk and Chen, Biometrika 79 (1992) 531; Maller and Zhou, Journal of Applied Probability, 30 (1993) 602; Peng and Dear, Biometrics, 56 (2000) 237; Sy and Taylor, Biometrics 56 (2000) 227) have proposed to use semiparametric cure models to analyze such data. Much of the existing work has been emphasized on cure detections and regression techniques. In contrast, this project focuses on the hypothesis testing in the presence of a cure fraction. Specifically, our interest lies in detecting whether there exists survival differences among noncured patients between treatment arms. For this purpose, we investigate the use of a modified Cramér-von Mises statistic for two-sample survival comparisons within the framework of cure models. Such a test has been studied by Tamura et al., (Statistics in Medicine 19, 2000, 2169) using bootstrap procedure. We will focus on developing asymptotic theory and convergent algorithms in this paper. We show that the limiting distributions of the Cramér-von Mises statistic under the null hypothesis can be represented by stochastic integrals and a weighted noncentral chi-squares. Both representations lead to concrete numerical schemes for computing the limiting distributions. The algorithms can be easily implemented for data analysis and significantly reduce computing time compared to the bootstrap approach. For illustrative purposes, we apply the proposed test to a published clinical trial.     

Using -splines to test the linearity of partially linear models

The nonparametric component in a partially linear model is estimated by a linear combination of fixed-knot cubic B-splines with a second-order difference penalty on the adjacent B-spline coefficients. The resulting penalized least-squares estimator is used to construct two Wald-type spline-based test statistics for the null hypothesis of the linearity of the nonparametric function. When the number of knots is fixed, the first test statistic asymptotically has the distribution of a linear combination of independent chi-squared random variables, each with one degree of freedom, under the null hypothesis. The smoothing parameter is determined by specifying a value for the asymptotically expected value of the test statistic under the null hypothesis. When the number of knots is fixed and under the null hypothesis, the second test statistic asymptotically has a chi-squared distribution with K=q+2 degrees of freedom, where q is the number of knots used for estimation. The power performances of the two proposed tests are investigated via simulation experiments, and the practicality of the proposed methodology is illustrated using a real-life data set.     

Power Comparison of Multivariate Wilcoxon-type Tests Based on the Jurečková–Kalina’s Ranks of Distances

A multivariate two-sample testing problem is one of the most important topics in nonparametric statistics. One of the multivariate two-sample testing problems based on the Jure?ková–Kalina ranks of distance is discussed in this article. Further, a multivariate Wilcoxon-type test is proposed for testing the equality of two continuous distribution functions. Simulations are used to investigate the power of this test for the two-sided alternative with various population distributions. The results show that the proposed test statistic is more suitable than various existing statistics for testing a shift in the locationt and location-scale parameters.     

A class of tests of proportional hazards assumption for left-truncated and right-censored data

In this paper, we proposed a class of tests of proportional hazards assumption for left-truncated and right-censored data based on a pair of estimators of the hazard ratio constant. Using counting process and martingale theory, the asymptotically normal distribution of the test statistic is derived and a family of consistent estimators of variance are also provided. Extensive simulation studies were conducted to evaluate the performance of the proposed test statistics under finite sample situations. Two real data sets are analyzed to illustrate our method.     

Testing of two sample proportional intensity assumption for non-homogeneous Poisson processes

For two independent non-homogeneous Poisson processes with unknown intensities we propose a test for testing the hypothesis that the ratio of the intensities is constant versus it is increasing on (0,t]. The existing test procedures for testing such relative trends are based on conditioning on the number of failures observed in (0,t] from the two processes. Our test is unconditional and is based on the original time truncated data which enables us to have meaningful asymptotics. We obtain the asymptotic null distribution (as t becomes large) of the proposed test statistic and show that the proposed test is consistent against several large classes of alternatives. It was observed by Park and Kim (IEEE. Trans. Rehab. 40 (1), 1992, 107–111) that it is difficult to distinguish between the power-law and log-linear processes for certain parameter values. We show that our test is consistent for such alternatives also.     

Closed testing procedures for all pairwise comparisons in a randomized block design

We consider multiple comparison test procedures among treatment effects in a randomized block design. We propose closed testing procedures based on maximum values of some two-sample t test statistics and based on F test statistics. It is shown that the proposed procedures are more powerful than single-step procedures and the REGW (Ryan/Einot–Gabriel/Welsch)-type tests. Next, we consider the randomized block design under simple ordered restrictions of treatment effects. We propose closed testing procedures based on maximum values of two-sample one-sided t test statistics and based on Batholomew’s statistics for all pairwise comparisons of treatment effects. Although single-step multiple comparison procedures are utilized in general, the power of these procedures is low for a large number of groups. The closed testing procedures stated in the present article are more powerful than the single-step procedures. Simulation studies are performed under the null hypothesis and some alternative hypotheses. In this studies, the proposed procedures show a good performance.     

Second order asymptotic and non-asymptotic optimality properties of combined tests

Let p independent test statistics be available to test a null hypothesis concerned with the same parameter. The p are assumed to be similar tests. Asymptotic and non-asymptotic optimality properties of combined tests are studied. The asymptotic study centers around two notions. The first is Bahadur efficiency. The second is based on a notion of second order comparisons. The non-asymptotic study is concerned with admissibility questions. Most of the popular combining methods are considered along with a method not studied in the past. Among the results are the following: Assume each of the p statistics has the same Bahadur slope. Then the combined test based on the sum of normal transforms, is asymptotically best among all tests studied, by virtue of second order considerations. Most of the popular combined tests are inadmissible for testing the noncentrality parameter of chi-square, t, and F distributions. For chi-square a combined test is offered which is admissible, asymptotically optimal (first order), asymptotically optimal (second order) among all tests studied, and for which critical values are obtainable in special cases. Extensions of the basic model are given.     

Function-based hypothesis testing in censored two-sample location-scale models

Function-based hypothesis testing in two-sample location-scale models has been addressed for uncensored data using the empirical characteristic function. A test of adequacy in censored two-sample location-scale models is lacking, however. A plug-in empirical likelihood approach is used to introduce a test statistic, which, asymptotically, is not distribution free. Hence for practical situations bootstrap is necessary for performing the test. A multiplier bootstrap and a model appropriate resampling procedure are given to approximate critical values from the null asymptotic distribution. Although minimum distance estimators of the location and scale are deployed for the plug-in, any consistent estimators can be used. Numerical studies are carried out that validate the proposed testing method, and real example illustrations are given.

     

Robustness without rank order statistics

An alternative to conventional rank tests based on a Euclidean distance analysis space is described. Comparisons based on exact probability values among classical two-sample t-tests and the Wilcoxon–Mann–Whitney test illustrate the advantages of the Euclidean distance analysis space alternative.     

Empirical phi-divergence test statistics for the difference of means of two populations

Empirical phi-divergence test statistics have demostrated to be a useful technique for the simple null hypothesis to improve the finite sample behavior of the classical likelihood ratio test statistic, as well as for model misspecification problems, in both cases for the one population problem. This paper introduces this methodology for two-sample problems. A simulation study illustrates situations in which the new test statistics become a competitive tool with respect to the classical z test and the likelihood ratio test statistic.     

Adaptive Test for Testing the Difference in Survival Distributions

An adaptive test is proposed for the problem of testing the difference in survival distributions when the shape of the hazard ratio is unknown, hence the efficient test is unknown. The proposed adaptive test selects a test statistic from a finite set of the weighted logrank statistics T on the basis of the estimates of the efficiencies of the tests in T for given data. The efficiency estimator uses the length of the test based nonparametric confidence interval for the shift in a time transformed shift model. The suggested adaptive test is shown to be asymptotically efficient among the tests in T under the time transformed shift model and conditions commonly used in survival analysis. Simulations demonstrate that the adaptive test enjoys good small sample properties and in most situations is more powerful than the test using the maximum of the tests in T.