Simulation-Based Tests that Can Use Any Number of Simulations

Conventional procedures for Monte Carlo and bootstrap tests require that B, the number of simulations, satisfy a specific relationship with the level of the test. Otherwise, a test that would instead be exact will either overreject or underreject for finite B. We present expressions for the rejection frequencies associated with existing procedures and propose a new procedure that yields exact Monte Carlo tests for any positive value of B. This procedure, which can also be used for bootstrap tests, is likely to be most useful when simulation is expensive.     

Bootstrapping analogs of the one way MANOVA test

Abstract

Analogs of the classical one way MANOVA model have recently been suggested that do not assume that population covariance matrices are equal or that the error vector distribution is known. These tests are based on the sample mean and sample covariance matrix corresponding to each of the p populations. We show how to extend these tests using other measures of location such as the trimmed mean or coordinatewise median. These new bootstrap tests can have some outlier resistance, and can perform better than the tests based on the sample mean if the error vector distribution is heavy tailed.     

Automatic Block-Length Selection for the Dependent Bootstrap

Abstract

We review the different block bootstrap methods for time series, and present them in a unified framework. We then revisit a recent result of Lahiri [Lahiri, S. N. (1999b). Theoretical comparisons of block bootstrap methods, Ann. Statist. 27:386–404] comparing the different methods and give a corrected bound on their asymptotic relative efficiency; we also introduce a new notion of finite-sample “attainable” relative efficiency. Finally, based on the notion of spectral estimation via the flat-top lag-windows of Politis and Romano [Politis, D. N., Romano, J. P. (1995). Bias-corrected nonparametric spectral estimation. J. Time Series Anal. 16:67–103], we propose practically useful estimators of the optimal block size for the aforementioned block bootstrap methods. Our estimators are characterized by the fastest possible rate of convergence which is adaptive on the strength of the correlation of the time series as measured by the correlogram.     

FAST DOUBLE BOOTSTRAP TESTS OF NONNESTED LINEAR REGRESSION MODELS

ABSTRACT

It has been shown in previous work that bootstrapping the J test for nonnested linear regression models dramatically improves its finite-sample performance. We provide evidence that a more sophisticated bootstrap procedure, which we call the fast double bootstrap, produces a very substantial further improvement in cases where the ordinary bootstrap does not work as well as it might. This FDB procedure is only about twice as expensive as the usual single bootstrap.     

On the Exact Size of Tests of Treatment Effects in Multi‐Arm Clinical Trials

When testing treatment effects in multi‐arm clinical trials, the Bonferroni method or the method of Simes 1986) is used to adjust for the multiple comparisons. When control of the family‐wise error rate is required, these methods are combined with the close testing principle of Marcus et al. (1976). Under weak assumptions, the resulting p‐values all give rise to valid tests provided that the basic test used for each treatment is valid. However, standard tests can be far from valid, especially when the endpoint is binary and when sample sizes are unbalanced, as is common in multi‐arm clinical trials. This paper looks at the relationship between size deviations of the component test and size deviations of the multiple comparison test. The conclusion is that multiple comparison tests are as imperfect as the basic tests at nominal size α/m where m is the number of treatments. This, admittedly not unexpected, conclusion implies that these methods should only be used when the component test is very accurate at small nominal sizes. For binary end‐points, this suggests use of the parametric bootstrap test. All these conclusions are supported by a detailed numerical study.     

Comments

In this paper we compare Bartlett-corrected, bootstrap, and fast double bootstrap tests on maximum likelihood estimates of cointegration parameters. The key result is that both the bootstrap and the Bartlett-corrected tests must be based on the unrestricted estimates of the cointegrating vectors: procedures based on the restricted estimates have almost no power. The small sample size bias of the asymptotic test appears so severe as to advise strongly against its use with the sample sizes commonly available; the fast double bootstrap test minimizes size bias, while the Bartlett-corrected test is somehow more powerful.     

Bootstrap versus traditional hypothesis testing procedures for coefficients in least absolute value regression

Several approaches to hypothesis testing for coefficients in least absolute value regression are compared using a Monte Carlo simulation: likelihood ratio test, Lagrange multiplier test, and three versions of the bootstrap hypothesis test. Factors considered that might influence test performance include the disturbance distribution, the type of independent variable, and the sample size. Overall, the likelihood ratio and the bootstrap tests perform best, with the likelihood ratio test being marginally more powerful. Least absolute value tests are also compared to the standard t test and three versions of the bootstrapped t test for least squares regression.     

Asymptotically refined score and GOF tests for inverse Gaussian models

ABSTRACT

The score test and the GOF test for the inverse Gaussian distribution, in particular the latter, are known to have large size distortion and hence unreliable power when referring to the asymptotic critical values. We show in this paper that with the appropriately bootstrapped critical values, these tests become second-order accurate, with size distortion being essentially eliminated and power more reliable. Two major generalizations of the score test are made: one is to allow the data to be right-censored, and the other is to allow the existence of covariate effects. A data mapping method is introduced for the bootstrap to be able to produce censored data that are conformable with the null model. Monte Carlo results clearly favour the proposed bootstrap tests. Real data illustrations are given.     

Robust Lagrange multiplier test for detecting ARCH/GARCH effect using permutation and bootstrap

The Lagrange Multiplier (LM) test is one of the principal tools to detect ARCH and GARCH effects in financial data analysis. However, when the underlying data are non‐normal, which is often the case in practice, the asymptotic LM test, based on the χ²‐approximation of critical values, is known to perform poorly, particularly for small and moderate sample sizes. In this paper we propose to employ two re‐sampling techniques to find critical values of the LM test, namely permutation and bootstrap. We derive the properties of exactness and asymptotically correctness for the permutation and bootstrap LM tests, respectively. Our numerical studies indicate that the proposed re‐sampled algorithms significantly improve size and power of the LM test in both skewed and heavy‐tailed processes. We also illustrate our new approaches with an application to the analysis of the Euro/USD currency exchange rates and the German stock index. The Canadian Journal of Statistics 40: 405–426; 2012 © 2012 Statistical Society of Canada     

Parametric bootstrap and approximate tests for two Poisson variates

The parametric bootstrap tests and the asymptotic or approximate tests for detecting difference of two Poisson means are compared. The test statistics used are the Wald statistics with and without log-transformation, the Cox F statistic and the likelihood ratio statistic. It is found that the type I error rate of an asymptotic/approximate test may deviate too much from the nominal significance level α under some situations. It is recommended that we should use the parametric bootstrap tests, under which the four test statistics are similarly powerful and their type I error rates are all close to α. We apply the tests to breast cancer data and injurious motor vehicle crash data.     

On the Power of Bootstrapped Specification Tests

Abstract

Decisions based on econometric model estimates may not have the expected effect if the model is misspecified. Thus, specification tests should precede any analysis. Bierens' specification test is consistent and has optimality properties against some local alternatives. A shortcoming is that the test statistic is not distribution free, even asymptotically. This makes the test unfeasible. There have been many suggestions to circumvent this problem, including the use of upper bounds for the critical values. However, these suggestions lead to tests that lose power and optimality against local alternatives. In this paper we show that bootstrap methods allow us to recover power and optimality of Bierens' original test. Bootstrap also provides reliable p-values, which have a central role in Fisher's theory of hypothesis testing. The paper also includes a discussion of the properties of the bootstrap Nonlinear Least Squares Estimator under local alternatives.     

Asymptotic properties of the bootstrap unit root test statistic under possibly infinite variance

ABSTRACT

In this article, the unit root test for the AR(1) model is discussed, under the condition that the innovations of the model are in the domain of attraction of the normal law with possibly infinite variances. By using residual bootstrap with sample size m < n (n being the size of the original sample), we bootstrap the least-squares estimator of the autoregressive parameter. Under some mild assumptions, we prove that the null distribution of the unit root test statistic based on the least-square estimator of the autoregressive parameter can be approximated by using residual bootstrap.     

The modified permutation entropy-based independence test of time series

Abstract

In time series, it is essential to check the independence of data by means of a proper method or an appropriate statistical test before any further analysis. Therefore, among different independence tests, a powerful and productive test has been introduced by Matilla-García and Marín via m-dimensional vectorial process, in which the value of the process at time t includes m-histories of the primary process. However, this method causes a dependency for the vectors even when the independence assumption of random variables is considered. Considering this dependency, a modified test is obtained in this article through presenting a new asymptotic distribution based on weighted chi-square random variables. Also, some other alterations to the test have been made via bootstrap method and by controlling the overlap. Compared with the primary test, it is obtained that not only the modified test is more accurate but also, it possesses higher power.     

Cross-sectional correlation robust tests for panel cointegration

We use meta-analytic procedures to develop new tests for panel cointegration, combining p-values from time-series cointegration tests on the units of the panel. The tests are robust to heterogeneity and cross-sectional dependence between the panel units. To achieve the latter, we employ a sieve bootstrap procedure with joint resampling of the units’ residuals. A simulation study shows that the tests can have substantially smaller size distortion than tests ignoring the presence of cross-sectional dependence while preserving high power. We apply the tests to a panel of post-Bretton Woods data to test for weak purchasing power parity.     

The size accuracy of combination tests

Adaptive clinical trials typically involve several independent stages. The P‐values from each stage are synthesized through a so‐called combination function which ensures that the overall test will be valid if the stagewise tests are valid. In practice however, approximate and possibly invalid stagewise tests are used. This paper studies how imperfections of the stagewise tests feed through into the combination test. Several general results are proven including some for discrete models. An approximation formula which directly links the combined size accuracy to the component size accuracy is given. In the wider context of adaptive clinical trials, the main conclusion is that the basic tests used should be size accurate at nominal sizes both much larger and also much smaller than nominal desired size. For binary outcomes, the implication is that the parametric bootstrap should be used.     

Bootstrap procedures for variance breaks test in time series with a changing trend

In this article, we consider the problem of testing for variance breaks in time series in the presence of a changing trend. In performing the test, we employ the cumulative sum of squares (CUSSQ) test introduced by Inclán and Tiao (1994, J.?Amer.?Statist.?Assoc., 89, 913 ? 923). It is shown that CUSSQ test is not robust in the case of broken trend and its asymptotic distribution does not convergence to the sup of a standard Brownian bridge. As a remedy, a bootstrap approximation method is designed to alleviate the size distortions of test statistic while preserving its high power. Via a bootstrap functional central limit theorem, the consistency of these bootstrap procedures is established under general assumptions. Simulation results are provided for illustration and an empirical example of application to a set of high frequency real data is given.     

Testing the Equality of Multivariate Distributions Using the Bootstrap and Integrated Empirical Processes

We introduce some projected integrated empirical processes for testing the equality of two multivariate distributions. The bootstrap is used for determining the approximate critical values. We show that the bootstrap test is consistent. A number-theoretic method is used for efficient computation of the bootstrap critical values. Some simulation results are also given.     

A Comparative Study of Model-based Tests of Independence for Ordinal Data Using the Bootstrap

When analyzing categorical data using loglinear models in sparse contingency tables, asymptotic results may fail. In this paper the empirical properties of three commonly used asymptotic tests of independence, based on the uniform association model for ordinal data, are investigated by means of Monte Carlo simulation. Five different bootstrapped tests of independence are presented and compared to the asymptotic tests. The comparisons are made with respect to both size and power properties of the tests. Results indicate that the asymptotic tests have poor size control. The test based on the estimated association parameter is severely conservative and the two chi-squared tests (Pearson, likelihood-ratio) are both liberal. The bootstrap tests that either use a parametric assumption or are based on non-pivotal test statistics do not perform better than the asymptotic tests in all situations. The bootstrap tests that are based on approximately pivotal statistics provide both adjustment of size and enhancement of power. These tests are therefore recommended for use in situations similar to those included in the simulation study.     

Testing for stationarity in heterogeneous panel data in the presence of cross-section dependence

The panel variant of the KPSS tests developed by Hadri [Hadri, K., 2000, Testing for stationarity in heterogeneous panels. Econometrics Journal, 3, 148–161] for the null of stationarity suffers from size distortions in the presence of cross-section dependence. However, applying the bootstrap methodology, we find that these tests are approximately correctly sized.     

On testing inference in beta regressions

This article deals with testing inference in the class of beta regression models with varying dispersion. We focus on inference in small samples. We perform a numerical analysis in order to evaluate the sizes and powers of different tests. We consider the likelihood ratio test, two adjusted likelihood ratio tests proposed by Ferrari and Pinheiro [Improved likelihood inference in beta regression, J. Stat. Comput. Simul. 81 (2011), pp. 431–443], the score test, the Wald test and bootstrap versions of the likelihood ratio, score and Wald tests. We perform tests on the parameters that index the mean submodel and also on the parameters in the linear predictor of the precision submodel. Overall, the numerical evidence favours the bootstrap tests. It is also shown that the score test is considerably less size-distorted than the likelihood ratio and Wald tests. An application that uses real (not simulated) data is presented and discussed.