Bootstrapping and permuting paired t-test type statistics

We study various bootstrap and permutation methods for matched pairs, whose distributions can have different shapes even under the null hypothesis of no treatment effect. Although the data may not be exchangeable under the null, we investigate different permutation approaches as valid procedures for finite sample sizes. It will be shown that permutation or bootstrap schemes, which neglect the dependency structure in the data, are asymptotically valid. Simulation studies show that these new tests improve the power of the t-test under non-normality.     

Comparing k Cumulative Incidence Functions Through Resampling Methods

Tests for the equality of k cumulative incidence functions in a competing risks model are proposed. Test statistics are based on a vector of processes related to the cumulative incidence functions. Since their asymptotic distributions appear very complicated and depend on the underlying distribution of the data, two resampling techniques, namely the well-known bootstrap method and the so-called random symmetrization method, are used to approximate the critical values of the tests. Without making any assumptions on the nature of dependence between the risks, the tests allow one to compare k risks simultaneously for k 2 under the random censorship model. Tests against ordered alternatives are also considered. Simulation studies indicate that the proposed tests perform very well with moderate sample size. A real application to cancer mortality data is given.     

Multivariate Tests of Mean-Variance Efficiency and Spanning With a Large Number of Assets and Time-Varying Covariances

We develop a finite-sample procedure to test the mean-variance efficiency and spanning hypotheses, without imposing any parametric assumptions on the distribution of model disturbances. In so doing, we provide an exact distribution-free method to test uniform linear restrictions in multivariate linear regression models. The framework allows for unknown forms of nonnormalities as well as time-varying conditional variances and covariances among the model disturbances. We derive exact bounds on the null distribution of joint F statistics to deal with the presence of nuisance parameters, and we show how to implement the resulting generalized nonparametric bounds tests with Monte Carlo resampling techniques. In sharp contrast to the usual tests that are not even computable when the number of test assets is too large, the power of the proposed test procedure potentially increases along both the time and cross-sectional dimensions.     

Multivariate nonparametric methods in two-way balanced designs: performances and limitations in small samples

Investigations of multivariate population are pretty common in applied researches, and the two-way crossed factorial design is a common design used at the exploratory phase in industrial applications. When assumptions such as multivariate normality and covariance homogeneity are violated, the conventional wisdom is to resort to nonparametric tests for hypotheses testing. In this paper we compare the performances, and in particular the power, of some nonparametric and semi-parametric methods that have been developed in recent years. Specifically, we examined resampling methods and robust versions of classical multivariate analysis of variance (MANOVA) tests. In a simulation study, we generate data sets with different configurations of factor''s effect, number of replicates, number of response variables under null hypothesis, and number of response variables under alternative hypothesis. The objective is to elicit practical advice and guides to practitioners regarding the sensitivity of the tests in the various configurations, the tradeoff between power and type I error, the strategic impact of increasing number of response variables, and the favourable performance of one test when the alternative is sparse. A real case study from an industrial engineering experiment in thermoformed packaging production is used to compare and illustrate the application of the various methods.     

On the Usefulness or Lack Thereof of Optimality Criteria for Structural Change Tests

Elliott and Müller (2006) considered the problem of testing for general types of parameter variations, including infrequent breaks. They developed a framework that yields optimal tests, in the sense that they nearly attain some local Gaussian power envelop. The main ingredient in their setup is that the variance of the process generating the changes in the parameters must go to zero at a fast rate. They recommended the so-called qL?L test, a partial sums type test based on the residuals obtained from the restricted model. We show that for breaks that are very small, its power is indeed higher than other tests, including the popular sup-Wald (SW) test. However, the differences are very minor. When the magnitude of change is moderate to large, the power of the test is very low in the context of a regression with lagged dependent variables or when a correction is applied to account for serial correlation in the errors. In many cases, the power goes to zero as the magnitude of change increases. The power of the SW test does not show this non-monotonicity and its power is far superior to the qL?L test when the break is not very small. We claim that the optimality of the qL?L test does not come from the properties of the test statistics but the criterion adopted, which is not useful to analyze structural change tests. Instead, we use fixed-break size asymptotic approximations to assess the relative efficiency or power of the two tests. When doing so, it is shown that the SW test indeed dominates the qL?L test and, in many cases, the latter has zero relative asymptotic efficiency.     

Wild bootstrap seasonal unit root tests for time series with periodic nonstationary volatility

We investigate the behavior of the well-known Hylleberg, Engle, Granger and Yoo (HEGY) regression-based seasonal unit root tests in cases where the driving shocks can display periodic nonstationary volatility and conditional heteroskedasticity. Our set up allows for periodic heteroskedasticity, nonstationary volatility and (seasonal) generalized autoregressive-conditional heteroskedasticity as special cases. We show that the limiting null distributions of the HEGY tests depend, in general, on nuisance parameters which derive from the underlying volatility process. Monte Carlo simulations show that the standard HEGY tests can be substantially oversized in the presence of such effects. As a consequence, we propose wild bootstrap implementations of the HEGY tests. Two possible wild bootstrap resampling schemes are discussed, both of which are shown to deliver asymptotically pivotal inference under our general conditions on the shocks. Simulation evidence is presented which suggests that our proposed bootstrap tests perform well in practice, largely correcting the size problems seen with the standard HEGY tests even under extreme patterns of heteroskedasticity, yet not losing finite sample relative to the standard HEGY tests.     

Evaluation of Linear Trend Tests Using Resampling Techniques

A number of parametric and non-parametric linear trend tests for time series are evaluated in terms of test size and power, using also resampling techniques to form the empirical distribution of the test statistics under the null hypothesis of no linear trend. For resampling, both bootstrap and surrogate data are considered. Monte Carlo simulations were done for several types of residuals (uncorrelated and correlated with normal and nonnormal distributions) and a range of small magnitudes of the trend coefficient. In particular for AR(1) and ARMA(1, 1) residual processes, we investigate the discrimination of strong autocorrelation from linear trend with respect to the sample size. The correct test size is obtained for larger data sizes as autocorrelation increases and only when a randomization test that accounts for autocorrelation is used. The overall results show that the type I and II errors of the trend tests are reduced with the use of resampled data. Following the guidelines suggested by the simulation results, we could find significant linear trend in the data of land air temperature and sea surface temperature.     

Bounding the resampling risk for sequential Monte Carlo implementation of hypothesis tests

Sequential designs can be used to save computation time in implementing Monte Carlo hypothesis tests. The motivation is to stop resampling if the early resamples provide enough information on the significance of the p-value of the original Monte Carlo test. In this paper, we consider a sequential design called the B-value design proposed by Lan and Wittes and construct the sequential design bounding the resampling risk, the probability that the accept/reject decision is different from the decision from complete enumeration. For the B-value design whose exact implementation can be done by using the algorithm proposed in Fay, Kim and Hachey, we first compare the expected resample size for different designs with comparable resampling risk. We show that the B-value design has considerable savings in expected resample size compared to a fixed resample or simple curtailed design, and comparable expected resample size to the iterative push out design of Fay and Follmann. The B-value design is more practical than the iterative push out design in that it is tractable even for small values of resampling risk, which was a challenge with the iterative push out design. We also propose an approximate B-value design that can be constructed without using a specially developed software and provides analytic insights on the choice of parameter values in constructing the exact B-value design.     

Estimating the p-values of robust tests for the linear model

In this paper, we study the estimation of p-values for robust tests for the linear regression model. The asymptotic distribution of these tests has only been studied under the restrictive assumption of errors with known scale or symmetric distribution. Since these robust tests are based on robust regression estimates, Efron's bootstrap (1979) presents a number of problems. In particular, it is computationally very expensive, and it is not resistant to outliers in the data. In other words, the tails of the bootstrap distribution estimates obtained by re-sampling the data may be severely affected by outliers.We show how to adapt the Robust Bootstrap (Ann. Statist 30 (2002) 556; Bootstrapping MM-estimators for linear regression with fixed designs, http://mathstat.carleton.ca/~matias/pubs.html) to this problem. This method is very fast to compute, resistant to outliers in the data, and asymptotically correct under weak regularity assumptions. In this paper, we show that the Robust Bootstrap can be used to obtain asymptotically correct, computationally simple p-value estimates. A simulation study indicates that the tests whose p-values are estimated with the Robust Bootstrap have better finite sample significance levels than those obtained from the asymptotic theory based on the symmetry assumption.Although this paper is focussed on robust scores-type tests (in: Directions in Robust Statistics and Diagnostics, Part I, Springer, New York), our approach can be applied to other robust tests (for example, Wald- and dispersion-type also discussed in Markatou et al., 1991).     

Positive quadrant dependence testing and constrained copula estimation

Positive quadrant dependence is a specific dependence structure that is of practical importance in for example modelling dependencies in insurance and actuarial sciences. This dependence structure imposes a constraint on the copula function. The interest in this paper is to test for positive quadrant dependence. One way to assess the distribution of the test statistics under the null hypothesis of positive quadrant dependence is to resample from a constrained copula. This requires constrained estimation of a copula function. We show that this use of resampling under a constrained copula improves considerably the power performance of existing testing procedures. We propose two resampling procedures, one based on a parametric constrained copula estimation and one relying on nonparametric estimation of a positive quadrant dependence copula, and discuss their properties. The finite‐sample performances of the resulting testing procedures are evaluated via a simulation study that also includes comparisons with existing tests. Finally, a data set of Danish fire insurance claims is tested for positive quadrant dependence. The Canadian Journal of Statistics 41: 36–64; 2013 © 2012 Statistical Society of Canada     

Bootstrap and permutation rank tests for proportional hazards under right censoring

We address the testing problem of proportional hazards in the two-sample survival setting allowing right censoring, i.e., we check whether the famous Cox model is underlying. Although there are many test proposals for this problem, only a few papers suggest how to improve the performance for small sample sizes. In this paper, we do exactly this by carrying out our test as a permutation as well as a wild bootstrap test. The asymptotic properties of our test, namely asymptotic exactness under the null and consistency, can be transferred to both resampling versions. Various simulations for small sample sizes reveal an actual improvement of the empirical size and a reasonable power performance when using the resampling versions. Moreover, the resampling tests perform better than the existing tests of Gill and Schumacher and Grambsch and Therneau . The tests’ practical applicability is illustrated by discussing real data examples.

     

Simulated Power Curves for some Location Tests

In this paper we compare the power properties of some location tests. The most widely used such test is Student's t. Recently bootstrap-based tests have received much attention in the literature. A bootstrap version of the t-test will be included in our comparison. Finally, the nonparametric tests based on the idea of permuting the signs will be represented in our comparison. Again, we will initially concentrate on a version of that test based on the mean. The permutation tests predate the bootstrap by about fourty years. Theoretical results of Pitman (1937) and Bickel & Freedman (1981) show that these three methods are asymptotically equivalent if the underlying distribution is symmetric and has finite second moment. In the modern literature, the use of the nonparametric techniques is advocated on the grounds that the size of the test would be either exact, or more nearly exact. In this paper we report on a simulation study that compares the power curves and we show that it is not necessary to use resampling tests with a statistic based on the mean of the sample.     

Testing the Martingale Hypothesis

We propose new tests of the martingale hypothesis based on generalized versions of the Kolmogorov–Smirnov and Cramér–von Mises tests. The tests are distribution-free and allow for a weak drift in the null model. The methods do not require either smoothing parameters or bootstrap resampling for their implementation and so are well suited to practical work. The article develops limit theory for the tests under the null and shows that the tests are consistent against a wide class of nonlinear, nonmartingale processes. Simulations show that the tests have good finite sample properties in comparison with other tests particularly under conditional heteroscedasticity and mildly explosive alternatives. An empirical application to major exchange rate data finds strong evidence in favor of the martingale hypothesis, confirming much earlier research.     

Goodness-of-fit Tests for Mixed Models

Abstract.  Mixed linear models have become a very useful tool for modelling experiments with dependent observations within subjects, but to establish their appropriateness several assumptions have to be checked. In this paper, we focus on the normality assumptions, using goodness-of-fit tests that make allowance for possible design imbalance. These tests rely on asymptotic results, which are established via empirical process theory. The power of the tests is explored empirically, and examples illustrate some aspects of the usage of the tests.     

A Test for Slope Heterogeneity in Fixed Effects Models

Typical panel data models make use of the assumption that the regression parameters are the same for each individual cross-sectional unit. We propose tests for slope heterogeneity in panel data models. Our tests are based on the conditional Gaussian likelihood function in order to avoid the incidental parameters problem induced by the inclusion of individual fixed effects for each cross-sectional unit. We derive the Conditional Lagrange Multiplier test that is valid in cases where N → ∞ and T is fixed. The test applies to both balanced and unbalanced panels. We expand the test to account for general heteroskedasticity where each cross-sectional unit has its own form of heteroskedasticity. The modification is possible if T is large enough to estimate regression coefficients for each cross-sectional unit by using the MINQUE unbiased estimator for regression variances under heteroskedasticity. All versions of the test have a standard Normal distribution under general assumptions on the error distribution as N → ∞. A Monte Carlo experiment shows that the test has very good size properties under all specifications considered, including heteroskedastic errors. In addition, power of our test is very good relative to existing tests, particularly when T is not large.     

Implementing Monte Carlo tests with p-value buckets

Software packages usually report the results of statistical tests using p-values. Users often interpret these values by comparing them with standard thresholds, for example, 0.1, 1, and 5%, which is sometimes reinforced by a star rating (***, **, and *, respectively). We consider an arbitrary statistical test whose p-value p is not available explicitly, but can be approximated by Monte Carlo samples, for example, by bootstrap or permutation tests. The standard implementation of such tests usually draws a fixed number of samples to approximate p. However, the probability that the exact and the approximated p-value lie on different sides of a threshold (the resampling risk) can be high, particularly for p-values close to a threshold. We present a method to overcome this. We consider a finite set of user-specified intervals that cover [0, 1] and that can be overlapping. We call these p-value buckets. We present algorithms that, with arbitrarily high probability, return a p-value bucket containing p. We prove that for both a bounded resampling risk and a finite runtime, overlapping buckets need to be employed, and that our methods both bound the resampling risk and guarantee a finite runtime for such overlapping buckets. To interpret decisions with overlapping buckets, we propose an extension of the star rating system. We demonstrate that our methods are suitable for use in standard software, including for low p-value thresholds occurring in multiple testing settings, and that they can be computationally more efficient than standard implementations.     

Distribution-free partially sequential piacment procedures

The concept of a partially sequential hypothesis test was introduced by Wolfe (1977a), an{associated procedures were developed for both parametric and nonparametric assumptions. In this paper we consider distribution-free extensions of those indicator tests, based on the placements of the sequentially obtained observations among the previously collected fixed size sample. Exact and asymptotic, as the fixed sample size in¬creases to infinity, properties of these sequential placements procedures are obtained, including statements about the power and expected number of sequentially obtained observations. The results of a Monte Carlo study are used to differentiate be¬tween various placement scoring schemes.     

A Distribution Free Test for the Equality of Scales

Situations where scale parameters are not nuisance factors to be controlled but outcomes to be explained arise in many contexts such as quality control, agricultural production systems, experimental education, the pharmaceutical industry and biology. Tests for homogeneity of variances are often of interest also as a preliminary to analysis of variance, dose-response modelling or discriminant analysis. The literature on tests for the equality of scales is vast. A test which usually stands out in terms of power and robustness against non normality is the modified Levene W50 test, however in the literature no test is found to be the most powerful one for every distribution. The goal of the article is to propose an effective method for comparing scales. More precisely, we propose a test for the equality of scales that, even though was not the most powerful one for every distribution, it has good overall performance under every type of distribution. This test has the form of a combined resampling test. It is important to note that non combined tests show good performance only in particular contexts. Size and power of the proposed test are studied via simulation and compared with many other robust tests for scale. A practical application to industrial quality control is discussed.     

The power of the tests of robinson (1994) in the context of fractionall[y integrated moving average models

We examine in this article the power of the tests of Robinson (1994) for testing I(d) statistical models in the presence of moving average (MA) disturbances. The results show that the tests behave relatively well if we correctly assume that the disturbances are MA. However, assuming white noise or autoregressive disturbances, the power of the tests against one-sided alternatives is very low.     

C194. A nearly flat prior,some unsolved problexs,anb soxe comments about improper priors

Two different approaches to obtaining finite-sample corrections to score tests are the analytical and the computational approaches. The former is based either on a Bartletttype correction to the test statistic or on the inversion of an Edgeworth expansion to its null distribution. The latter, on the other hand, is usually based on a bootstrapping resampling scheme. This paper provides a numerical comparison of the size and power properties of these two approaches both under correct model specification and under model misspecification.