Analytical approximations for iterated bootstrap confidence intervals

Standard algorithms for the construction of iterated bootstrap confidence intervals are computationally very demanding, requiring nested levels of bootstrap resampling. We propose an alternative approach to constructing double bootstrap confidence intervals that involves replacing the inner level of resampling by an analytical approximation. This approximation is based on saddlepoint methods and a tail probability approximation of DiCiccio and Martin (1991). Our technique significantly reduces the computational expense of iterated bootstrap calculations. A formal algorithm for the construction of our approximate iterated bootstrap confidence intervals is presented, and some crucial practical issues arising in its implementation are discussed. Our procedure is illustrated in the case of constructing confidence intervals for ratios of means using both real and simulated data. We repeat an experiment of Schenker (1985) involving the construction of bootstrap confidence intervals for a variance and demonstrate that our technique makes feasible the construction of accurate bootstrap confidence intervals in that context. Finally, we investigate the use of our technique in a more complex setting, that of constructing confidence intervals for a correlation coefficient.     

A note on tests for interaction in quantal response data

There are few distribution-free methods for detecting interaction in fixed-dose trials involving quantal response data, despite the fact that such trials are common. We present three new tests to address this issue, including a simple bootstrap procedure. We examine the power of the likelihood ratio test and our new bootstrap test statistic using an innovative linear extrapolation power-estimation technique described in Boos, D. D. and Zhang, J. (2000) in Monte Carlo evaluation of resampling-based hypothesis tests. Journal of the American Statistical Association, 95, 486–492.     

A recentred bootstrap procedure for constructing uniformly correct confidence sets under smooth function models

It has been found, under a smooth function model setting, that the n out of n bootstrap is inconsistent at stationary points of the smooth function, but that the m out of n bootstrap is consistent, provided that a correct convergence rate is specified of the plug-in smooth function estimator. By considering a more general moving-parameter framework, we show that neither of the above bootstrap methods is consistent uniformly over neighbourhoods of stationary points, so that anomalies often arise of coverages of bootstrap sets over certain subsets of parameter values. We propose a recentred bootstrap procedure for constructing confidence sets with uniformly correct coverages over compact sets containing stationary points. A weighted bootstrap procedure is also proposed as an alternative under more general circumstances. Unlike the m out of n bootstrap, both procedures do not require knowledge of the convergence rate of the smooth function estimator. Empirical performance of our procedures is illustrated with numerical examples.     

The estimating function bootstrap

The authors propose a bootstrap procedure which estimates the distribution of an estimating function by resampling its terms using bootstrap techniques. Studentized versions of this so‐called estimating function (EF) bootstrap yield methods which are invariant under reparametrizations. This approach often has substantial advantage, both in computation and accuracy, over more traditional bootstrap methods and it applies to a wide class of practical problems where the data are independent but not necessarily identically distributed. The methods allow for simultaneous estimation of vector parameters and their components. The authors use simulations to compare the EF bootstrap with competing methods in several examples including the common means problem and nonlinear regression. They also prove symptotic results showing that the studentized EF bootstrap yields higher order approximations for the whole vector parameter in a wide class of problems.     

Two‐Sample Bootstrap Hypothesis Tests for Three‐Dimensional Labelled Landmark Data

Abstract. We investigate resampling methodologies for testing the null hypothesis that two samples of labelled landmark data in three dimensions come from populations with a common mean reflection shape or mean reflection size‐and‐shape. The investigation includes comparisons between (i) two different test statistics that are functions of the projection onto tangent space of the data, namely the James statistic and an empirical likelihood statistic; (ii) bootstrap and permutation procedures; and (iii) three methods for resampling under the null hypothesis, namely translating in tangent space, resampling using weights determined by empirical likelihood and using a novel method to transform the original sample entirely within refection shape space. We present results of extensive numerical simulations, on which basis we recommend a bootstrap test procedure that we expect will work well in practise. We demonstrate the procedure using a data set of human faces, to test whether humans in different age groups have a common mean face shape.     

Simultaneous bootstrap confidence bands in regression

We suggest pivotal methods for constructing simultaneous bootstrap confidence bands in regression. Most attention is given to the problem of simple linear regression, but our techniques admit trivial extension to other cases, including polynomial regression. The advantages of our bootstrap approach are twofold. Firstly, the bootstrap allows a very general distribution for the errors, and secondly, it admits a wide variety of shapes for the confidence band. In our technique the shape of each envelope of the band is determined by a general template, chosen by the experimenter, and bootstrap methods are used to select the scale of the template.     

A novel bootstrap procedure for assessing the relationship between class size and achievement

Summary. There is on-going concern about the relationship between class size and achievement for children in their first years of schooling. The Institute of Education's class size project was set up to address this issue and began recruiting in the autumn of 1996. However, because of the non-normality of achievement measures, especially in mathematics, the results have hitherto been presented by using transformed achievement measures. This makes the interpretation difficult for non-statisticians. Ideally, the data would be modelled on the original scale and a bootstrap procedure used to ensure that inferences are robust to non-normality. However, the data are multilevel. In the paper we therefore propose a nonparametric residual bootstrap procedure that is suitable for multilevel models, show that it is consistent and present a simulation study which demonstrates its potential to yield substantial reductions in the difference between nominal and actual confidence interval coverage, compared with a parametric bootstrap, when the underlying distribution of the data is non-normal. We then apply our approach to estimate the relationship between class size and achievement for children in their reception year, after adjusting for other possible determinants.     

Improved estimation procedures for multilevel models with binary response: a case-study

During recent years, analysts have been relying on approximate methods of inference to estimate multilevel models for binary or count data. In an earlier study of random-intercept models for binary outcomes we used simulated data to demonstrate that one such approximation, known as marginal quasi-likelihood, leads to a substantial attenuation bias in the estimates of both fixed and random effects whenever the random effects are non-trivial. In this paper, we fit three-level random-intercept models to actual data for two binary outcomes, to assess whether refined approximation procedures, namely penalized quasi-likelihood and second-order improvements to marginal and penalized quasi-likelihood, also underestimate the underlying parameters. The extent of the bias is assessed by two standards of comparison: exact maximum likelihood estimates, based on a Gauss–Hermite numerical quadrature procedure, and a set of Bayesian estimates, obtained from Gibbs sampling with diffuse priors. We also examine the effectiveness of a parametric bootstrap procedure for reducing the bias. The results indicate that second-order penalized quasi-likelihood estimates provide a considerable improvement over the other approximations, but all the methods of approximate inference result in a substantial underestimation of the fixed and random effects when the random effects are sizable. We also find that the parametric bootstrap method can eliminate the bias but is computationally very intensive.     

Bootstrap prediction regions for multivariate autoregressive processes

Two new methods for improving prediction regions in the context of vector autoregressive (VAR) models are proposed. These methods, which are based on the bootstrap technique, take into account the uncertainty associated with the estimation of the model order and parameters. In particular, by exploiting an independence property of the prediction error, we will introduce a bootstrap procedure that allows for better estimates of the forecasting distribution, in the sense that the variability of its quantile estimators is substantially reduced, without requiring additional bootstrap replications. The proposed methods have a good performance even if the disturbances distribution is not Gaussian. An application to a real data set is presented.     

Bootstrapping an inhomogeneous point process

In this paper, we focus on resampling non-stationary weakly dependent point processes in two dimensions to make inference on the inhomogeneous K function ( Baddeley et al., 2000). We provide theoretical results that show a consistency result of the bootstrap estimates of the variance as the observation region and resampling blocks increase in size. We present results of a simulation study that examines the performance of nominal 95% confidence intervals for the inhomogeneous K function obtained via our bootstrap procedure. The procedure is also applied to a rainforest dataset.     

The Wild Bootstrap for Multilevel Models

In this paper, we study the performance of the most popular bootstrap schemes for multilevel data. Also, we propose a modified version of the wild bootstrap procedure for hierarchical data structures. The wild bootstrap does not require homoscedasticity or assumptions on the distribution of the error processes. Hence, it is a valuable tool for robust inference in a multilevel framework. We assess the finite size performances of the schemes through a Monte Carlo study. The results show that for big sample sizes it always pays off to adopt an agnostic approach as the wild bootstrap outperforms other techniques.     

Confidence Intervals for the Hyperparameters in Structural Models

This article deals with the bootstrap as an alternative method to construct confidence intervals for the hyperparameters of structural models. The bootstrap procedure considered is the classical nonparametric bootstrap in the residuals of the fitted model using a well-known approach. The performance of this procedure is empirically obtained through Monte Carlo simulations implemented in Ox. Asymptotic and percentile bootstrap confidence intervals for the hyperparameters are built and compared by means of the coverage percentages. The results are similar but the bootstrap procedure is better for small sample sizes. The methods are applied to a real time series and confidence intervals are built for the hyperparameters.     

ON BOOTSTRAP HYPOTHESIS TESTING

We describe methods for constructing bootstrap hypothesis tests, illustrating our approach using analysis of variance. The importance of pivotalness is discussed. Pivotal statistics usually result in improved accuracy of level. We note that hypothesis tests and confidence intervals call for different methods of resampling, so as to ensure that accurate critical point estimates are obtained in the former case even when data fail to comply with the null hypothesis. Our main points are illustrated by a simulation study and application to three real data sets.     

Random Weighted Bootstrap Method for Recurrent Events with Informative Censoring

Using the data from the AIDS Link to Intravenous Experiences cohort study as an example, an informative censoring model was used to characterize the repeated hospitalization process of a group of patients. Under the informative censoring assumption, the estimators of the baseline rate function and the regression parameters were shown to be related to a latent variable. Hence, it becomes impractical to directly estimate the unknown quantities in the moments of the estimators for the bandwidth selection of a smoothing estimator and the construction of confidence intervals, which are respectively based on the asymptotic mean squared errors and the asymptotic distributions of the estimators. To overcome these difficulties, we develop a random weighted bootstrap procedure to select appropriate bandwidths and to construct approximated confidence intervals. One can see that our method is simple and faster to implement from a practical point of view, and is at least as accurate as other bootstrap methods. In this article, it is shown that the proposed method is useful through the performance of a Monte Carlo simulation. An application of our procedure is also illustrated by a recurrent event sample of intravenous drug users for inpatient cares over time.     

The Finite Sample Performance of Inference Methods for Propensity Score Matching and Weighting Estimators

ABSTRACT

This article investigates the finite sample properties of a range of inference methods for propensity score-based matching and weighting estimators frequently applied to evaluate the average treatment effect on the treated. We analyze both asymptotic approximations and bootstrap methods for computing variances and confidence intervals in our simulation designs, which are based on German register data and U.S. survey data. We vary the design w.r.t. treatment selectivity, effect heterogeneity, share of treated, and sample size. The results suggest that in general, theoretically justified bootstrap procedures (i.e., wild bootstrapping for pair matching and standard bootstrapping for “smoother” treatment effect estimators) dominate the asymptotic approximations in terms of coverage rates for both matching and weighting estimators. Most findings are robust across simulation designs and estimators.     

Accuracy Analysis of the Percentile Method for Estimating Non Normal Manufacturing Quality

We consider asymmetric kernel estimates based on grouped data. We propose an iterated scheme for constructing such an estimator and apply an iterated smoothed bootstrap approach for bandwidth selection. We compare our approach with competing methods in estimating actuarial loss models using both simulations and data studies. The simulation results show that with this new method, the estimated density from grouped data matches the true density more closely than with competing approaches.     

Importance resampling for the smoothed bootstrap

Alternative methods of estimating properties of unknown distributions include the bootstrap and the smoothed bootstrap. In the standard bootstrap setting, Johns (1988) introduced an importance resam¬pling procedure that results in more accurate approximation to the bootstrap estimate of a distribution function or a quantile. With a suitable “exponential tilting” similar to that used by Johns, we derived a smoothed version of importance resampling in the framework of the smoothed bootstrap. Smoothed importance resampling procedures were developed for the estimation of distribution functions of the Studentized mean, the Studentized variance, and the correlation coefficient. Implementation of these procedures are presented via simulation results which concentrate on the problem of estimation of distribution functions of the Studentized mean and Studentized variance for different sample sizes and various pre-specified smoothing bandwidths for the normal data; additional simulations were conducted for the estimation of quantiles of the distribution of the Studentized mean under an optimal smoothing bandwidth when the original data were simulated from three different parent populations: lognormal, t(3) and t(10). These results suggest that in cases where it is advantageous to use the smoothed bootstrap rather than the standard bootstrap, the amount of resampling necessary might be substantially reduced by the use of importance resampling methods and the efficiency gains depend on the bandwidth used in the kernel density estimation.     

Bootstrapping mean‐squared errors of robust small‐area estimators: Application to the method‐of‐payments surveys data

This paper proposes a new bootstrap procedure for mean‐squared errors of robust small‐area estimators. We formally prove the asymptotic validity of the proposed bootstrap method and examine its finite‐sample performance through Monte Carlo simulations. The results show that our procedure performs well and competes with existing ones. We also provide an application to the estimation of the total volume and value of cash, debit card, and credit card transactions in Canada as well as in its provinces and subgroups of households. In particular, we found that there is a significant average annual decline rate of 3.1% in the volume of cash transactions and that this decline is relatively higher among high‐income households living in heavily populated provinces. Our bootstrap estimator also provides indicators of quality useful in selecting the best small‐area predictor among several alternatives in practice.     

The local fractional bootstrap

We introduce a bootstrap procedure for high‐frequency statistics of Brownian semistationary processes. More specifically, we focus on a hypothesis test on the roughness of sample paths of Brownian semistationary processes, which uses an estimator based on a ratio of realized power variations. Our new resampling method, the local fractional bootstrap, relies on simulating an auxiliary fractional Brownian motion that mimics the fine properties of high‐frequency differences of the Brownian semistationary process under the null hypothesis. We prove the first‐order validity of the bootstrap method, and in simulations, we observe that the bootstrap‐based hypothesis test provides considerable finite‐sample improvements over an existing test that is based on a central limit theorem. This is important when studying the roughness properties of time series data. We illustrate this by applying the bootstrap method to two empirical data sets: We assess the roughness of a time series of high‐frequency asset prices and we test the validity of Kolmogorov's scaling law in atmospheric turbulence data.     

Comparing three bootstrap methods for survey data

Various bootstrap methods for variance estimation and confidence intervals in complex survey data, where sampling is done without replacement, have been proposed in the literature. The oldest, and perhaps the most intuitively appealing, is the without-replacement bootstrap (BWO) method proposed by Gross (1980). Unfortunately, the BWO method is only applicable to very simple sampling situations. We first introduce extensions of the BWO method to more complex sampling designs. The performance of the BWO and two other bootstrap methods, the rescaling bootstrap (Rao and Wu 1988) and the mirror-match bootstrap (Sitter 1992), are then compared through a simulation study. Together these three methods encompass the various bootstrap proposals.