Diagnostics for the bootstrap and fast double bootstrap

ABSTRACT

The bootstrap is typically less reliable in the context of time-series models with serial correlation of unknown form than when regularity conditions for the conventional IID bootstrap apply. It is, therefore, useful to have diagnostic techniques capable of evaluating bootstrap performance in specific cases. Those suggested in this paper are closely related to the fast double bootstrap (FDB) and are not computationally intensive. They can also be used to gauge the performance of the FDB itself. Examples of bootstrapping time series are presented, which illustrate the diagnostic procedures, and show how the results can cast light on bootstrap performance.     

A hybrid approach based on saddlepoint and importance sampling methods for bootstrap tail probability estimation

We propose a simple hybrid method which makes use of both saddlepoint and importance sampling techniques to approximate the bootstrap tail probability of an M-estimator. The method does not rely on explicit formula of the Lugannani-Rice type, and is computationally more efficient than both uniform bootstrap sampling and importance resampling suggested in earlier literature. The method is also applied to construct confidence intervals for smooth functions of M-estimands.     

A simple bootstrap bandwidth selector for local polynomial fitting

A new, fully data-driven bandwidth selector with a double smoothing (DS) bias term and a data-driven variance estimator is developed following the bootstrap idea. The data-driven variance estimation does not involve any additional bandwidth selection. The proposed bandwidth selector convergences faster than a plug-in one due to the DS bias estimate, whereas the data-driven variance improves its finite sample performance clearly and makes it stable. Asymptotic results of the proposals are obtained. A comparative simulation study was done to show the overall gains and the gains obtained by improving either the bias term or the variance estimate, respectively. It is shown that the use of a good variance estimator is more important when the sample size is relatively small.     

Asymptotic bootstrap validity for finite markov chains

Two methods of bootstrap, viz., standard, and conditional, are presented for estimating the transition probabilities of a finite state Markov chain. Asymptotic validity of the bootstrap estimates are established for both methods. An applica- tion to a bootstrapped statistic for testing independence is briefly discussed together with some simulation results.     

Importance resampling for the smoothed bootstrap

Alternative methods of estimating properties of unknown distributions include the bootstrap and the smoothed bootstrap. In the standard bootstrap setting, Johns (1988) introduced an importance resam¬pling procedure that results in more accurate approximation to the bootstrap estimate of a distribution function or a quantile. With a suitable “exponential tilting” similar to that used by Johns, we derived a smoothed version of importance resampling in the framework of the smoothed bootstrap. Smoothed importance resampling procedures were developed for the estimation of distribution functions of the Studentized mean, the Studentized variance, and the correlation coefficient. Implementation of these procedures are presented via simulation results which concentrate on the problem of estimation of distribution functions of the Studentized mean and Studentized variance for different sample sizes and various pre-specified smoothing bandwidths for the normal data; additional simulations were conducted for the estimation of quantiles of the distribution of the Studentized mean under an optimal smoothing bandwidth when the original data were simulated from three different parent populations: lognormal, t(3) and t(10). These results suggest that in cases where it is advantageous to use the smoothed bootstrap rather than the standard bootstrap, the amount of resampling necessary might be substantially reduced by the use of importance resampling methods and the efficiency gains depend on the bandwidth used in the kernel density estimation.     

Intentionally biased bootstrap methods

A class of weighted bootstrap techniques, called biased bootstrap or b-bootstrap methods, is introduced. It is motivated by the need to adjust empirical methods, such as the 'uniform' bootstrap, in a surgical way to alter some of their features while leaving others unchanged. Depending on the nature of the adjustment, the b-bootstrap can be used to reduce bias, or to reduce variance or to render some characteristic equal to a predetermined quantity. Examples of the last application include a b-bootstrap approach to hypothesis testing in nonparametric contexts, where the b-bootstrap enables simulation 'under the null hypothesis', even when the hypothesis is false, and a b-bootstrap competitor to Tibshirani's variance stabilization method. An example of the bias reduction application is adjustment of Nadaraya–Watson kernel estimators to make them competitive with local linear smoothing. Other applications include density estimation under constraints, outlier trimming, sensitivity analysis, skewness or kurtosis reduction and shrinkage.     

Smoothed bootstrap bandwidth selection for nonparametric hazard rate estimation

A smoothed bootstrap method is presented for the purpose of bandwidth selection in nonparametric hazard rate estimation for iid data. In this context, two new bootstrap bandwidth selectors are established based on the exact expression of the bootstrap version of the mean integrated squared error of some approximations of the kernel hazard rate estimator. This is very useful since Monte Carlo approximation is no longer needed for the implementation of the two bootstrap selectors. A simulation study is carried out in order to show the empirical performance of the new bootstrap bandwidths and to compare them with other existing selectors. The methods are illustrated by applying them to a diabetes data set.     

Generalised regression estimation via the bootstrap

A generalised regression estimation procedure is proposed that can lead to much improved estimation of population characteristics, such as quantiles, variances and coefficients of variation. The method involves conditioning on the discrepancy between an estimate of an auxiliary parameter and its known population value. The key distributional assumption is joint asymptotic normality of the estimates of the target and auxiliary parameters. This assumption implies that the relationship between the estimated target and the estimated auxiliary parameters is approximately linear with coefficients determined by their asymptotic covariance matrix. The main contribution of this paper is the use of the bootstrap to estimate these coefficients, which avoids the need for parametric distributional assumptions. First‐order correct conditional confidence intervals based on asymptotic normality can be improved upon using quantiles of a conditional double bootstrap approximation to the distribution of the studentised target parameter estimate.     

Mathematical proof of the third order accuracy of the speedy double bootstrap method

Abstract

In our previous research, we proposed a speedy double bootstrap method for assessing the reliability of statistical models with maximum log-likelihood criterion. It can provide 3rd order accurate probabilities. In this study, our focus switches to the mathematical proof. We propose an alternative proof of the third order accuracy in the context of the multivariate normal model. Our proof is based on tube formula differential geometric methodology and an Taylor series approach to the asymptotic analysis of the bootstrap method.     

A weighted bootstrap approach to bootstrap iteration

The operation of resampling from a bootstrap resample, encountered in applications of the double bootstrap, maybe viewed as resampling directly from the sample but using probability weights that are proportional to the numbers of times that sample values appear in the resample. This suggests an approximate approach to double-bootstrap Monte Carlo simulation, where weighted bootstrap methods are used to circumvent much of the labour involved in compounded Monte Carlo approximation. In the case of distribution estimation or, equivalently, confidence interval calibration, the new method may be used to reduce the computational labour. Moreover, the method produces the same order of magnitude of coverage error for confidence intervals, or level error for hypothesis tests, as a full application of the double bootstrap.     

A Monte Carlo comparison of three consistent bootstrap procedures

Since bootstrap samples are simple random samples with replacement from the original sample, the information content of some bootstrap samples can be very low. To avoid this fact, several variants of the classical bootstrap have been proposed. In this paper, we consider two of them: the sequential or Poisson bootstrap and the reduced bootstrap. Both of these, like the ordinary bootstrap, can yield second-order accurate distribution estimators, that is, the three bootstrap procedures are asymptotically equivalent. The question that naturally arises is which of them should be used in a practical situation, in other words, which of them should be used for finite sample sizes. To try to answer this question, we have carried out a simulation study. Although no method was found to exhibit best performance in all the considered situations, some recommendations are given.     

A program to calculate bootstrap confidence intervals for the process capability index Cpk

Franklin and Wasserman (1991) introduced the use of Bootstrap sampling procedures for deriving nonparametric confidence intervals for the process capability index, C_pk, which are applicable for instances when at least twenty data points are available. This represents a significant reduction in the usually recommended sample requirement of 100 observations (see Gunther 1989). To facilitate and encourage the use of these procedures. a FORTRAN program is provided for computation of confidence intervals for C_pk. Three methods are provided for this calculation including the standard method, the percentile confidence interval, and the biased - corrected percentile confidence interval.     

The use of Smooth Bootstrap Techniques for Estimating the Error Rate of a Prediction Rule

In this paper we present a simulation study for comparing differents methods for estimating the prediction error rate in a discrimination problem. We consider the Cross-validation, Bootstrap and Bayesian Bootstrap methods for such as problem, while also elaborating on both simple and Bayesian Bootstrap methods by smoothing techniques. We observe as the smoothing procedure lead to improvements in the estimation of the true error rate of the discrimination rule, specially in the case of the smooth Bayesian Bootstrap estimator, whose reduction in M.S.E. resulted from the high positive correlation between the true error rate and its estimations based in this method.     

New block bootstrap methods: Sufficient and/or ordered

In this study, we propose sufficient time series bootstrap methods that achieve better results than conventional non-overlapping block bootstrap, but with less computing time and lower standard errors of estimation. Also, we propose using a new technique using ordered bootstrapped blocks, to better preserve the dependency structure of the original data. The performance of the proposed methods are compared in a simulation study for MA(2) and AR(2) processes and in an example. The results show that our methods are good competitors that often exhibit improved performance over the conventional block methods.     

Bootstrapping stochastic regression models under homoskedasticity: wild bootstrap vs. pairs bootstrap

We investigate by simulation how the wild bootstrap and pairs bootstrap perform in t and F tests of regression parameters in the stochastic regression model, where explanatory variables are stochastic and not given and there exists no heteroskedasticity. The wild bootstrap procedure due to Davidson and Flachaire [The wild bootstrap, tamed at last, Working paper, IER#1000, Queen's University, 2001] with restricted residuals works best but its dominance is not strong compared to the result of Flachaire [Bootstrapping heteroskedastic regression models: wild bootstrap vs. pairs bootstrap, Comput. Statist. Data Anal. 49 (2005), pp. 361–376] in the fixed regression model where explanatory variables are fixed and there exists heteroskedasticity.     

Sufficient m-out-of-n (m/n) bootstrap

Traditional resampling methods for estimating sampling distributions sometimes fail, and alternative approaches are then needed. For example, if the classical central limit theorem does not hold and the naïve bootstrap fails, the m/n bootstrap, based on smaller-sized resamples, may be used as an alternative. An alternative to the naïve bootstrap, the sufficient bootstrap, which uses only the distinct observations in a bootstrap sample, is another recently proposed bootstrap approach that has been suggested to reduce the computational burden associated with bootstrapping. It works as long as naïve bootstrap does. However, if the naïve bootstrap fails, so will the sufficient bootstrap. In this paper, we propose combining the sufficient bootstrap with the m/n bootstrap in order to both regain consistent estimation of sampling distributions and to reduce the computational burden of the bootstrap. We obtain necessary and sufficient conditions for asymptotic normality of the proposed method, and propose new values for the resample size m. We compare the proposed method with the naïve bootstrap, the sufficient bootstrap, and the m/n bootstrap by simulation.     

Comparison of bootstrap and generalized bootstrap methods for estimating high quantiles

The generalized bootstrap is a parametric bootstrap method in which the underlying distribution function is estimated by fitting a generalized lambda distribution to the observed data. In this study, the generalized bootstrap is compared with the traditional parametric and non-parametric bootstrap methods in estimating the quantiles at different levels, especially for high quantiles. The performances of the three methods are evaluated in terms of cover rate, average interval width and standard deviation of width of the 95% bootstrap confidence intervals. Simulation results showed that the generalized bootstrap has overall better performance than the non-parametric bootstrap in high quantile estimation.     

Robust bootstrap forecast densities for GARCH returns and volatilities

Bootstrap procedures are useful to obtain forecast densities for both returns and volatilities in the context of generalized autoregressive conditional heteroscedasticity models. In this paper, we analyse the effect of additive outliers on the finite sample properties of these bootstrap densities and show that, when obtained using maximum likelihood estimates of the parameters and standard filters for the volatilities, they are badly affected with dramatic consequences on the estimation of Value-at-Risk. We propose constructing bootstrap densities for returns and volatilities using a robust parameter estimator based on variance targeting implemented together with an adequate modification of the volatility filter. We show that the performance of the proposed procedure is adequate when compared with available robust alternatives. The results are illustrated with both simulated and real data.     

Robustness of residual-based bootstrap to the composition of serially correlated errors

In a simple autoregressive model with serially correlated errors, we evaluate size distortions resulting from the residual bootstrap when the Wold innovation is serially dependent and hence is expected to contaminate the inference. The small distortions caused by the presence of strong conditional heteroskedasticity or other nonlinearities can be partly removed further by using the wild bootstrap.