Bootstrapping a time series model: some empirical results

The bootstrap is a methodology for estimating standard errors. The idea is to use a Monte Carlo simulation experiment based on a nonparametric estimate of the error distribution. The main objective of this article is to demonstrate the use of the bootstrap to attach standard errors to coefficient estimates in a second-order autoregressive model fitted by least squares and maximum likelihood estimation. Additionally, a comparison of the bootstrap and the conventional methodology is made. As it turns out, the conventional asymptotic formulae (both the least squares and maximum likelihood estimates) for estimating standard errors appear to overestimate the true standard errors. But there are two problems:i. The first two observations y₁ and y₂ have been fixed, and ii. The residuals have not been inflated. After these two factors are considered in the trial and bootstrap experiment, both the conventional maximum likelihood and bootstrap estimates of the standard errors appear to be performing quite well.     

Testing the equality of several inverse Gaussian means under heterogeneity

Abstract

We consider the problem of testing the equality of several inverse Gaussian means when the scale parameters and sample sizes are possibly unequal. We propose four parametric bootstrap (PB) tests based on the uniformly minimum variance unbiased estimators of parameters. We also compare our proposed tests with the existing ones via an extensive simulation study in terms of controlling the Type I error rate and power performance. Simulation results show the merits of the PB tests.     

A nonparametric estimator of the number of classes based on a stratified random sample

Under stratified random sampling, we develop a k^th-order bootstrap bias-corrected estimator of the number of classes θ which exist in a study region. This research extends Smith and van Belle’s (1984) first-order bootstrap bias-corrected estimator under simple random sampling. Our estimator has applicability for many settings including: estimating the number of animals when there are stratified capture periods, estimating the number of species based on stratified random sampling of subunits (say, quadrats) from the region, and estimating the number of errors/defects in a product based on observations from two or more types of inspectors. When the differences between the strata are large, utilizing stratified random sampling and our estimator often results in superior performance versus the use of simple random sampling and its bootstrap or jackknife [Burnham and Overton (1978)] estimator. The superior performance is often associated with more observed classes, and we provide insights into optimal designation of the strata and optimal allocation of sample sectors to strata.     

Estimation of Standard Errors of Empirical Bayes Estimators in Capm-Type Models

This paper is concerned with the problem of estimating the standard errors of the empirical Bayes estimators in linear regression models. The problem of deriving an exact expression for the standard error of this estimator is generally intractable. We suggest a procedure based on Efron’s bootstrap method as a way of estimating the standard error. It is shown, through simulations, that the bootstrap method provides a more accurate estimate of the standard error of the empirical Bayes estimator than the traditional large sample method.     

A bootstrap procedure in linear regression with nonstationary errors

In the context of linear regression with dependent and nonstationary errors, the classical moving-block bootstrap (MBB) fails to capture the nonstationarity of the errors. A new bootstrap procedure called the blocking external bootstrap (BEB) is proposed to overcome the problem. The consistency of the BEB in estimating the variance of the least-squares estimator is studied in the case of α-mixing and nonstationary sequence of errors. It is shown that the BEB only achieves partial correction if the block size is fixed. Complete consistency is achieved by the BEB when the block size is allowed to go to infinity. We also study the first-order consistency of the least squares estimator based on the BEB. A simulation study is carried out to assess the performance of the BEB versus the MBB in estimating the variance of the least-squares estimator. Finally, some open problems are discussed.     

Profile likelihood-based confidence interval for the dispersion parameter in count data

The importance of the dispersion parameter in counts occurring in toxicology, biology, clinical medicine, epidemiology, and other similar studies is well known. A couple of procedures for the construction of confidence intervals (CIs) of the dispersion parameter have been investigated, but little attention has been paid to the accuracy of its CIs. In this paper, we introduce the profile likelihood (PL) approach and the hybrid profile variance (HPV) approach for constructing the CIs of the dispersion parameter for counts based on the negative binomial model. The non-parametric bootstrap (NPB) approach based on the maximum likelihood (ML) estimates of the dispersion parameter is also considered. We then compare our proposed approaches with an asymptotic approach based on the ML and the restricted ML (REML) estimates of the dispersion parameter as well as the parametric bootstrap (PB) approach based on the ML estimates of the dispersion parameter. As assessed by Monte Carlo simulations, the PL approach has the best small-sample performance, followed by the REML, HPV, NPB, and PB approaches. Three examples to biological count data are presented.     

Diagnostics for the bootstrap and fast double bootstrap

ABSTRACT

The bootstrap is typically less reliable in the context of time-series models with serial correlation of unknown form than when regularity conditions for the conventional IID bootstrap apply. It is, therefore, useful to have diagnostic techniques capable of evaluating bootstrap performance in specific cases. Those suggested in this paper are closely related to the fast double bootstrap (FDB) and are not computationally intensive. They can also be used to gauge the performance of the FDB itself. Examples of bootstrapping time series are presented, which illustrate the diagnostic procedures, and show how the results can cast light on bootstrap performance.     

Sufficient m-out-of-n (m/n) bootstrap

Traditional resampling methods for estimating sampling distributions sometimes fail, and alternative approaches are then needed. For example, if the classical central limit theorem does not hold and the naïve bootstrap fails, the m/n bootstrap, based on smaller-sized resamples, may be used as an alternative. An alternative to the naïve bootstrap, the sufficient bootstrap, which uses only the distinct observations in a bootstrap sample, is another recently proposed bootstrap approach that has been suggested to reduce the computational burden associated with bootstrapping. It works as long as naïve bootstrap does. However, if the naïve bootstrap fails, so will the sufficient bootstrap. In this paper, we propose combining the sufficient bootstrap with the m/n bootstrap in order to both regain consistent estimation of sampling distributions and to reduce the computational burden of the bootstrap. We obtain necessary and sufficient conditions for asymptotic normality of the proposed method, and propose new values for the resample size m. We compare the proposed method with the naïve bootstrap, the sufficient bootstrap, and the m/n bootstrap by simulation.     

CONDITIONING VS. STANDARDIZATION FOR CONTRASTS ON ERRORS ATTRACTED TO STABLE LAWS

In multiple regression and other settings one encounters the problem of estimating sampling distributions for contrast operations applied to i.i.d. errors. Permutation bootstrap applied to least squares residuals has been proven to consistently estimate conditionalsampling distributions of contrasts, conditional upon order statistics of errors, even for long-tailed error distributions. How does this compare with the unconditional sampling distribution of the contrast when standardizing by the sample s.d. of the errors (or the residuals)? For errors belonging to the domain of attraction of a normal we present a limit theorem proving that these distributions are far closer to one another than they are to the limiting standard normal distribution. For errors attracted to α-stable laws with α ≤ 2 we construct random variables possessing these conditional and unconditional sampling distributions and develop a Poisson representation for their a.s. limit correlation ρ_α. We prove that ρ₂= 1, ρ_α→ 1 for α → 0 + or 2 ?, and ρ_α< 1 a.s. for α < 2.     

Robustness of residual-based bootstrap to the composition of serially correlated errors

In a simple autoregressive model with serially correlated errors, we evaluate size distortions resulting from the residual bootstrap when the Wold innovation is serially dependent and hence is expected to contaminate the inference. The small distortions caused by the presence of strong conditional heteroskedasticity or other nonlinearities can be partly removed further by using the wild bootstrap.     

Comments

In this paper we compare Bartlett-corrected, bootstrap, and fast double bootstrap tests on maximum likelihood estimates of cointegration parameters. The key result is that both the bootstrap and the Bartlett-corrected tests must be based on the unrestricted estimates of the cointegrating vectors: procedures based on the restricted estimates have almost no power. The small sample size bias of the asymptotic test appears so severe as to advise strongly against its use with the sample sizes commonly available; the fast double bootstrap test minimizes size bias, while the Bartlett-corrected test is somehow more powerful.     

Asymptotic properties of the bootstrap unit root test statistic under possibly infinite variance

ABSTRACT

In this article, the unit root test for the AR(1) model is discussed, under the condition that the innovations of the model are in the domain of attraction of the normal law with possibly infinite variances. By using residual bootstrap with sample size m < n (n being the size of the original sample), we bootstrap the least-squares estimator of the autoregressive parameter. Under some mild assumptions, we prove that the null distribution of the unit root test statistic based on the least-square estimator of the autoregressive parameter can be approximated by using residual bootstrap.     

Control variates and importance sampling for efficient bootstrap simulations

Importance sampling and control variates have been used as variance reduction techniques for estimating bootstrap tail quantiles and moments, respectively. We adapt each method to apply to both quantiles and moments, and combine the methods to obtain variance reductions by factors from 4 to 30 in simulation examples.We use two innovations in control variates—interpreting control variates as a re-weighting method, and the implementation of control variates using the saddlepoint; the combination requires only the linear saddlepoint but applies to general statistics, and produces estimates with accuracy of order n ^-1/2 B ^-1, where n is the sample size and B is the bootstrap sample size.We discuss two modifications to classical importance sampling—a weighted average estimate and a mixture design distribution. These modifications make importance sampling robust and allow moments to be estimated from the same bootstrap simulation used to estimate quantiles.     

Parametric bootstrap tests for unbalanced nested designs under heteroscedasticity

In this article, we consider the two-factor unbalanced nested design model without the assumption of equal error variance. For the problem of testing ‘main effects’ of both factors, we propose a parametric bootstrap (PB) approach and compare it with the existing generalized F (GF) test. The Type I error rates of the tests are evaluated using Monte Carlo simulation. Our studies show that the PB test performs better than the GF test. The PB test performs very satisfactorily even for small samples while the GF test exhibit poor Type I error properties when the number of factorial combinations or treatments goes up. It is also noted that the same tests can be used to test the significance of the random effect variance component in a two-factor mixed effects nested model under unequal error variances.     

Uncertainty estimation in heterogeneous capture–recapture count data

The Conway–Maxwell–Poisson estimator is considered in this paper as the population size estimator. The benefit of using the Conway–Maxwell–Poisson distribution is that it includes the Bernoulli, the Geometric and the Poisson distributions as special cases and, furthermore, allows for heterogeneity. Little emphasis is often placed on the variability associated with the population size estimate. This paper provides a deep and extensive comparison of bootstrap methods in the capture–recapture setting. It deals with the classical bootstrap approach using the true population size, the true bootstrap, and the classical bootstrap using the observed sample size, the reduced bootstrap. Furthermore, the imputed bootstrap, as well as approximating forms in terms of standard errors and confidence intervals for the population size, under the Conway–Maxwell–Poisson distribution, have been investigated and discussed. These methods are illustrated in a simulation study and in benchmark real data examples.     

New block bootstrap methods: Sufficient and/or ordered

In this study, we propose sufficient time series bootstrap methods that achieve better results than conventional non-overlapping block bootstrap, but with less computing time and lower standard errors of estimation. Also, we propose using a new technique using ordered bootstrapped blocks, to better preserve the dependency structure of the original data. The performance of the proposed methods are compared in a simulation study for MA(2) and AR(2) processes and in an example. The results show that our methods are good competitors that often exhibit improved performance over the conventional block methods.     

Importance resampling for the smoothed bootstrap

Alternative methods of estimating properties of unknown distributions include the bootstrap and the smoothed bootstrap. In the standard bootstrap setting, Johns (1988) introduced an importance resam¬pling procedure that results in more accurate approximation to the bootstrap estimate of a distribution function or a quantile. With a suitable “exponential tilting” similar to that used by Johns, we derived a smoothed version of importance resampling in the framework of the smoothed bootstrap. Smoothed importance resampling procedures were developed for the estimation of distribution functions of the Studentized mean, the Studentized variance, and the correlation coefficient. Implementation of these procedures are presented via simulation results which concentrate on the problem of estimation of distribution functions of the Studentized mean and Studentized variance for different sample sizes and various pre-specified smoothing bandwidths for the normal data; additional simulations were conducted for the estimation of quantiles of the distribution of the Studentized mean under an optimal smoothing bandwidth when the original data were simulated from three different parent populations: lognormal, t(3) and t(10). These results suggest that in cases where it is advantageous to use the smoothed bootstrap rather than the standard bootstrap, the amount of resampling necessary might be substantially reduced by the use of importance resampling methods and the efficiency gains depend on the bandwidth used in the kernel density estimation.     

Bootstrapping in least absolute value regression: an application to hypothesis testing

A Monte Carlo simulation is used to study the performance of hypothesis tests for regression coefficients when least absolute value regression methods are used. In small samples, the results of the simulation suggest that using the bootstrap method to compute standard errors will provide improved test performance     

Asymptotic bootstrap validity for finite markov chains

Two methods of bootstrap, viz., standard, and conditional, are presented for estimating the transition probabilities of a finite state Markov chain. Asymptotic validity of the bootstrap estimates are established for both methods. An applica- tion to a bootstrapped statistic for testing independence is briefly discussed together with some simulation results.     

SOME PROPERTIES OF PROFILE BOOTSTRAP CONFIDENCE INTERVALS

We consider the problem of finding an equi-tailed confidence interval, with coverage probability (1-α), for a scalar parameter θ₀ in the presence of a (possibly infinite dimensional) nuisance parameter ψ₀. It is supposed that the value taken by θ₀ does not restrict the value that ψ₀ may take and vice-versa. Given a sensible estimate ψ_n of ψ₀, profile bootstrap confidence interval for θ₀ is defined to be the exact equi-tailed confidence interval with coverage probability (1-α) assuming that ψ₀=ψ_n. We compare the properties of the profile bootstrap confidence interval and the ordinary bootstrap confidence interval when they are based on studentised and unstudentised quantities. Under mild regularity conditions the profile bootstrap confidence interval is always a subset of the set of allowable values of θ₀ and is transformation-respecting when based on either an unstudentised quantity or a studentised quantity satisfying certain restrictions. As a confidence interval for the autoregressive parameter of an AR(1) process, the profile bootstrap confidence interval has important advantages over the ordinary bootstrap confidence interval based on a studentised quantity.