Obtaining prediction intervals for FARIMA processes using the sieve bootstrap

The sieve bootstrap (SB) prediction intervals for invertible autoregressive moving average (ARMA) processes are constructed using resamples of residuals obtained by fitting a finite degree autoregressive approximation to the time series. The advantage of this approach is that it does not require the knowledge of the orders, p and q, associated with the ARMA(p, q) model. Up until recently, the application of this method has been limited to ARMA processes whose autoregressive polynomials do not have fractional unit roots. The authors, in a 2012 publication, introduced a version of the SB suitable for fractionally integrated autoregressive moving average (FARIMA (p,d,q)) processes with 0<d<0.5 and established its asymptotic validity. Herein, we study the finite sample properties this new method and compare its performance against an older method introduced by Bisaglia and Grigoletto in 2001. The sieve bootstrap (SB) method is a numerically simpler alternative to the older method which requires the estimation of p, d, and q at every bootstrap step. Monte-Carlo simulation studies, carried out under the assumption of normal, mixture of normals, and exponential distributions for the innovations, show near nominal coverages for short-term and long-term SB prediction intervals under most situations. In addition, the sieve bootstrap method yields better coverage and narrower intervals compared to the Bisaglia–Grigoletto method in some situations, especially when the error distribution is a mixture of normals.     

Sufficient m-out-of-n (m/n) bootstrap

Traditional resampling methods for estimating sampling distributions sometimes fail, and alternative approaches are then needed. For example, if the classical central limit theorem does not hold and the naïve bootstrap fails, the m/n bootstrap, based on smaller-sized resamples, may be used as an alternative. An alternative to the naïve bootstrap, the sufficient bootstrap, which uses only the distinct observations in a bootstrap sample, is another recently proposed bootstrap approach that has been suggested to reduce the computational burden associated with bootstrapping. It works as long as naïve bootstrap does. However, if the naïve bootstrap fails, so will the sufficient bootstrap. In this paper, we propose combining the sufficient bootstrap with the m/n bootstrap in order to both regain consistent estimation of sampling distributions and to reduce the computational burden of the bootstrap. We obtain necessary and sufficient conditions for asymptotic normality of the proposed method, and propose new values for the resample size m. We compare the proposed method with the naïve bootstrap, the sufficient bootstrap, and the m/n bootstrap by simulation.     

Introducing model uncertainty by moving blocks bootstrap

It is common in parametric bootstrap to select the model from the data, and then treat as if it were the true model. Chatfield (1993, 1996) has shown that ignoring the model uncertainty may seriously undermine the coverage accuracy of prediction intervals. In this paper, we propose a method based on moving block bootstrap for introducing the model selection step in the resampling algorithm. We present a Monte Carlo study comparing the finite sample properties of the proposel method with those of alternative methods in the case of prediction intervas.     

Forecast of the expected non-epidemic morbidity of acute diseases using resampling methods

In epidemiological surveillance it is important that any unusual increase of reported cases be detected as rapidly as possible. Reliable forecasting based on a suitable time series model for an epidemiological indicator is necessary for estimating the expected non-epidemic indicator and to elaborate an alert threshold. Time series analyses of acute diseases often use Gaussian autoregressive integrated moving average models. However, these approaches can be adversely affected by departures from the true underlying distribution. The objective of this paper is to introduce a bootstrap procedure for obtaining prediction intervals in linear models in order to avoid the normality assumption. We present a Monte Carlo study comparing the finite sample properties of bootstrap prediction intervals with those of alternative methods. Finally, we illustrate the performance of the proposed method with a meningococcal disease incidence series.     

Bootstrap for U-statistics: a new approach

Bootstrap for nonlinear statistics like U-statistics of dependent data has been studied by several authors. This is typically done by producing a bootstrap version of the sample and plugging it into the statistic. We suggest an alternative approach of getting a bootstrap version of U-statistics. We will show the consistency of the new method and compare its finite sample properties in a simulation study and by applying both methods to financial data.     

Integration of support vector machines and control charts for multivariate process monitoring

Statistical process control tools have been used routinely to improve process capabilities through reliable on-line monitoring and diagnostic processes. In the present paper, we propose a novel multivariate control chart that integrates a support vector machine (SVM) algorithm, a bootstrap method, and a control chart technique to improve multivariate process monitoring. The proposed chart uses as the monitoring statistic the predicted probability of class (PoC) values from an SVM algorithm. The control limits of SVM-PoC charts are obtained by a bootstrap approach. A simulation study was conducted to evaluate the performance of the proposed SVM–PoC chart and to compare it with other data mining-based control charts and Hotelling's T ² control charts under various scenarios. The results showed that the proposed SVM–PoC charts outperformed other multivariate control charts in nonnormal situations. Further, we developed an exponential weighed moving average version of the SVM–PoC charts for increasing sensitivity to small shifts.     

Weighting methods for ties between event times and covariate change times

Recent work has shown that the presence of ties between an outcome event and the time that a binary covariate changes or jumps can lead to biased estimates of regression coefficients in the Cox proportional hazards model. One proposed solution is the Equally Weighted method. The coefficient estimate of the Equally Weighted method is defined to be the average of the coefficient estimates of the Jump Before Event method and the Jump After Event method, where these two methods assume that the jump always occurs before or after the event time, respectively. In previous work, the bootstrap method was used to estimate the standard error of the Equally Weighted coefficient estimate. However, the bootstrap approach was computationally intensive and resulted in overestimation. In this article, two new methods for the estimation of the Equally Weighted standard error are proposed. Three alternative methods for estimating both the regression coefficient and the corresponding standard error are also proposed. All the proposed methods are easy to implement. The five methods are investigated using a simulation study and are illustrated using two real datasets.     

Resampling Methods for Time Series Level Crossings

We present an application of subsampling and bootstrap methods for time series to determine the distribution of the estimator of zero crossings. The zero crossings method provides an alternative estimator of the lag-1 autocorrelation coefficient that is reducing the data storage requirements and is more robust with respect to outliers when compared to the classical estimator. The main results here are showing the consistency of subsampling, the consistency of moving block bootstrap, the consistency of non overlapping block bootstrap and the consistency of stationary bootstrap for this estimator. Theorems are formulated for Gaussian processes, elliptically symmetric processes and processes which are transformed Gaussian processes. Theoretical results are illustrated by simulations and practical data analysis. We have also shown that in practice the MBB method behaves better than the subsampling method.     

Monte carlo approximation to edgeworth expansions

Since the 1930s, empirical Edgeworth expansions have been employed to develop techniques for approximate, nonparametric statistical inference. The introduction of bootstrap methods has increased the potential usefulness of Edgeworth approximations. In particular, a recent paper by Lee & Young introduced a novel approach to approximating bootstrap distribution functions, using first an empirical Edgeworth expansion and then a more traditional bootstrap approximation to the remainder. In principle, either direct calculation or computer algebra could be used to compute the Edgeworth component, but both methods would often be difficult to implement in practice, not least because of the sheer algebraic complexity of a general Edgeworth expansion. In the present paper we show that a simple but nonstandard Monte Carlo technique is a competitive alternative. It exploits properties of Edgeworth expansions, in particular their parity and the degrees of their polynomial terms, to develop particularly accurate approximations.     

Bootstrap in functional linear regression

We have considered the functional linear model with scalar response and functional explanatory variable. One of the most popular methodologies for estimating the model parameter is based on functional principal components analysis (FPCA). In recent literature, weak convergence for a wide class of FPCA-type estimates has been proved, and consequently asymptotic confidence sets can be built. In this paper, we have proposed an alternative approach in order to obtain pointwise confidence intervals by means of a bootstrap procedure, for which we have obtained its asymptotic validity. Besides, a simulation study allows us to compare the practical behaviour of asymptotic and bootstrap confidence intervals in terms of coverage rates for different sample sizes.     

Mathematical proof of the third order accuracy of the speedy double bootstrap method

Abstract

In our previous research, we proposed a speedy double bootstrap method for assessing the reliability of statistical models with maximum log-likelihood criterion. It can provide 3rd order accurate probabilities. In this study, our focus switches to the mathematical proof. We propose an alternative proof of the third order accuracy in the context of the multivariate normal model. Our proof is based on tube formula differential geometric methodology and an Taylor series approach to the asymptotic analysis of the bootstrap method.     

Confidence Intervals for the Hyperparameters in Structural Models

This article deals with the bootstrap as an alternative method to construct confidence intervals for the hyperparameters of structural models. The bootstrap procedure considered is the classical nonparametric bootstrap in the residuals of the fitted model using a well-known approach. The performance of this procedure is empirically obtained through Monte Carlo simulations implemented in Ox. Asymptotic and percentile bootstrap confidence intervals for the hyperparameters are built and compared by means of the coverage percentages. The results are similar but the bootstrap procedure is better for small sample sizes. The methods are applied to a real time series and confidence intervals are built for the hyperparameters.     

Bootstrap confidence intervals of generalized process capability index Cpyk for Lindley and power Lindley distributions

One of the indicators for evaluating the capability of a process is the process capability index. In this article, bootstrap confidence intervals of the generalized process capability index (GPCI) proposed by Maiti et al. are studied through simulation, when the underlying distributions are Lindley and Power Lindley distributions. The maximum likelihood method is used to estimate the parameters of the models. Three bootstrap confidence intervals namely, standard bootstrap (SB), percentile bootstrap (PB), and bias-corrected percentile bootstrap (BCPB) are considered for obtaining confidence intervals of GPCI. A Monte Carlo simulation has been used to investigate the estimated coverage probabilities and average width of the bootstrap confidence intervals. Simulation results show that the estimated coverage probabilities of the percentile bootstrap confidence interval and the bias-corrected percentile bootstrap confidence interval get closer to the nominal confidence level than those of the standard bootstrap confidence interval. Finally, three real datasets are analyzed for illustrative purposes.     

Comparison of bootstrap and generalized bootstrap methods for estimating high quantiles

The generalized bootstrap is a parametric bootstrap method in which the underlying distribution function is estimated by fitting a generalized lambda distribution to the observed data. In this study, the generalized bootstrap is compared with the traditional parametric and non-parametric bootstrap methods in estimating the quantiles at different levels, especially for high quantiles. The performances of the three methods are evaluated in terms of cover rate, average interval width and standard deviation of width of the 95% bootstrap confidence intervals. Simulation results showed that the generalized bootstrap has overall better performance than the non-parametric bootstrap in high quantile estimation.     

Bootstrapping with auxiliary information

In the nonparametric setting, the standard bootstrap method is based on the empirical distribution function of a random sample. The author proposes, by means of the empirical likelihood technique, an alternative bootstrap procedure under a nonparametric model in which one has some auxiliary information about the population distribution. By proving the almost sure weak convergence of the modified bootstrapped empirical process, the validity of the proposed bootstrap procedure is established. This new result is used to obtain bootstrap confidence bands for the population distribution function and to perform the bootstrap Kolmogorov test in the presence of auxiliary information. Other applications include bootstrapping means and variances with auxiliary information. Three simulation studies are presented to demonstrate the performance of the proposed bootstrap procedure for small samples.     

Bootstrap approaches for estimation and confidence intervals of long memory processes

In this work, we investigate an alternative bootstrap approach based on a result of Ramsey [F.L. Ramsey, Characterization of the partial autocorrelation function, Ann. Statist. 2 (1974), pp. 1296–1301] and on the Durbin–Levinson algorithm to obtain a surrogate series from linear Gaussian processes with long range dependence. We compare this bootstrap method with other existing procedures in a wide Monte Carlo experiment by estimating, parametrically and semi-parametrically, the memory parameter d. We consider Gaussian and non-Gaussian processes to prove the robustness of the method to deviations from normality. The approach is also useful to estimate confidence intervals for the memory parameter d by improving the coverage level of the interval.     

Robust confidence regions for multinomial probabilities

Generally, confidence regions for the probabilities of a multinomial population are constructed based on the Pearson χ² statistic. Morales et al. (Bootstrap confidence regions in multinomial sampling. Appl Math Comput. 2004;155:295–315) considered the bootstrap and asymptotic confidence regions based on a broader family of test statistics known as power-divergence test statistics. In this study, we extend their work and propose penalized power-divergence test statistics-based confidence regions. We only consider small sample sizes where asymptotic properties fail and alternative methods are needed. Both bootstrap and asymptotic confidence regions are constructed. We consider the percentile and the bias corrected and accelerated bootstrap confidence regions. The latter confidence region has not been studied previously for the power-divergence statistics much less for the penalized ones. Designed simulation studies are carried out to calculate average coverage probabilities. Mean absolute deviation between actual and nominal coverage probabilities is used to compare the proposed confidence regions.     

A Bootstrap Likelihood Approach to Bayesian Computation

There is an increasing amount of literature focused on Bayesian computational methods to address problems with intractable likelihood. One approach is a set of algorithms known as Approximate Bayesian Computational (ABC) methods. One of the problems with these algorithms is that their performance depends on the appropriate choice of summary statistics, distance measure and tolerance level. To circumvent this problem, an alternative method based on the empirical likelihood has been introduced. This method can be easily implemented when a set of constraints, related to the moments of the distribution, is specified. However, the choice of the constraints is sometimes challenging. To overcome this difficulty, we propose an alternative method based on a bootstrap likelihood approach. The method is easy to implement and in some cases is actually faster than the other approaches considered. We illustrate the performance of our algorithm with examples from population genetics, time series and stochastic differential equations. We also test the method on a real dataset.     

A note reconciling ANOVA tests under unequal error variances

The purpose of this note is to derive simple testing procedures for ANOVA under heteroscedasticity by a single approach that are equivalent to the prior art in the literature obtained by the Parametric Bootstrap and the Generalized Fiducial approach. By similar approach, researchers are encouraged to derive generalized tests in other applications, as alternative to parametric bootstrap tests and fiducial tests, including ANCOVA and MANOVA under heteroscedasticity, especially in Mixed Model applications, where the bootstrap approach fails.     

Bootstrapping time series models

This paper surveys recent development in bootstrap methods and the modifications needed for their applicability in time series models. The paper discusses some guidelines for empirical researchers in econometric analysis of time series. Different sampling schemes for bootstrap data generation and different forms of bootstrap test statistics are discussed. The paper also discusses the applicability of direct bootstrapping of data in dynamic models and cointegrating regression models. It is argued that bootstrapping residuals is the preferable approach. The bootstrap procedures covered include the recursive bootstrap, the moving block bootstrap and the stationary bootstrap.