Bootstrapping a time series model: some empirical results

The bootstrap is a methodology for estimating standard errors. The idea is to use a Monte Carlo simulation experiment based on a nonparametric estimate of the error distribution. The main objective of this article is to demonstrate the use of the bootstrap to attach standard errors to coefficient estimates in a second-order autoregressive model fitted by least squares and maximum likelihood estimation. Additionally, a comparison of the bootstrap and the conventional methodology is made. As it turns out, the conventional asymptotic formulae (both the least squares and maximum likelihood estimates) for estimating standard errors appear to overestimate the true standard errors. But there are two problems:i. The first two observations y₁ and y₂ have been fixed, and ii. The residuals have not been inflated. After these two factors are considered in the trial and bootstrap experiment, both the conventional maximum likelihood and bootstrap estimates of the standard errors appear to be performing quite well.     

Adaptive Lasso Variable Selection for the Accelerated Failure Models

This article considers the adaptive lasso procedure for the accelerated failure time model with multiple covariates based on weighted least squares method, which uses Kaplan-Meier weights to account for censoring. The adaptive lasso method can complete the variable selection and model estimation simultaneously. Under some mild conditions, the estimator is shown to have sparse and oracle properties. We use Bayesian Information Criterion (BIC) for tuning parameter selection, and a bootstrap variance approach for standard error. Simulation studies and two real data examples are carried out to investigate the performance of the proposed method.     

Estimation in the simple linear regression model when there is heteroscedasticity of unknown form

A well-known problem is that ordinary least squares estimation of the parameters in the usual linear model can be highly ineficient when the error term has a heavy-tailed distribution. Inefficiency is also associated with situations where the error term is heteroscedastic, and standard confidence intervals can have probability coverage substantially different from the nominal level. This paper compares the small-sample efficiency of six methods that address this problem, three of which model the variance heterogeneity nonparametrically. Three methods were found to be relatively ineffective, but the other three perform relatively well. One of the six (M-regression with a Huber φ function and Schweppe weights) was found to have the highest efficiency for most of the situations considered in the simulations, but there might be situations where one of two other methods gives better results. One of these is a new method that uses a running interval smoother to estimate the optimal weights in weighted least squares, and the other is a method recently proposed by Cohen, Dalal, and Tukey. Computing a confidence interval for the slope using a bootstrap technique is also considered.     

模糊线性回归模型的最小二乘方法

针对自变量和因变量皆模糊的数据系统中的回归分析问题,为避免自变量退化成数值变量时可能引致的估计误差增大而带来的问题,提出系统中引入模糊调整项的回归模型的一般结构,并运用基于模糊数间完备距离的最小二乘法研究模型解析表达式;利用水平截集概念将模糊多元回归模型转化成两个传统回归模型,根据模糊数间距离采用最小二乘法得到参数估计,给出员工工作绩效评估的算例说明方法的有效性,并结合Bootstrap方法的应用,研究回归参数所具有的随机不确定性动态变化。     

Missing data in linear models with correlated errors

One common method for analyzing data in experimental designs when observations are missing was devised by Yates (1933), who developed his procedure based upon a suggestion by R. A. Fisher. Considering a linear model with independent, equi-variate errors, Yates substituted algebraic values for the missing data and then minimized the error sum of squares with respect to both the unknown parameters and the algebraic values. Yates showed that this procedure yielded the correct error sum of squares and a positively biased hypothesis sum of squares.

Others have elaborated on this technique. Chakrabarti (1962) gave a formal proof of Fisher's rule that produced a way to simplify the calculations of the auxiliary values to be used in place of the missing observations. Kshirsagar (1971) proved that the hypothesis sum of squares based on these values was biased, and developed an easy way to compute that bias. Sclove     

Block bootstrap for dependent errors-in-variables

A linear errors-in-variables (EIV) model that contains measurement errors in the input and output data is considered. Weakly dependent (α- and ?-mixing) errors, not necessarily stationary nor identically distributed, are taken into account within the EIV model. Parameters of the EIV model are estimated by the total least squares approach, which provides highly non linear estimates. Because of this, many statistical procedures for constructing confidence intervals and testing hypotheses cannot be applied. One possible solution to this dilemma is a block bootstrap. An appropriate moving block bootstrap procedure is provided and its correctness proved. The results are illustrated through a simulation study and applied on real data as well.     

AMEMIYA'S FORM OF THE WEIGHTED LEAST SQUARES ESTIMATOR

Amemiya's estimator is a weighted least squares estimator of the regression coefficients in a linear model with heteroscedastic errors. It is attractive because the heteroscedasticity is not parametrized and the weights (which depend on the error covariance matrix) are estimated nonparametrically. This paper derives an asymptotic expansion for Amemiya's form of the weighted least squares estimator, and uses it to discuss the effects of estimating the weights, of the number of iterations, and of the choice of the initial estimate. The paper also discusses the special case of normally distributed errors and clarifies the particular consequences of assuming normality.     

Estimation of Standard Errors of Empirical Bayes Estimators in Capm-Type Models

This paper is concerned with the problem of estimating the standard errors of the empirical Bayes estimators in linear regression models. The problem of deriving an exact expression for the standard error of this estimator is generally intractable. We suggest a procedure based on Efron’s bootstrap method as a way of estimating the standard error. It is shown, through simulations, that the bootstrap method provides a more accurate estimate of the standard error of the empirical Bayes estimator than the traditional large sample method.     

Bootstrapping time series models

This paper surveys recent development in bootstrap methods and the modifications needed for their applicability in time series models. The paper discusses some guidelines for empirical researchers in econometric analysis of time series. Different sampling schemes for bootstrap data generation and different forms of bootstrap test statistics are discussed. The paper also discusses the applicability of direct bootstrapping of data in dynamic models and cointegrating regression models. It is argued that bootstrapping residuals is the preferable approach. The bootstrap procedures covered include the recursive bootstrap, the moving block bootstrap and the stationary bootstrap.     

Regression analysis of autocorrelated poisson-distribution) data

A regression model assuming Poisson-dia distributed data. with autocorrelated errors falls into the class of regression models that; have the error structure which is both heteroscedastic and autocorrelated. In general, this class of regression models are not estimable. However, due to the properties of the Poisson distribution that the variance is equal to the mean, this regression model on Poisson-distributed data with autocorrelated. errors is estimable. In this note the special structure of the covarlance matrix of the model with the first order auto-correlated error Is derived utilizing this property, A method based on the least squares method of Frome, Kutner, and Beauchamp (1973), supplemented by steps for handling autocorrelation in studies of time series analysis, nonlinear regression, and econometrics is presented for obtaining generalized least squares estimates for the parameters of the model.     

Use of the Bootstrap and Cross-Validation in Ridge Regression

Several existing methods for the choice of the ridge parameter are reviewed, and a bootstrap method is proposed. The bootstrap provides independent measures of prediction errors based on multiple predictions along with an estimate of the standard error of prediction. The bootstrap and selected competitors are compared through Monte Carlo simulations for various degrees of design matrix collinearity and varying levels of signal-to-noise ratio. The procedure is also illustrated by application to two published data sets. In one case, the bootstrap choice of the ridge parameter leads to a smaller mean squared error of prediction than the ridge trace method. In the second case, an optimal choice of no perturbation is confirmed. Benefits of the bootstrap choice include its less subjective nature, ease of implementation, and robustness.     

Estimation of linear models for field experiments

General mixed linear models for experiments conducted over a series of sltes and/or years are described. The ordinary least squares (OLS) estlmator is simple to compute, but is not the best unbiased estimator. Also, the usuaL formula for the varlance of the OLS estimator is not correct and seriously underestimates the true variance. The best linear unbiased estimator is the generalized least squares (GLS) estimator. However, t requires an inversion of the variance-covariance matrix V, whlch is usually of large dimension. Also, in practice, V is unknown.

We presented an estlmator [Vcirc] of the matrix V using the estimators of variance components [for sites, blocks (sites), etc.]. We also presented a simple transformation of the data, such that an ordinary least squares regression of the transformed data gives the estimated generalized least squares (EGLS) estimator. The standard errors obtained from the transformed regression serve as asymptotic standard errors of the EGLS estimators. We also established that the EGLS estlmator is unbiased.

An example of fitting a linear model to data for 18 sites (environments) located in Brazil is given. One of the site variables (soil test phosphorus) was measured by plot rather than by site and this established the need for a covariance model such as the one used rather than the usual analysis of variance model. It is for this variable that the resulting parameter estimates did not correspond well between the OLS and EGLS estimators. Regression statistics and the analysis of variance for the example are presented and summarized.     

Series Estimation in Partially Linear In‐Slide Regression Models

Abstract. The partially linear in‐slide model (PLIM) is a useful tool to make econometric analyses and to normalize microarray data. In this article, by using series approximations and a least squares procedure, we propose a semiparametric least squares estimator (SLSE) for the parametric component and a series estimator for the non‐parametric component. Under weaker conditions than those imposed in the literature, we show that the SLSE is asymptotically normal and that the series estimator attains the optimal convergence rate of non‐parametric regression. We also investigate the estimating problem of the error variance. In addition, we propose a wild block bootstrap‐based test for the form of the non‐parametric component. Some simulation studies are conducted to illustrate the finite sample performance of the proposed procedure. An example of application on a set of economical data is also illustrated.     

Bootstrap variance and bias estimation in linear models

Let θ be a nonlinear function of the regression parameters and θ be its estimator based on the least-squares method. This paper studies the bootstrap estimators of the variance and bias of θ. The bootstrap estimators are shown to be consistent and asymptotically unbiased under some conditions. Asymptotic orders of the mean squared errors of the bootstrap estimators are also obtained. The bootstrap and the classical linearization method are compared in a simulation study. Discussions about when to use the bootstrap are given.     

Linear Regression With Nested Errors Using Probability‐Linked Data

Probabilistic matching of records is widely used to create linked data sets for use in health science, epidemiological, economic, demographic and sociological research. Clearly, this type of matching can lead to linkage errors, which in turn can lead to bias and increased variability when standard statistical estimation techniques are used with the linked data. In this paper we develop unbiased regression parameter estimates to be used when fitting a linear model with nested errors to probabilistically linked data. Since estimation of variance components is typically an important objective when fitting such a model, we also develop appropriate modifications to standard methods of variance components estimation in order to account for linkage error. In particular, we focus on three widely used methods of variance components estimation: analysis of variance, maximum likelihood and restricted maximum likelihood. Simulation results show that our estimators perform reasonably well when compared to standard estimation methods that ignore linkage errors.     

Statistical inference on partial linear additive models with distortion measurement errors

We consider statistical inference for partial linear additive models (PLAMs) when the linear covariates are measured with errors and distorted by unknown functions of commonly observable confounding variables. A semiparametric profile least squares estimation procedure is proposed to estimate unknown parameter under unrestricted and restricted conditions. Asymptotic properties for the estimators are established. To test a hypothesis on the parametric components, a test statistic based on the difference between the residual sums of squares under the null and alternative hypotheses is proposed, and we further show that its limiting distribution is a weighted sum of independent standard chi-squared distributions. A bootstrap procedure is further proposed to calculate critical values. Simulation studies are conducted to demonstrate the performance of the proposed procedure and a real example is analyzed for an illustration.     

A fast Monte Carlo expectation–maximization algorithm for estimation in latent class model analysis with an application to assess diagnostic accuracy for cervical neoplasia in women with atypical glandular cells

In this article, we use a latent class model (LCM) with prevalence modeled as a function of covariates to assess diagnostic test accuracy in situations where the true disease status is not observed, but observations on three or more conditionally independent diagnostic tests are available. A fast Monte Carlo expectation–maximization (MCEM) algorithm with binary (disease) diagnostic data is implemented to estimate parameters of interest; namely, sensitivity, specificity, and prevalence of the disease as a function of covariates. To obtain standard errors for confidence interval construction of estimated parameters, the missing information principle is applied to adjust information matrix estimates. We compare the adjusted information matrix-based standard error estimates with the bootstrap standard error estimates both obtained using the fast MCEM algorithm through an extensive Monte Carlo study. Simulation demonstrates that the adjusted information matrix approach estimates the standard error similarly with the bootstrap methods under certain scenarios. The bootstrap percentile intervals have satisfactory coverage probabilities. We then apply the LCM analysis to a real data set of 122 subjects from a Gynecologic Oncology Group study of significant cervical lesion diagnosis in women with atypical glandular cells of undetermined significance to compare the diagnostic accuracy of a histology-based evaluation, a carbonic anhydrase-IX biomarker-based test and a human papillomavirus DNA test.     

Measurement error in regression analysis

Consider the linear regression model Y = Xθ+ ε where Y denotes a vector of n observations on the dependent variable, X is a known matrix, θ is a vector of parameters to be estimated and e is a random vector of uncorrelated errors. If X'X is nearly singular, that is if the smallest characteristic root of X'X s small then a small perurbation in the elements of X, such as due to measurement errors, induces considerable variation in the least squares estimate of θ. In this paper we examine for the asymptotic case when n is large the effect of perturbation with regard to the bias and mean squared error of the estimate.     

On the polynomial structural relationship

In estimating a linear measurement error model, extra information is generally needed to identify the model. Here the authors show that the polynomial structural model with errors in the endogenous and exogenous variables can be identified without any extra information if the degree is greater than one. They also show that a weighted least squares approach for the estimation of the parameters in the model leads to the same estimates as the solutions of a system of estimating equations.     

Total least squares and bootstrapping with applications in calibration

The solution to the errors-in-variables problem computed through total least squares is highly nonlinear. Because of this, many statistical procedures for constructing confidence intervals and testing hypotheses cannot be applied. One possible solution to this dilemma is bootstrapping. A nonparametric bootstrap technique could fail. Here, the proper nonparametric bootstrap procedure is provided and its correctness is proved. On the other hand, a residual bootstrap is not valid and suitable in this case. The results are illustrated through a simulation study. An application of this approach to calibration data is presented.