Asymptotic and Bootstrap Inference for AR(∞) Processes with Conditional Heteroskedasticity

The main contribution of this paper is a proof of the asymptotic validity of the application of the bootstrap to AR(∞) processes with unmodelled conditional heteroskedasticity. We first derive the asymptotic properties of the least-squares estimator of the autoregressive sieve parameters when the data are generated by a stationary linear process with martingale difference errors that are possibly subject to conditional heteroskedasticity of unknown form. These results are then used in establishing that a suitably constructed bootstrap estimator will have the same limit distribution as the least-squares estimator. Our results provide theoretical justification for the use of either the conventional asymptotic approximation based on robust standard errors or the bootstrap approximation of the distribution of autoregressive parameters. A simulation study suggests that the bootstrap approach tends to be more accurate in small samples.     

Asymptotic and Bootstrap Inference for AR(∞) Processes with Conditional Heteroskedasticity

The main contribution of this paper is a proof of the asymptotic validity of the application of the bootstrap to AR(∞) processes with unmodelled conditional heteroskedasticity. We first derive the asymptotic properties of the least-squares estimator of the autoregressive sieve parameters when the data are generated by a stationary linear process with martingale difference errors that are possibly subject to conditional heteroskedasticity of unknown form. These results are then used in establishing that a suitably constructed bootstrap estimator will have the same limit distribution as the least-squares estimator. Our results provide theoretical justification for the use of either the conventional asymptotic approximation based on robust standard errors or the bootstrap approximation of the distribution of autoregressive parameters. A simulation study suggests that the bootstrap approach tends to be more accurate in small samples.     

Valid Resampling of Higher-Order Statistics Using the Linear Process Bootstrap and Autoregressive Sieve Bootstrap

We show that the linear process bootstrap (LPB) and the autoregressive sieve bootstrap (AR sieve) are, in general, not valid for statistics whose large-sample distribution depends on moments of order higher than two, irrespective of whether the data come from a linear time series or not. Inspired by the block-of-blocks bootstrap, we circumvent this non-validity by applying the LPB and AR sieve to suitably blocked data and not to the original data itself. In a simulation study, we compare the LPB, AR sieve, and moving block bootstrap applied directly and to blocked data.     

The sample autocorrelation function of I(1) processes

In case of a random walk the theoretical autocorrelations tend to one asymptotically. The sample autocorrelations, however, may decline rather fast even with large samples. We will explain this observation by deriving the asymptotic distribution that turns out to be closely related to the Dickey-Fuller (1979) distribution. Moreover we discuss the behaviour of the sample autocorrelations of integrated MA(1) and AR(1) processes. In order to prove our results we consider more general I(1) processes and apply the functional central limit theorem injected to time series analysis by Phillips (1987). We obtain unit root tests that are based on autocorrelation estimators of higher lags. We discuss their finite sample behaviour experimentally.     

Penalized regression models with autoregressive error terms

Penalized regression methods have recently gained enormous attention in statistics and the field of machine learning due to their ability of reducing the prediction error and identifying important variables at the same time. Numerous studies have been conducted for penalized regression, but most of them are limited to the case when the data are independently observed. In this paper, we study a variable selection problem in penalized regression models with autoregressive (AR) error terms. We consider three estimators, adaptive least absolute shrinkage and selection operator, bridge, and smoothly clipped absolute deviation, and propose a computational algorithm that enables us to select a relevant set of variables and also the order of AR error terms simultaneously. In addition, we provide their asymptotic properties such as consistency, selection consistency, and asymptotic normality. The performances of the three estimators are compared with one another using simulated and real examples.     

Efficiency and Validity Analyses of Two-Stage Estimation Procedures and Derived Testing Procedures in Quantitative Linear Models with AR(1) Errors

Abstract

In a quantitative linear model with errors following a stationary Gaussian, first-order autoregressive or AR(1) process, Generalized Least Squares (GLS) on raw data and Ordinary Least Squares (OLS) on prewhitened data are efficient methods of estimation of the slope parameters when the autocorrelation parameter of the error AR(1) process, ρ, is known. In practice, ρ is generally unknown. In the so-called two-stage estimation procedures, ρ is then estimated first before using the estimate of ρ to transform the data and estimate the slope parameters by OLS on the transformed data. Different estimators of ρ have been considered in previous studies. In this article, we study nine two-stage estimation procedures for their efficiency in estimating the slope parameters. Six of them (i.e., three noniterative, three iterative) are based on three estimators of ρ that have been considered previously. Two more (i.e., one noniterative, one iterative) are based on a new estimator of ρ that we propose: it is provided by the sample autocorrelation coefficient of the OLS residuals at lag 1, denoted r(1). Lastly, REstricted Maximum Likelihood (REML) represents a different type of two-stage estimation procedure whose efficiency has not been compared to the others yet. We also study the validity of the testing procedures derived from GLS and the nine two-stage estimation procedures. Efficiency and validity are analyzed in a Monte Carlo study. Three types of explanatory variable x in a simple quantitative linear model with AR(1) errors are considered in the time domain: Case 1, x is fixed; Case 2, x is purely random; and Case 3, x follows an AR(1) process with the same autocorrelation parameter value as the error AR(1) process. In a preliminary step, the number of inadmissible estimates and the efficiency of the different estimators of ρ are compared empirically, whereas their approximate expected value in finite samples and their asymptotic variance are derived theoretically. Thereafter, the efficiency of the estimation procedures and the validity of the derived testing procedures are discussed in terms of the sample size and the magnitude and sign of ρ. The noniterative two-stage estimation procedure based on the new estimator of ρ is shown to be more efficient for moderate values of ρ at small sample sizes. With the exception of small sample sizes, REML and its derived F-test perform the best overall. The asymptotic equivalence of two-stage estimation procedures, besides REML, is observed empirically. Differences related to the nature, fixed or random (uncorrelated or autocorrelated), of the explanatory variable are also discussed.     

Quasi-maximum likelihood estimation and inference in dynamic models with time-varying covariances

We study the properties of the quasi-maximum likelihood estimator (QMLE) and related test statistics in dynamic models that jointly parameterize conditional means and conditional covariances, when a normal log-likelihood os maximized but the assumption of normality is violated. Because the score of the normal log-likelihood has the martingale difference property when the forst two conditional moments are correctly specified, the QMLE is generally Consistent and has a limiting normal destribution. We provide easily computable formulas for asymptotic standard errors that are valid under nonnormality. Further, we show how robust LM tests for the adequacy of the jointly parameterized mean and variance can be computed from simple auxiliary regressions. An appealing feature of these robyst inference procedures is that only first derivatives of the conditional mean and variance functions are needed. A monte Carlo study indicates that the asymptotic results carry over to finite samples. Estimation of several AR and AR-GARCH time series models reveals that in most sotuations the robust test statistics compare favorably to the two standard (nonrobust) formulations of the Wald and IM tests. Also, for the GARCH models and the sample sizes analyzed here, the bias in the QMLE appears to be relatively small. An empirical application to stock return volatility illustrates the potential imprtance of computing robust statistics in practice.     

Quasi-maximum likelihood estimation and inference in dynamic models with time-varying covariances

We study the properties of the quasi-maximum likelihood estimator (QMLE) and related test statistics in dynamic models that jointly parameterize conditional means and conditional covariances, when a normal log-likelihood os maximized but the assumption of normality is violated. Because the score of the normal log-likelihood has the martingale difference property when the forst two conditional moments are correctly specified, the QMLE is generally Consistent and has a limiting normal destribution. We provide easily computable formulas for asymptotic standard errors that are valid under nonnormality. Further, we show how robust LM tests for the adequacy of the jointly parameterized mean and variance can be computed from simple auxiliary regressions. An appealing feature of these robyst inference procedures is that only first derivatives of the conditional mean and variance functions are needed. A monte Carlo study indicates that the asymptotic results carry over to finite samples. Estimation of several AR and AR-GARCH time series models reveals that in most sotuations the robust test statistics compare favorably to the two standard (nonrobust) formulations of the Wald and IM tests. Also, for the GARCH models and the sample sizes analyzed here, the bias in the QMLE appears to be relatively small. An empirical application to stock return volatility illustrates the potential imprtance of computing robust statistics in practice.     

Estimating nonlinear additive models with nonstationarities and correlated errors

In this paper, we study a nonparametric additive regression model suitable for a wide range of time series applications. Our model includes a periodic component, a deterministic time trend, various component functions of stochastic explanatory variables, and an AR(p) error process that accounts for serial correlation in the regression error. We propose an estimation procedure for the nonparametric component functions and the parameters of the error process based on smooth backfitting and quasimaximum likelihood methods. Our theory establishes convergence rates and the asymptotic normality of our estimators. Moreover, we are able to derive an oracle‐type result for the estimators of the AR parameters: Under fairly mild conditions, the limiting distribution of our parameter estimators is the same as when the nonparametric component functions are known. Finally, we illustrate our estimation procedure by applying it to a sample of climate and ozone data collected on the Antarctic Peninsula.     

On Sample Skewness and Kurtosis

It is well documented in the literature that the sample skewness and excess kurtosis can be severely biased in finite samples. In this paper, we derive analytical results for their finite-sample biases up to the second order. In general, the bias results depend on the cumulants (up to the sixth order) as well as the dependency structure of the data. Using an AR(1) process for illustration, we show that a feasible bias-correction procedure based on our analytical results works remarkably well for reducing the bias of the sample skewness. Bias-correction works reasonably well also for the sample kurtosis under some moderate degree of dependency. In terms of hypothesis testing, bias-correction offers power improvement when testing for normality, and bias-correction under the null provides also size improvement. However, for testing nonzero skewness and/or excess kurtosis, there exist nonnegligible size distortions in finite samples and bias-correction may not help.     

Binomial AR(1) processes: moments,cumulants, and estimation

The modelling and analysis of count-data time series are areas of emerging interest with various applications in practice. We consider the particular case of the binomial AR(1) model, which is well suited for describing binomial counts with a first-order autoregressive serial dependence structure. We derive explicit expressions for the joint (central) moments and cumulants up to order 4. Then, we apply these results for expressing moments and asymptotic distribution of the squared difference estimator as an alternative to the sample autocovariance. We also analyse the asymptotic distribution of the conditional least-squares estimators of the parameters of the binomial AR(1) model. The finite-sample performance of these estimators is investigated in a simulation study, and we apply them to real data about computerized workstations.     

Time series analysis of categorical data using auto-odds ratio function

In this paper, we consider the auto-odds ratio function (AORF) as a measure of serial association for a stationary time series process of categorical data at two different time points. Numerical measures such as the autocorrelation function (ACF) have no meaningful interpretation, unless the time series data are numerical. Instead, we use the AORF as a measure of association to study the serial dependency of the categorical time series for both ordinal and nominal categories. Biswas and Song [Discrete-valued ARMA processes. Stat Probab Lett. 2009;79(17):1884–1889] provided some results on this measure for Pegram's operator-based AR(1) process with binary responses. Here, we extend this measure to more general set-ups, i.e. for AR(p) and MA(q) processes and for a general number of categories. We discuss how this method can effectively be used in parameter estimation and model selection. Following Weiß [Empirical measures of signed serial dependence in categorical time series. J Stat Comput Simul. 2011;81(4):411–429], we derive the large sample distribution of the estimator of the AORF under independent and identically distributed (iid) set-up. Some simulation results and two categorical data examples (one is ordinal and other nominal) are presented to illustrate the proposed method.     

Asymmetric autoregressive models: statistical aspects and a financial application under COVID-19 pandemic

In the present study, we provide a motivating example with a financial application under COVID-19 pandemic to investigate autoregressive (AR) modeling and its diagnostics based on asymmetric distributions. The objectives of this work are: (i) to formulate asymmetric AR models and their estimation and diagnostics; (ii) to assess the performance of the parameters estimators and of the local influence technique for these models; and (iii) to provide a tool to show how data following an asymmetric distribution under an AR structure should be analyzed. We take the advantages of the stochastic representation of the skew-normal distribution to estimate the parameters of the corresponding AR model efficiently with the expectation-maximization algorithm. Diagnostic analytics are conducted by using the local influence technique with four perturbation schemes. By employing Monte Carlo simulations, we evaluate the statistical behavior of the corresponding estimators and of the local influence technique. An illustration with financial data updated until 2020, analyzed using the methodology introduced in the present work, is presented as an example of effective applications, from where it is possible to explain atypical cases from the COVID-19 pandemic.     

Intrinsic Bayes factor approach to a test for the power law process

The versatile new criterion called the intrinsic Bayes factor (IBF), introduced by Berger and Pericchi [J. Amer. Statist. Assoc. 91 (1996) 109–122], has made it possible to perform model selection and hypotheses testing using standard (improper) noninformative priors in a variety of situations. In this paper, we use their methodology to test several hypotheses regarding the shape parameter of the power law process, which has been widely used to model failure times of repairable systems. Assuming that we have data from the process according to the time-truncation sampling scheme, we derive the arithmetic IBFs using four default priors, including the reference and Jeffreys priors. We establish the frequentist probability matching properties of these priors. We also identify two priors that are justifiable under both time-truncation and failure-truncation schemes, so that the IBFs for both schemes can be unified. Deducing the intrinsic priors of a certain canonical form, as the time of truncation tends to infinity, we show that the arithmetic IBFs correspond asymptotically to actual Bayes factors. We also discuss the expected IBFs, which are useful with small samples. We then use these results to analyze an actual data set on the interruption times of a transmission line, summarizing our results under the default priors.     

Estimation for the three-parameter gamma distribution based on progressively censored data

Some work has been done in the past on the estimation for the three-parameter gamma distribution based on complete and censored samples. In this paper, we develop estimation methods based on progressively Type-II censored samples from a three-parameter gamma distribution. In particular, we develop some iterative methods for the determination of the maximum likelihood estimates (MLEs) of all three parameters. It is shown that the proposed iterative scheme converges to the MLEs. In this context, we propose another method of estimation which is based on missing information principle and moment estimators. Simple alternatives to the above two methods are also suggested. The proposed estimation methods are then illustrated with a numerical example. We also consider the interval estimation based on large-sample theory and examine the actual coverage probabilities of these confidence intervals in case of small samples using a Monte Carlo simulation study.     

Non-penalty shrinkage estimation of random effect models for longitudinal data with AR(1) errors

In this paper, we consider the non-penalty shrinkage estimation method of random effect models with autoregressive errors for longitudinal data when there are many covariates and some of them may not be active for the response variable. In observational studies, subjects are followed over equally or unequally spaced visits to determine the continuous response and whether the response is associated with the risk factors/covariates. Measurements from the same subject are usually more similar to each other and thus are correlated with each other but not with observations of other subjects. To analyse this data, we consider a linear model that contains both random effects across subjects and within-subject errors that follows autoregressive structure of order 1 (AR(1)). Considering the subject-specific random effect as a nuisance parameter, we use two competing models, one includes all the covariates and the other restricts the coefficients based on the auxiliary information. We consider the non-penalty shrinkage estimation strategy that shrinks the unrestricted estimator in the direction of the restricted estimator. We discuss the asymptotic properties of the shrinkage estimators using the notion of asymptotic biases and risks. A Monte Carlo simulation study is conducted to examine the relative performance of the shrinkage estimators with the unrestricted estimator when the shrinkage dimension exceeds two. We also numerically compare the performance of the shrinkage estimators to that of the LASSO estimator. A longitudinal CD4 cell count data set will be used to illustrate the usefulness of shrinkage and LASSO estimators.     

New block bootstrap methods: Sufficient and/or ordered

In this study, we propose sufficient time series bootstrap methods that achieve better results than conventional non-overlapping block bootstrap, but with less computing time and lower standard errors of estimation. Also, we propose using a new technique using ordered bootstrapped blocks, to better preserve the dependency structure of the original data. The performance of the proposed methods are compared in a simulation study for MA(2) and AR(2) processes and in an example. The results show that our methods are good competitors that often exhibit improved performance over the conventional block methods.     

The Stabilisation of Model Parameter Estimates from Repeated Surveys with Rare Observations

In many surveys, the domains of study are small and the samples that carry information on a domain can be very small indeed. If the survey is conducted repeatedly there is often a high degree of overlap in samples over time. We show how to use the richness of information over time to compensate for the paucity of cross‐sectional information. We propose a model‐based estimator of the population total which makes use of stabilised parameter estimates that combine information from different survey periods that are adjacent in time. The motivating example for this research was the ProdCom survey as implemented in the UK.     

Parameter estimation for binomial AR(1) models with applications in finance and industry

Methods for analyzing and modeling count data time series are used in various fields of practice, and they are particularly relevant for applications in finance and economy. We consider the binomial autoregressive (AR(1)) model for count data processes with a first-order AR dependence structure and a binomial marginal distribution. We present four approaches for estimating its model parameters based on given time series data, and we derive expressions for the asymptotic distribution of these estimators. Then we investigate the finite-sample performance of the estimators and of the respective asymptotic approximations in a simulation study, including a discussion of the 2-block jackknife. We illustrate our methods and findings with a real-data example about transactions at the Korea stock market. We conclude with an application of our results for obtaining reliable estimates for process capability indices.     

Robust parameter estimation of regression model with AR(p) error terms

In this article, we consider a linear regression model with AR(p) error terms with the assumption that the error terms have a t distribution as a heavy-tailed alternative to the normal distribution. We obtain the estimators for the model parameters by using the conditional maximum likelihood (CML) method. We conduct an iteratively reweighting algorithm (IRA) to find the estimates for the parameters of interest. We provide a simulation study and three real data examples to illustrate the performance of the proposed robust estimators based on t distribution.