Penalized weighted composite quantile regression in the linear regression model with heavy-tailed autocorrelated errors

In this paper, a penalized weighted composite quantile regression estimation procedure is proposed to estimate unknown regression parameters and autoregression coefficients in the linear regression model with heavy-tailed autoregressive errors. Under some conditions, we show that the proposed estimator possesses the oracle properties. In addition, we introduce an iterative algorithm to achieve the proposed optimization problem, and use a data-driven method to choose the tuning parameters. Simulation studies demonstrate that the proposed new estimation method is robust and works much better than the least squares based method when there are outliers in the dataset or the autoregressive error distribution follows heavy-tailed distributions. Moreover, the proposed estimator works comparably to the least squares based estimator when there are no outliers and the error is normal. Finally, we apply the proposed methodology to analyze the electricity demand dataset.     

Robust estimation for the coefficient of a first order autoregressive process

Nonparametric methods, Theil's method and Hussain's method have been applied to simple linear regression problems for estimating the slope of the regression line.We extend these methods and propose a robust estimator to estimate the coefficient of a first order autoregressive process under various distribution shapes, A simulation study to compare Theil's estimator, Hus-sain's estimator, the least squares estimator, and the proposed estimator is also presented.     

A likelihood based estimator for vector autoregressive processes

A onestep estimator, which is an approximation to the unconditional maximum likelihood estimator (MLE) of the coefficient matrices of a Gaussian vector autoregressive process is presented. The onestep estimator is easy to compute and can be computed using standard software. Unlike the computation of the unconditional MLE, the computation of the onestep estimator does not require any iterative optimization and the computation is numerically stable. In finite samples the onestep estimator generally has smaller mean square error than the ordinary least squares estimator. In a simple model, where the unconditional MLE can be computed, numerical investigation shows that the onestep estimator is slightly worse than the unconditional MLE in terms of mean square error but superior to the ordinary least squares estimator. The limiting distribution of the onestep estimator for processes with some unit roots is derived.     

The Empirical Likelihood for First-Order Random Coefficient Integer-Valued Autoregressive Processes

This article studies the empirical likelihood method for the first-order random coefficient integer-valued autoregressive process. The limiting distribution of the log empirical likelihood ratio statistic is established. Confidence region for the parameter of interest and its coverage probabilities are given, and hypothesis testing is considered. The maximum empirical likelihood estimator for the parameter is derived and its asymptotic properties are established. The performances of the estimator are compared with the conditional least squares estimator via simulation.     

On asymptotic properties of bootstrap for AR(1) processes

We consider a first-order autoregressive process when the autoregressive parameter β may vary over the entire real line. The standard bootstrap approximation to the sampling distribution of the least squares estimator of β is shown to converge weakly to a random (i.e., nondegenerate) limit for the usual choice of the bootstrap sample size when β equals 1 or −1. The bootstrap approximation, however, is asymptotically valid in probability, or even almost surely, for suitably selected resample sizes, whatever β may be.     

A new estimator for the unit root

We compare the ordinary least squares, weighted symmetric, modified weighted symmetric (MWS), maximum likelihood, and our new modification for least squares (MLS) estimator for first-order autoregressive in the case of unit root using Monte Carlo method. The Monte Carlo study sheds some light on how well the estimators and the predictors perform on different samples sizes. We found that MLS estimator is less biased and has less mean squared error (MSE) than any other estimators, and MWS predictor error performs well, in the sense of MSE, than any other predictors’ methods. The sample percentiles for the distribution of the τ statistic for the first, second, and third periods in the future, for alternative estimators, are reported to know if it agrees with those of normal distribution or not.     

AN EXAMINATION OF ESTIMATED RESIDUALS IN A REGRESSION WITH AN INFINITE ORDER PARAMETRIC MODEL

We consider a linear regression with the error term that obeys an autoregressive model of infinite order and estimate parameters of the models. The parameters of the autoregressive model should be estimated based on estimated residuals obtained by means of the method of ordinary least squares, because the errors are unobservable. The consistency of the coefficients, variance and spectral density of the model obeyed by the error term is shown. Further, we estimate the coefficients of the linear regression by means of the method of estimated generalized least squares. We also show the consistency of the estimator.

     

Nonlinear Autoregression with Positive Innovations

Use of nonlinear models in analyzing time series data is becoming increasingly popular. This paper considers a broad class of nonlinear autoregressive models where the autoregressive part is additive and the terms are nonlinear functions of the past data. Also, the innovation distribution is supported on the non-negative reals and satisfies a tail regularity condition. The linear parameters of the autoregression are estimated using a linear programming recipe which yields much more accurate estimates than traditional methods such as conditional least squares. Limiting distribution of the linear programming estimators is obtained. Simulation studies validate the asymptotic results and reveal excellent small sample properties of the LPE estimator.     

Adaptive LASSO estimation for ARDL models with GARCH innovations

ABSTRACT

In this paper, we show the validity of the adaptive least absolute shrinkage and selection operator (LASSO) procedure in estimating stationary autoregressive distributed lag(p,q) models with innovations in a broad class of conditionally heteroskedastic models. We show that the adaptive LASSO selects the relevant variables with probability converging to one and that the estimator is oracle efficient, meaning that its distribution converges to the same distribution of the oracle-assisted least squares, i.e., the least square estimator calculated as if we knew the set of relevant variables beforehand. Finally, we show that the LASSO estimator can be used to construct the initial weights. The performance of the method in finite samples is illustrated using Monte Carlo simulation.     

Relative efficiency of first difference estimator in panel data regression with serially correlated error components

We consider the pooled cross-sectional and time series regression model when the disturbances follow a serially correlated one-way error components. In this context we discovered that the first difference estimator for the regression coefficients is equivalent to the generalized least squares estimator irrespective of the particular form of the regressor matrix when the disturbances are generated by a first order autoregressive process where the autocorrelation is close to unity.     

Mildly explosive autoregression with mixing innovations

In this paper, the limit distribution of the least squares estimator for mildly explosive autoregressive models with strong mixing innovations is established, which is shown to be Cauchy as in the iid case. The result is applied to identify the onset and the end of an explosive period of an econometric time series. Simulations and data analysis are also conducted to demonstrate the usefulness of the result.     

Empirical study of robust estimation methods for PAR models with application to the air quality area

Abstract

This paper compares three estimators for periodic autoregressive (PAR) models. The first is the classical periodic Yule-Walker estimator (YWE). The second is a robust version of YWE (RYWE) which uses the robust autocovariance function in the periodic Yule-Walker equations, and the third is the robust least squares estimator (RLSE) based on iterative least squares with robust versions of the original time series. The daily mean particulate matter concentration (PM10) data is used to illustrate the methodologies in a real application, that is, in the Air Quality area.     

Parameter estimation for generalized random coefficient autoregressive processes

A generalized random coefficient autoregressive (GRCA) process is introduced in which the random coefficients are permitted to be correlated with the error process. The ordinary random coefficient autoregressive process, the Markovian bilinear model and its generalization, and the random coefficient exponential autoregressive process, among others, are seen to be special cases of the GRCA process. Conditional least squares, and weighted least-squares estimators of the mean of the random coefficient vector are derived and their limit distributions are studied. Estimators of the variance-covariance parameters are also discussed. A simulation study is presented which shows that the weighted least-squares estimator dominates the unweighted least-squares estimator.     

Adaptive estimation of vector autoregressive models with time-varying variance: Application to testing linear causality in mean

Linear vector autoregressive (VAR) models where the innovations could be unconditionally heteroscedastic are considered. The volatility structure is deterministic and quite general, including breaks or trending variances as special cases. In this framework we propose ordinary least squares (OLS), generalized least squares (GLS) and adaptive least squares (ALS) procedures. The GLS estimator requires the knowledge of the time-varying variance structure while in the ALS approach the unknown variance is estimated by kernel smoothing with the outer product of the OLS residual vectors. Different bandwidths for the different cells of the time-varying variance matrix are also allowed. We derive the asymptotic distribution of the proposed estimators for the VAR model coefficients and compare their properties. In particular we show that the ALS estimator is asymptotically equivalent to the infeasible GLS estimator. This asymptotic equivalence is obtained uniformly with respect to the bandwidth(s) in a given range and hence justifies data-driven bandwidth rules. Using these results we build Wald tests for the linear Granger causality in mean which are adapted to VAR processes driven by errors with a nonstationary volatility. It is also shown that the commonly used standard Wald test for the linear Granger causality in mean is potentially unreliable in our framework (incorrect level and lower asymptotic power). Monte Carlo experiments illustrate the use of the different estimation approaches for the analysis of VAR models with time-varying variance innovations.     

Partially adaptive estimation of nonlinear models via a normal mixture

This paper extends the partially adaptive method Phillips (1994) provided for linear models to nonlinear models. Asymptotic results are established under conditions general enough they cover both cross-sectional and time series applications. The sampling efficiency of the new estimator is illustrated in a small Monte Carlo study in which the parameters of an autoregressive moving average are estimated. The study indicates that, for non-normal distributions, the new estimator improves on the nonlinear least squares estimator in terms of efficiency.     

Partially adaptive estimation of nonlinear models via a normal mixture

This paper extends the partially adaptive method Phillips (1994) provided for linear models to nonlinear models. Asymptotic results are established under conditions general enough they cover both cross-sectional and time series applications. The sampling efficiency of the new estimator is illustrated in a small Monte Carlo study in which the parameters of an autoregressive moving average are estimated. The study indicates that, for non-normal distributions, the new estimator improves on the nonlinear least squares estimator in terms of efficiency.     

Asymptotics of the weighted least squares estimation for AR(1) processes with applications to confidence intervals

For the first-order autoregressive model, we establish the asymptotic theory of the weighted least squares estimations whether the underlying autoregressive process is stationary, unit root, near integrated or even explosive under a weaker moment condition of innovations. The asymptotic limit of this estimator is always normal. It is shown that the empirical log-likelihood ratio at the true parameter converges to the standard chi-square distribution. An empirical likelihood confidence interval is proposed for interval estimations of the autoregressive coefficient. The results improve the corresponding ones of Chan et al. (Econ Theory 28:705–717, 2012). Some simulations are conducted to illustrate the proposed method.     

DISTRIBUTION OF THE LEAST SQUARES ESTIMATOR IN A FIRST-ORDER AUTOREGRESSIVE MODEL

This paper investigates the finite sample distribution of the least squares estimator of the autoregressive parameter in a first-order autoregressive model. A uniform asymptotic expansion for the distribution applicable to both stationary and nonstationary cases is obtained. Accuracy of the approximation to the distribution by a first few terms of this expansion is then investigated. It is found that the leading term of this expansion approximates well the distribution. The approximation is, in almost all cases, accurate to the second decimal place throughout the distribution. In the literature, there exist a number of approximations to this distribution which are specifically designed to apply in some special cases of this model. The present approximation compares favorably with those approximations and in fact, its accuracy is, with almost no exception, as good as or better than these other approximations. Convenience of numerical computations seems also to favor the present approximations over the others. An application of the finding is illustrated with examples.     

A note on nonlinear regression for the autoregressive moving average with non-hd errors

Estimation by nonlinear regression of the parameters for the stationary and invertible autoregressive moving average (ARMA) model with mixing or martingale difference errors is considered. Simple proofs of consistency and asymptotic normality for the nonlinear least squares estimator are given by exploiting results from nonlinear estimation theory and mixing and mixingale theory.     

Monotony relations between distribution functions and their application to experimental design

In a linear regression model an estimator of the unknown coefficients is considered which, in special cases, includes the least squares estimator. In the ease of stable symmetric error distribution and by means of a certain monotony relation between distribution functions optimality of this estimator is proved and the designing problem is investigated. A robustness property of optimal designs against the designing criterion and some conclusions are given concerning the least squares estimator in the case of G- and C-optimality.