Strong consistency of the regularized least-squares estimates of infinite autoregressive models

Our main interest is on-line parameter estimation of infinite AR models with exponentially decaying coefficients. The practical importance of the problem follows from the fact that the class of such models includes (but not limited to) all causal invertible ARMA(p,q$p,q$) models. On-line parameter estimation means that the length of the observed data sample is not known a priori and may indefinitely increase. Hence, the parameter estimates should be refined upon arrival of every new observation. So use of the maximum likelihood (ML) method is not feasible due to the high computational burden, and recursive estimation procedures are preferable.