Empirical likelihood estimation for linear regression models with AR(p) error terms with numerical examples

Linear regression models are useful statistical tools to analyze data sets in different fields. There are several methods to estimate the parameters of a linear regression model. These methods usually perform under normally distributed and uncorrelated errors. If error terms are correlated the Conditional Maximum Likelihood (CML) estimation method under normality assumption is often used to estimate the parameters of interest. The CML estimation method is required a distributional assumption on error terms. However, in practice, such distributional assumptions on error terms may not be plausible. In this paper, we propose to estimate the parameters of a linear regression model with autoregressive error term using Empirical Likelihood (EL) method, which is a distribution free estimation method. A small simulation study is provided to evaluate the performance of the proposed estimation method over the CML method. The results of the simulation study show that the proposed estimators based on EL method are remarkably better than the estimators obtained from CML method in terms of mean squared errors (MSE) and bias in almost all the simulation configurations. These findings are also confirmed by the results of the numerical and real data examples.     

On the efficiency of regression analysis with AR(p) errors

In this paper we will consider a linear regression model with the sequence of error terms following an autoregressive stationary process. The statistical properties of the maximum likelihood and least squares estimators of the regression parameters will be summarized. Then, it will be proved that, for some typical cases of the design matrix, both methods produce asymptotically equivalent estimators. These estimators are also asymptotically efficient. Such cases include the most commonly used models to describe trend and seasonality like polynomial trends, dummy variables and trigonometric polynomials. Further, a very convenient asymptotic formula for the covariance matrix will be derived. It will be illustrated through a brief simulation study that, for the simple linear trend model, the result applies even for sample sizes as small as 20.     

Robust estimation for ordinal regression

Ordinal regression is used for modelling an ordinal response variable as a function of some explanatory variables. The classical technique for estimating the unknown parameters of this model is Maximum Likelihood (ML). The lack of robustness of this estimator is formally shown by deriving its breakdown point and its influence function. To robustify the procedure, a weighting step is added to the Maximum Likelihood estimator, yielding an estimator with bounded influence function. We also show that the loss in efficiency due to the weighting step remains limited. A diagnostic plot based on the Weighted Maximum Likelihood estimator allows to detect outliers of different types in a single plot.     

A simple recursive estimation method for linear regression models with AR(p) disturbances

An exact transformation that reduces the AR(p) process into white noise is well known in statistics, see Fuller (1976). However, practitioners still use and econometric textbooks still recommend the Cochrane-Orcutt procedure for p>2, see Greene (1990). This paper derives an alternative exact transformation for the AR(p) process which is computationally simple. Based on this transformation, a GLS estimator is proposed, requiring only least squares regressions and recursive computations. This is illustrated for the AR(3) case.     

Nonlinear semiparametric AR(1) model with skew-symmetric innovations

In this paper, we expand a first-order nonlinear autoregressive (AR) model with skew normal innovations. A semiparametric method is proposed to estimate a nonlinear part of model by using the conditional least squares method for parametric estimation and the nonparametric kernel approach for the AR adjustment estimation. Then computational techniques for parameter estimation are carried out by the maximum likelihood (ML) approach using Expectation-Maximization (EM) type optimization and the explicit iterative form for the ML estimators are obtained. Furthermore, in a simulation study and a real application, the accuracy of the proposed methods is verified.     

Bayesian estimation of an AR(1) process with exponential white noise

The object of this paper is a Bayesian analysis of the autoregressive model X _t ?=?ρX _t?1?+?Y _t where 0?Y _t are independent random variables with an exponential distribution of parameter θ. Our study generalizes some results obtained by Turkmann (1990 Amaral Turkmann, M. A. (1990). Bayesian analysis of an autoregressive process with exponential white noise. Statistics, 4: 601–608.  [Google Scholar]). Our analysis is based on a more general non-informative prior which allows us to improve the estimators of ρ and θ.     

On the one parameter unit-Lindley distribution and its associated regression model for proportion data

In this paper considering an appropriate transformation on the Lindley distribution, we propose the unit-Lindley distribution and investigate some of its statistical properties. An important fact associated with this new distribution is that it is possible to obtain the analytical expression for bias correction of the maximum likelihood estimator. Moreover, it belongs to the exponential family. This distribution allows us to incorporate covariates directly in the mean and consequently to quantify their influences on the average of the response variable. Finally, a practical application is presented to show that our model fits much better than the Beta regression.     

A method for parameter estimation of non-linear regression with autocorrelated errors

For a class of non-linear models with stationary dependent residuals an estimating procedure is introduced and its statistical properties are derived. This procedure is useful when no basis exists for assuming a specific parametric model for the error process. For application of the procedure a two step iterative method is described and a small simulation study is performed.     

Robust nonparametric estimation for spatial regression

In this paper, we investigate a nonparametric robust estimation for spatial regression. More precisely, given a strictly stationary random field _Zi=(_Xi,_{Yi)i∈NNN≥1}$Z_{\mathbf{i}}=(X_{\mathbf{i}},Y_{\mathbf{i}}{)}_{\mathbf{i}\in {\mathbb{N}}^{N}N\ge 1}$, we consider a family of robust nonparametric estimators for a regression function based on the kernel method. Under some general mixing assumptions, the almost complete consistency and the asymptotic normality of these estimators are obtained. A robust procedure to select the smoothing parameter adapted to the spatial data is also discussed.     

Linear trimmed means for the linear regression with AR(1) errors model

For the linear regression with AR(1) errors model, the robust generalized and feasible generalized estimators of Lai et al. (2003) of regression parameters are shown to have the desired property of a robust Gauss Markov theorem. This is done by showing that these two estimators are the best among classes of linear trimmed means. Monte Carlo and data analysis for this technique have been performed.     

Robust estimation of multivariate regression model

This paper studies robust estimation of multivariate regression model using kernel weighted local linear regression. A robust estimation procedure is proposed for estimating the regression function and its partial derivatives. The proposed estimators are jointly asymptotically normal and attain nonparametric optimal convergence rate. One-step approximations to the robust estimators are introduced to reduce computational burden. The one-step local M-estimators are shown to achieve the same efficiency as the fully iterative local M-estimators as long as the initial estimators are good enough. The proposed estimators inherit the excellent edge-effect behavior of the local polynomial methods in the univariate case and at the same time overcome the disadvantages of the local least-squares based smoothers. Simulations are conducted to demonstrate the performance of the proposed estimators. Real data sets are analyzed to illustrate the practical utility of the proposed methodology. This work was supported by the National Natural Science Foundation of China (Grant No. 10471006).     

A nonparametric estimation procedure for the Hawkes process: comparison with maximum likelihood estimation

In earlier work, Kirchner [An estimation procedure for the Hawkes process. Quant Financ. 2017;17(4):571–595], we introduced a nonparametric estimation method for the Hawkes point process. In this paper, we present a simulation study that compares this specific nonparametric method to maximum-likelihood estimation. We find that the standard deviations of both estimation methods decrease as power-laws in the sample size. Moreover, the standard deviations are proportional. For example, for a specific Hawkes model, the standard deviation of the branching coefficient estimate is roughly 20% larger than for MLE – over all sample sizes considered. This factor becomes smaller when the true underlying branching coefficient becomes larger. In terms of runtime, our method clearly outperforms MLE. The present bias of our method can be well explained and controlled. As an incidental finding, we see that also MLE estimates seem to be significantly biased when the underlying Hawkes model is near criticality. This asks for a more rigorous analysis of the Hawkes likelihood and its optimization.     

A comparison of estimation procedures for the parameters of the star model

A simulation experiment compares the accuracy and precision of three alternate estimation techniques for the parameters of the STARMA model. Maximum likelihood estimation, in most ways the "best" estimation procedure, involves a large amount of computational effort so that two approximate techniques, exact least squares and conditional maximum likelihood, are often proposed for series of moderate lengths. This simulation experiment compares the accuracy of these three estimation procedures for simulated series of various lengths, and discusses the appropriateness of the three procedures as a function of the length of the observed series.     

Robust estimation and variable selection for varying-coefficient single-index models based on modal regression

ABSTRACT

In this paper, we propose a new efficient and robust penalized estimating procedure for varying-coefficient single-index models based on modal regression and basis function approximations. The proposed procedure simultaneously solves two types of problems: separation of varying and constant effects and selection of variables with non zero coefficients for both non parametric and index components using three smoothly clipped absolute deviation (SCAD) penalties. With appropriate selection of the tuning parameters, the new method possesses the consistency in variable selection and the separation of varying and constant coefficients. In addition, the estimators of varying coefficients possess the optimal convergence rate and the estimators of constant coefficients and index parameters have the oracle property. Finally, we investigate the finite sample performance of the proposed method through a simulation study and real data analysis.     

Robust nonparametric estimation with missing data

In this paper, under a nonparametric regression model, we introduce two families of robust procedures to estimate the regression function when missing data occur in the response. The first proposal is based on a local M$M$-functional applied to the conditional distribution function estimate adapted to the presence of missing data. The second proposal imputes the missing responses using the local M$M$-smoother based on the observed sample and then estimates the regression function with the completed sample. We show that the robust procedures considered are consistent and asymptotically normally distributed. A robust procedure to select the smoothing parameter is also discussed.     

Multiple linear regression model with stochastic design variables

In a simple multiple linear regression model, the design variables have traditionally been assumed to be non-stochastic. In numerous real-life situations, however, they are stochastic and non-normal. Estimators of parameters applicable to such situations are developed. It is shown that these estimators are efficient and robust. A real-life example is given.     

Efficient estimation of auto-regression parameters and innovation distributions for semiparametric integer-valued AR(p) models

Summary.  Integer-valued auto-regressive (INAR) processes have been introduced to model non-negative integer-valued phenomena that evolve over time. The distribution of an INAR( p ) process is essentially described by two parameters: a vector of auto-regression coefficients and a probability distribution on the non-negative integers, called an immigration or innovation distribution. Traditionally, parametric models are considered where the innovation distribution is assumed to belong to a parametric family. The paper instead considers a more realistic semiparametric INAR( p ) model where there are essentially no restrictions on the innovation distribution. We provide an (semiparametrically) efficient estimator of both the auto-regression parameters and the innovation distribution.