On sufficient conditions for the strong consistency of least-squares estimates

The strong consistency of the least-squares estimates in regression models is obtained when the errors are i.i.d. with absolute moment of order r, 0<r? 2. The assumptions presented for the random error sequence will permit us to obtain improvements of the conditions on the regressors in order to obtain the strong consistency of the least-squares estimates in linear and nonlinear regression models.     

Improved autoregressive forecasts in the presence of non-normal errors

This paper is concerned with obtaining more accurate point forecasts in the presence of non-normal errors. Specifically, we apply the residual augmented least-squares (RALS) estimator to autoregressive models to utilize the additional moment restrictions embodied in non-normal errors. Monte Carlo experiments are performed to compare our RALS forecasts to forecasts based on the ordinary least-squares estimator and the least absolute deviations (LAD) estimator. We find that the RALS approach provides superior forecasts when the data are skewed. Compared to the LAD forecast, the RALS forecast has smaller mean squared prediction errors in the baseline case with normal errors.     

Efficient Estimation of the Seemingly Unrelated Regression Cointegration Model and Testing for Purchasing Power Parity

This paper studies the efficient estimation of seemingly unrelated linear models with integrated regressors and stationary errors. We consider two cases. The first one has no common regressor among the equations. In this case, we show that by adding leads and lags of the first differences of the regressors and estimating this augmented dynamic regression model by generalized least squares using the long-run covariance matrix, we obtain an efficient estimator of the cointegrating vector that has a limiting mixed normal distribution. In the second case we consider, there is a common regressor to all equations, and we discuss efficient minimum distance estimation in this context. Simulation results suggests that our new estimator compares favorably with others already proposed in the literature. We apply these new estimators to the testing of the proportionality and symmetry conditions implied by purchasing power parity (PPP) among the G-7 countries. The tests based on the efficient estimates easily reject the joint hypotheses of proportionality and symmetry for all countries with either the United States or Germany as numeraire. Based on individual tests, our results suggest that Canada and Germany are the most likely countries for which the proportionality condition holds, and that Italy and Japan for the symmetry condition relative to the United States.     

The feasible generalized restricted ridge regression estimator

The presence of autocorrelation in errors and multicollinearity among the regressors have undesirable effects on the least-squares regression. There are a wide range of methods which are proposed to overcome the usefulness of the ordinary least-squares estimator or the generalized least-squares estimator, such as the Stein-rule, restricted least-squares or ridge estimator. Therefore, we introduce a new feasible generalized restricted ridge regression (FGRR) estimator to examine multicollinearity and autocorrelation problems simultaneously for the general linear regression model. We also derive some statistical properties of the FGRR estimator and comparisons have been conducted using matrix mean-square error. Moreover, a Monte Carlo simulation experiment is performed to investigate the performance of the proposed estimator over the others.     

Efficient Estimation of the Seemingly Unrelated Regression Cointegration Model and Testing for Purchasing Power Parity

Abstract

This paper studies the efficient estimation of seemingly unrelated linear models with integrated regressors and stationary errors. We consider two cases. The first one has no common regressor among the equations. In this case, we show that by adding leads and lags of the first differences of the regressors and estimating this augmented dynamic regression model by generalized least squares using the long-run covariance matrix, we obtain an efficient estimator of the cointegrating vector that has a limiting mixed normal distribution. In the second case we consider, there is a common regressor to all equations, and we discuss efficient minimum distance estimation in this context. Simulation results suggests that our new estimator compares favorably with others already proposed in the literature. We apply these new estimators to the testing of the proportionality and symmetry conditions implied by purchasing power parity (PPP) among the G-7 countries. The tests based on the efficient estimates easily reject the joint hypotheses of proportionality and symmetry for all countries with either the United States or Germany as numeraire. Based on individual tests, our results suggest that Canada and Germany are the most likely countries for which the proportionality condition holds, and that Italy and Japan for the symmetry condition relative to the United States.     

Local composite quantile regression estimation of time-varying parameter vector for multidimensional time-inhomogeneous diffusion models

This paper is dedicated to the study of the composite quantile regression (CQR) estimations of time-varying parameter vectors for multidimensional diffusion models. Based on the local linear fitting for parameter vectors, we propose the local linear CQR estimations of the drift parameter vectors, and verify their asymptotic biases, asymptotic variances and asymptotic normality. Moreover, we discuss the asymptotic relative efficiency (ARE) of the local linear CQR estimations with respect to the local linear least-squares estimations. We obtain that the local estimations that we proposed are much more efficient than the local linear least-squares estimations. Simulation studies are constructed to show the performance of the estimations proposed.     

Least absolute deviations estimation via the EM algorithm

This paper derives EM and generalized EM (GEM) algorithms for calculating least absolute deviations (LAD) estimates of the parameters of linear and nonlinear regression models. It shows that Schlossmacher's iterative reweighted least squares algorithm for calculating LAD estimates (E.J. Schlossmacher, Journal of the American Statistical Association 68: 857–859, 1973) is an EM algorithm. A GEM algorithm for computing LAD estimates of the parameters of nonlinear regression models is also provided and is applied in some examples.     

On additive and block decompositions of WLSEs under a multiple partitioned regression model

While considering the mechanism of weighted least-squares estimators (WLSEs) of regression coefficients in a partitioned linear model, Tian and Takane [On sum decompositions of weighted least-squares estimators under the partitioned linear model, Comm. Statist. Theory Methods 37 (2008), pp. 55–69] gave some identifying conditions for the WLSEs to be the sum of WLSEs under its two small models based on orthogonality of regressors with respect to the given weight matrix. The purpose of this paper is to show how to establish additive and block decompositions of WLSEs under a multiple partitioned linear model and its k small models based on orthogonality of regressors with respect to a given weight matrix.     

Comparison of estimation methods for the Marshall–Olkin extended Lindley distribution

The aim of this paper is to compare the parameters' estimations of the Marshall–Olkin extended Lindley distribution obtained by six estimation methods: maximum likelihood, ordinary least-squares, weighted least-squares, maximum product of spacings, Cramér–von Mises and Anderson–Darling. The bias, root mean-squared error, average absolute difference between the true and estimate distributions' functions and the maximum absolute difference between the true and estimate distributions' functions are used as comparison criteria. Although the maximum product of spacings method is not widely used, the simulation study concludes that it is highly competitive with the maximum likelihood method.     

Semiparametric Sieve-Type Generalized Least Squares Inference

This article considers the problem of statistical inference in linear regression models with dependent errors. A sieve-type generalized least squares (GLS) procedure is proposed based on an autoregressive approximation to the generating mechanism of the errors. The asymptotic properties of the sieve-type GLS estimator are established under general conditions, including mixingale-type conditions as well as conditions which allow for long-range dependence in the stochastic regressors and/or the errors. A Monte Carlo study examines the finite-sample properties of the method for testing regression hypotheses.     

Strong consistency in nonlinear regression with multipucative error

Sufficient conditions are given to ensure the existence of a sequence of strongly consistent estimators of the true parameter for a nonlinear regression model naving random regressors and a multiplicative disturbance term. Special cases of this result include the least absolute value and the least squares estimation procedures.     

Estimation of the SURE model

The paper presents the essentials of the SURE model and the estimation of its parameters β and ω. Two alternative compact representations of the model are being used. The parameter β is estimated by least squares (LS), generalized least squares (GLS) and maximum likelihood (ML) (under normality). For ω two estimators are being considered, viz an LS-related estimator and a maximum likelihood estimator (under normality). Attention is being given to the study of asymptotic properties of all estimators examined. It turns out that the LS-related and ML estimators of ω follow the same asymptotic (normal) distribution. Efficiency comparisons for the various estimators of β conclude the paper.     

Estimation of Sparse Structural Parameters with Many Endogenous Variables

We apply the generalized method of moments–least absolute shinkage and selection operator (GMM-Lasso) (Caner, 2009) to a linear structural model with many endogenous regressors. If the true parameter is sufficiently sparse, we can establish a new oracle inequality, which implies that GMM-Lasso performs almost as well as if we knew a priori the identities of the relevant variables. Sparsity, meaning that most of the true coefficients are too small to matter, naturally arises in econometric applications where the model can be derived from economic theory. In addition, we propose to use a modified version of AIC or BIC to select the tuning parameter in practical implementation. Simulations provide supportive evidence concerning the finite sample properties of the GMM-Lasso.     

Iterated Feasible Generalized Least-Squares Estimation of Augmented Dynamic Panel Data Models

This article provides the large sample distribution of the iterated feasible generalized least-squares (IFGLS) estimator of an augmented dynamic panel data model. The regressors in the model include lagged values of the dependent variable and may include other explanatory variables that, while exogenous with respect to the time-varying error component, may be correlated with an unobserved time-invariant component. The article compares the finite sample properties of the IFGLS estimator to that of GMM estimators using both simulated and real data and finds that the IFGLS estimator compares favorably.     

Complexity of penalized likelihood estimation

We show that for a class of penalty functions, finding the global optimizer in the penalized least-squares estimation is equivalent to the ‘exact cover by 3-sets’ problem, which belongs to a class of NP-hard problems. The NP-hardness result is then extended to the cases of penalized least absolute deviations regression and a special class of penalized support vector machines. We discuss its implication in statistics. To the best of our knowledge, this is the first formal documentation on the complexity of this type of problem.     

Testing parameter significance in instrumental variables probit estimators: some simulation

Binary choice models that contain endogenous regressors can now be estimated routinely using modern software. Each of the two packages, Stata 11 [Stata Statistical Software: Release 11, StataCorp LP, College Station, TX, 2009] and Limdep 9 [Econometric Software Inc., Plainview, NY, 2008], contains two estimators that can be used to estimate such a model. This paper compares the performance of maximum likelihood, Newey's Amemiya's generalized least-squares (AGLS) estimator, an instrumental variables plug-in estimator and others in samples of sizes 200 and 1000 using simulation. Specifically, this paper focuses on tests of parameter significance under various degrees of instrument strength and severity of endogeneity. Although the maximum-likelihood estimator performs well in large samples, there is some evidence that the more computationally robust AGLS estimator may perform better in smaller samples when instruments are weak. It also appears that instruments in endogenous probit estimation need to be even stronger than when used in linear instrumental variables (IV) estimation.     

Two-stage Huber estimation

In this paper we propose a new robust estimator in the context of two-stage estimation methods directed towards the correction of endogeneity problems in linear models. Our estimator is a combination of Huber estimators for each of the two stages, with scale corrections implemented using preliminary median absolute deviation estimators. In this way we obtain a two-stage estimation procedure that is an interesting compromise between concerns of simplicity of calculation, robustness and efficiency. This method compares well with other possible estimators such as two-stage least-squares (2SLS) and two-stage least-absolute-deviations (2SLAD), asymptotically and in finite samples. It is notably interesting to deal with contamination affecting more heavily the distribution tails than a few outliers and not losing as much efficiency as other popular estimators in that case, e.g. under normality. An additional originality resides in the fact that we deal with random regressors and asymmetric errors, which is not often the case in the literature on robust estimators.     

Multiple-response repeated measurement or multivariate growth curve model with distribution-free errors

In this article, we propose a new modeling approach for the multivariate growth curve model with distribution-free errors, which is a useful tool for analyzing multiple-response repeated measurements. We first use the outer product least-squares technique to directly estimate covariance and then explore the feasible generalized least-squares technique to derive the estimator of regression coefficients. Large-sample properties are investigated for these estimators. Moreover, the above estimations for covariance and regression coefficients are extended to the situation under certain null hypothesis tests and the best subset BIC is used for variable selection. A real dataset is analyzed to demonstrate the usefulness and competency of the proposed methodology for model specification (identification) and model fitting (parameter estimation) in multiple-response repeated measurements.     

Assessing Influence on the Liu Estimates in Linear Regression Models

The Liu estimator has been developed as an alternative to the ordinary least squares estimator in the presence of collinearity among the elements of regressors in linear regression models. We present the DFFITS and different versions of the Cook distance analogous to the ones given for the ordinary linear regression models of each individual observation on the Liu estimates. We suggest a version of the Cook distance based on one-step approximation. The mean shift outlier model for the Liu regression has also been investigated. Moreover, using the Sherman-Morrison-Woodbury theorem, we find approximate versions of the DFFITS and the Cook distance. The proposed diagnostics are evaluated on two data sets and yield promising results.     

Semiparametric estimation of moment condition models with weakly dependent data

This paper develops the asymptotic theory for the estimation of smooth semiparametric generalized estimating equations models with weakly dependent data. The paper proposes new estimation methods based on smoothed two-step versions of the generalised method of moments and generalised empirical likelihood methods. An important aspect of the paper is that it allows the first-step estimation to have an effect on the asymptotic variances of the second-step estimators and explicitly characterises this effect for the empirically relevant case of the so-called generated regressors. The results of the paper are illustrated with a partially linear model that has not been previously considered in the literature. The proofs of the results utilise a new uniform strong law of large numbers and a new central limit theorem for U-statistics with varying kernels that are of independent interest.