Efficient improved estimation of the parameters in two seemingly unrelated regression models

For a system of two seemingly unrelated regression equations, this paper proposes a two-stage covariance improved estimator of the regression coefficients. The new estimator is shown to uniformly dominate the present estimators in terms of generalized mean square error criterion. In addition, we also propose the exact generalized mean square error of new estimator.     

The Equivalence of Parameter Estimates from Growth Curve Models and Seemingly Unrelated Regression Models

Seemingly unrelated regression models and growth curve models are examples of multivariate models that require special estimation techniques. Parameters in seemingly unrelated regression models can be estimated by using two-stage Aitken estimation based on unrestricted residuals; parameters in growth curve models can be estimated by using a Potthoff-Roy (1964) transformation based on an estimate of the dispersion. With proper choice of the seemingly unrelated regression model, the two multivariate models and corresponding parameter estimates are shown to be equivalent. Recognition of the equivalence simplifies the presentation of these more complicated multivariate models. The connection is also of interest for more flexible growth curve models.     

Estimation of parameters in a generalized GMANOVA model based on an outer product analogy and least squares

This paper investigates estimation of parameters in a combination of the multivariate linear model and growth curve model, called a generalized GMANOVA model. Making analogy between the outer product of data vectors and covariance yields an approach to directly do least squares to covariance. An outer product least squares estimator of covariance (COPLS estimator) is obtained and its distribution is presented if a normal assumption is imposed on the error matrix. Based on the COPLS estimator, two-stage generalized least squares estimators of the regression coefficients are derived. In addition, asymptotic normalities of these estimators are investigated. Simulation studies have shown that the COPLS estimator and two-stage GLS estimators are alternative competitors with more efficiency in the sense of sample mean, standard deviations and mean of the variance estimates to the existing ML estimator in finite samples. An example of application is also illustrated.     

Multivariate regression models for discrete and continuous repeated measurements

A general class of multivariate regression models is considered for repeated measurements with discrete and continuous outcome variables. The proposed model is based on the seemingly unrelated regression model (Zellner, 1962) and an extension of the model of Park and Woolson(1992). The regression parameters of the model are consistently estimated using the two-stage least squares method. When the out come variables are multivariate normal, the two-stage estimator reduces to Zellner’s two-stage estimator. As a special case, we consider the marginal distribution described by Liang and Zeger (1986). Under this this distributional assumption, we show that the two-stage estimator has similar asymptotic properties and comparable small sample properties to Liang and Zeger's estimator. Since the proposed approach is based on the least squares method, however, any distributional assumption is not required for variables outcome variables. As a result, the proposed estimator is more robust to the marginal distribution of outcomes.     

Lattice conditional independence models for seemingly unrelated regressions

Seemingly unrelated regressions (SUR) models appear frequently in econometrics and in the analyses of repeated measures designs and longitudinal data. It is known that iterative algorithms are generally required to obtain the MLEs of the regression parameters. Under a minimal set of lattice conditional independence (LCI) restrictions imposed on the covariance structure, however, closed-form MLEs can be obtained by standard linear regression techniques (Andersson and Perlman, 1993, 1994, 1998). In this paper, simulation is used to study the efficiency of these LCI model-based estimators. We also propose two possible improvements of the usual two-stage estimators for the regression parameters.     

Maximum likelihood estimation of variance components of heteroscedastic random anova model

the estimation of variance components of heteroscedastic random model is discussed in this paper. Maximum Likelihood (ML) is described for one-way heteroscedastic random models. The proportionality condition that cell variance is proportional to the cell sample size, is used to eliminate the efffect of heteroscedasticity. The algebraic expressions of the estimators are obtained for the model. It is seen that the algebraic expressions of the estimators depend mainly on the inverse of the variance-covariance matrix of the observation vector. So, the variance-covariance matrix is obtained and the formulae for the inversions are given. A Monte Carlo study is conducted. Five different variance patterns with different numbers of cells are considered in this study. For each variance pattern, 1000 Monte Carlo samples are drawn. Then the Monte Carlo biases and Monte Carlo MSE’s of the estimators of variance components are calculated. In respect of both bias and MSE, the Maximum Likelihood (ML) estimators of variance components are found to be sufficiently good.     

Seemingly unrelated regressions with covariance matrix of cross-equation ridge regression residuals

Generalized least squares estimation of a system of seemingly unrelated regressions is usually a two-stage method: (1) estimation of cross-equation covariance matrix from ordinary least squares residuals for transforming data, and (2) application of least squares on transformed data. In presence of multicollinearity problem, conventionally ridge regression is applied at stage 2. We investigate the usage of ridge residuals at stage 1, and show analytically that the covariance matrix based on the least squares residuals does not always result in more efficient estimator. A simulation study and an application to a system of firms' gross investment support our finding.     

Two-stage Huber estimation

In this paper we propose a new robust estimator in the context of two-stage estimation methods directed towards the correction of endogeneity problems in linear models. Our estimator is a combination of Huber estimators for each of the two stages, with scale corrections implemented using preliminary median absolute deviation estimators. In this way we obtain a two-stage estimation procedure that is an interesting compromise between concerns of simplicity of calculation, robustness and efficiency. This method compares well with other possible estimators such as two-stage least-squares (2SLS) and two-stage least-absolute-deviations (2SLAD), asymptotically and in finite samples. It is notably interesting to deal with contamination affecting more heavily the distribution tails than a few outliers and not losing as much efficiency as other popular estimators in that case, e.g. under normality. An additional originality resides in the fact that we deal with random regressors and asymmetric errors, which is not often the case in the literature on robust estimators.     

High-correlated residuals improved estimation in the high-dimensional SUR model

This article focuses on estimating regression coefficients in the high-dimensional seemingly unrelated regression model. When the number of equations exceeds that of the observations, both the maximum likelihood estimator and Zellner’s two-stage estimator do not exist. As an alternative, we propose a two-stage conditional expectation improved estimator. The new estimator is further improved by the high-correlated residuals, and the high correlation is determined by hypothesis testings. Simulations show that the new estimator outperforms the ordinary least-squares estimator in terms of mean square errors, especially when high-correlated residuals exist between the equations.     

A comparison of LS/ML and GMM estimation in a simple AR(1) model

This paper compares least squares (LS)/maximum likelihood (ML) and generalised method of moments (GMM) estimation in a simple. Gaussian autoregressive of order one (AR(1)) model. First, we show that the usual LS/ML estimator is a corner solution to a general minimisation problem that involves two moment conditions, while the new GMM we devise is not. Secondly, we examine asymptotic and finite sample properties of the new GMM estimator in comparison to the usual LS/ML estimator in a simple AR(1) model. For both stable and unstable (unit root) specifications, we show asymptotic equivalence of the distributions of the two estimators. However, in finite samples, the new GMM estimator performs better.     

A conditional perspective of weighted variance estimation of the optimal regression estimator

The estimation of the variance for the GREG (general regression) estimator by weighted residuals is widely accepted as a method which yields estimators with good conditional properties. Since the optimal (regression) estimator shares the properties of GREG estimators which are used in the construction of weighted variance estimators, we introduce the weighting procedure also for estimating the variance of the optimal estimator. This method of variance estimation was originally presented in a seemingly ad hoc manner, and we shall discuss it from a conditional point of view and also look at an alternative way of utilizing the weights. Examples that stress conditional behaviour of estimators are then given for elementary sampling designs such as simple random sampling, stratified simple random sampling and Poisson sampling, where for the latter design we have conducted a small simulation study.     

Efficient mean estimation in log-normal linear models

Log-normal linear models are widely used in applications, and many times it is of interest to predict the response variable or to estimate the mean of the response variable at the original scale for a new set of covariate values. In this paper we consider the problem of efficient estimation of the conditional mean of the response variable at the original scale for log-normal linear models. Several existing estimators are reviewed first, including the maximum likelihood (ML) estimator, the restricted ML (REML) estimator, the uniformly minimum variance unbiased (UMVU) estimator, and a bias-corrected REML estimator. We then propose two estimators that minimize the asymptotic mean squared error and the asymptotic bias, respectively. A parametric bootstrap procedure is also described to obtain confidence intervals for the proposed estimators. Both the new estimators and the bootstrap procedure are very easy to implement. Comparisons of the estimators using simulation studies suggest that our estimators perform better than the existing ones, and the bootstrap procedure yields confidence intervals with good coverage properties. A real application of estimating the mean sediment discharge is used to illustrate the methodology.     

Minimum distance estimation in linear regression with strong mixing errors

Abstract

Minimum distance estimation on the linear regression model with independent errors is known to yield an efficient and robust estimator. We extend the method to the model with strong mixing errors and obtain an estimator of the vector of the regression parameters. The goal of this article is to demonstrate the proposed estimator still retains efficiency and robustness. To that end, this article investigates asymptotic distributional properties of the proposed estimator and compares it with other estimators. The efficiency and the robustness of the proposed estimator are empirically shown, and its superiority over the other estimators is established.     

Weak Instrumental Variables Models for Longitudinal Data

This article considers the estimation and testing of a within-group two-stage least squares (TSLS) estimator for instruments with varying degrees of weakness in a longitudinal (panel) data model. We show that adding the repeated cross-sectional information into a regression model can improve the estimation in weak instruments. Moreover, the consistency and limiting distribution of the TSLS estimator are established when both N and T tend to infinity. Some asymptotically pivotal tests are extended to a longitudinal data model and their asymptotic properties are examined. A Monte Carlo experiment is conducted to evaluate the finite sample performance of the proposed estimators.     

A two-stage estimation for panel data models with grouped fixed effects

ABSTRACT

This paper considers panel data models with fixed effects which have grouped patterns with unknown group membership. A two-stage estimation (TSE) procedure is developed to improve the properties of the GFE estimators of common parameters when the time span is small. Firstly, the common parameters are estimated. Subsequently, the optimal group assignment and the estimators of group effects are obtained by the K-means algorithm. Monte Carlo results reveal that the TSE estimator has a much smaller bias than the GFE estimator when the values of difference between effects are moderately small or at high variance of the idiosyncratic error.     

Small-area estimation by combining time-series and cross-sectional data

A model involving autocorrelated random effects and sampling errors is proposed for small-area estimation, using both time-series and cross-sectional data. The sampling errors are assumed to have a known block-diagonal covariance matrix. This model is an extension of a well-known model, due to Fay and Herriot (1979), for cross-sectional data. A two-stage estimator of a small-area mean for the current period is obtained under the proposed model with known autocorrelation, by first deriving the best linear unbiased prediction estimator assuming known variance components, and then replacing them with their consistent estimators. Extending the approach of Prasad and Rao (1986, 1990) for the Fay-Herriot model, an estimator of mean squared error (MSE) of the two-stage estimator, correct to a second-order approximation for a small or moderate number of time points, T, and a large number of small areas, m, is obtained. The case of unknown autocorrelation is also considered. Limited simulation results on the efficiency of two-stage estimators and the accuracy of the proposed estimator of MSE are présentés.     

Nonparametric curve estimation with missing data: A general empirical process approach

A general nonparametric imputation procedure, based on kernel regression, is proposed to estimate points as well as set- and function-indexed parameters when the data are missing at random (MAR). The proposed method works by imputing a specific function of a missing value (and not the missing value itself), where the form of this specific function is dictated by the parameter of interest. Both single and multiple imputations are considered. The associated empirical processes provide the right tool to study the uniform convergence properties of the resulting estimators. Our estimators include, as special cases, the imputation estimator of the mean, the estimator of the distribution function proposed by Cheng and Chu [1996. Kernel estimation of distribution functions and quantiles with missing data. Statist. Sinica 6, 63–78], imputation estimators of a marginal density, and imputation estimators of regression functions.     

Maximum likelihood estimation of the parameters of the bivariate binomial distribution

We consider the problem of maximum likelihood estimation of the parameters of the bxvariate binomial distribution, In the statistical literature, this problem is solved when the observed sample is available in the form of a 2x2 contingency table, that is, with all four cell fre quencies given,, The present paper provides a solution for this problem when only the marginal totals of the 2x2 table are observed, which is the natural set-up in a bivariate sampling situation.. Thus, based on a sample [(X_i,Y_i:), i = 1, …, k] from a bivariate binomial population, we derive maximum likelihood (ML) estimators for the two marginal parameters p₁,p₂: and the covariance parameter p₁₁: It. turns out that the ML estimators for P₁: and P₂: are expressed explicitly in terms of the sample values, whereas the ML estimator for p₁₁: can only be obtained numerically by iterative methods Two nu merical illustrations are also presented     

A Seemingly Unrelated Nonparametric Additive Model with Autoregressive Errors

This article considers a nonparametric additive seemingly unrelated regression model with autoregressive errors, and develops estimation and inference procedures for this model. Our proposed method first estimates the unknown functions by combining polynomial spline series approximations with least squares, and then uses the fitted residuals together with the smoothly clipped absolute deviation (SCAD) penalty to identify the error structure and estimate the unknown autoregressive coefficients. Based on the polynomial spline series estimator and the fitted error structure, a two-stage local polynomial improved estimator for the unknown functions of the mean is further developed. Our procedure applies a prewhitening transformation of the dependent variable, and also takes into account the contemporaneous correlations across equations. We show that the resulting estimator possesses an oracle property, and is asymptotically more efficient than estimators that neglect the autocorrelation and/or contemporaneous correlations of errors. We investigate the small sample properties of the proposed procedure in a simulation study.     

Sequential Estimation Of The Mean Logistic Response Function.

In this paper we show that the maximum likelihood estimators of LD50 and the scale parameter in the logistic case are equivalent to the Spearman-Karber type of estimators, when the subsample sizes at each dose level are equal. We derive a simple expression for the asymptotic variance-covariance matrix of the estimators. We also obtain correction terms to the bias and the variance of the Spearman-Karber estimator of LD50. Using these we construct sequential fixed-width and risk-efficient procedures for LD50. Numerical calculations show that these procedures are simple to carry out and stop at fewer dose levels than those proposed earlier by Nanthakumar and Govindarajulu (1994, 1999).