Estimation of parameters in a generalized GMANOVA model based on an outer product analogy and least squares

This paper investigates estimation of parameters in a combination of the multivariate linear model and growth curve model, called a generalized GMANOVA model. Making analogy between the outer product of data vectors and covariance yields an approach to directly do least squares to covariance. An outer product least squares estimator of covariance (COPLS estimator) is obtained and its distribution is presented if a normal assumption is imposed on the error matrix. Based on the COPLS estimator, two-stage generalized least squares estimators of the regression coefficients are derived. In addition, asymptotic normalities of these estimators are investigated. Simulation studies have shown that the COPLS estimator and two-stage GLS estimators are alternative competitors with more efficiency in the sense of sample mean, standard deviations and mean of the variance estimates to the existing ML estimator in finite samples. An example of application is also illustrated.     

High-correlated residuals improved estimation in the high-dimensional SUR model

This article focuses on estimating regression coefficients in the high-dimensional seemingly unrelated regression model. When the number of equations exceeds that of the observations, both the maximum likelihood estimator and Zellner’s two-stage estimator do not exist. As an alternative, we propose a two-stage conditional expectation improved estimator. The new estimator is further improved by the high-correlated residuals, and the high correlation is determined by hypothesis testings. Simulations show that the new estimator outperforms the ordinary least-squares estimator in terms of mean square errors, especially when high-correlated residuals exist between the equations.     

Model Reduction for Prediction in Regression Models

We look at prediction in regression models under squared loss for the random x case with many explanatory variables. Model reduction is done by conditioning upon only a small number of linear combinations of the original variables. The corresponding reduced model will then essentially be the population model for the chemometricians' partial least squares algorithm. Estimation of the selection matrix under this model is briefly discussed, and analoguous results for the case with multivariate response are formulated. Finally, it is shown that an assumption of multinormality may be weakened to assuming elliptically symmetric distribution, and that some of the results are valid without any distributional assumption at all.     

Weak Instrumental Variables Models for Longitudinal Data

This article considers the estimation and testing of a within-group two-stage least squares (TSLS) estimator for instruments with varying degrees of weakness in a longitudinal (panel) data model. We show that adding the repeated cross-sectional information into a regression model can improve the estimation in weak instruments. Moreover, the consistency and limiting distribution of the TSLS estimator are established when both N and T tend to infinity. Some asymptotically pivotal tests are extended to a longitudinal data model and their asymptotic properties are examined. A Monte Carlo experiment is conducted to evaluate the finite sample performance of the proposed estimators.     

Nonparametric Shrinkage Estimation for Aalen's Additive Hazards Model

Aalen's nonparametric additive model in which the regression coefficients are assumed to be unspecified functions of time is a flexible alternative to Cox's proportional hazards model when the proportionality assumption is in doubt. In this paper, we incorporate a general linear hypothesis into the estimation of the time‐varying regression coefficients. We combine unrestricted least squares estimators and estimators that are restricted by the linear hypothesis and produce James‐Stein‐type shrinkage estimators of the regression coefficients. We develop the asymptotic joint distribution of such restricted and unrestricted estimators and use this to study the relative performance of the proposed estimators via their integrated asymptotic distributional risks. We conduct Monte Carlo simulations to examine the relative performance of the estimators in terms of their integrated mean square errors. We also compare the performance of the proposed estimators with a recently devised LASSO estimator as well as with ridge‐type estimators both via simulations and data on the survival of primary billiary cirhosis patients.     

Equivalence of maximum likelihood estimation and iterative two-stage estimation for seemingly unrelated regression models

The seemingly unrelated regression model is viewed in the context of repeated measures analysis. Regression parameters and the variance-covariance matrix of the seemingly unrelated regression model can be estimated by using two-stage Aitken estimation. The first stage is to obtain a consistent estimator of the variance-covariance matrix. The second stage uses this matrix to obtain the generalized least squares estimators of the regression parameters. The maximum likelihood (ML) estimators of the regression parameters can be obtained by performing the two-stage estimation iteratively. The iterative two-stage estimation procedure is shown to be equivalent to the EM algorithm (Dempster, Laird, and Rubin, 1977) proposed by Jennrich and Schluchter (1986) and Laird, Lange, and Stram (1987) for repeated measures data. The equivalence of the iterative two-stage estimator and the ML estimator has been previously demonstrated empirically in a Monte Carlo study by Kmenta and Gilbert (1968). It does not appear to be widely known that the two estimators are equivalent theoretically. This paper demonstrates this equivalence.     

Heteroskedastic linear regression model with compositional response and covariates

Compositional data are known as a sort of complex multidimensional data with the feature that reflect the relative information rather than absolute information. There are a variety of models for regression analysis with compositional variables. Similar to the traditional regression analysis, the heteroskedasticity still exists in these models. However, the existing heteroskedastic regression analysis methods cannot apply in these models with compositional error term. In this paper, we mainly study the heteroskedastic linear regression model with compositional response and covariates. The parameter estimator is obtained through weighted least squares method. For the hypothesis test of parameter, the test statistic is based on the original least squares estimator and corresponding heteroskedasticity-consistent covariance matrix estimator. When the proposed method is applied to both simulation and real example, we use the original least squares method as a comparison during the whole process. The results implicate the model's practicality and effectiveness in regression analysis with heteroskedasticity.     

A unified approach to estimation of nonlinear mixed effects and Berkson measurement error models

Mixed effects models and Berkson measurement error models are widely used. They share features which the author uses to develop a unified estimation framework. He deals with models in which the random effects (or measurement errors) have a general parametric distribution, whereas the random regression coefficients (or unobserved predictor variables) and error terms have nonparametric distributions. He proposes a second-order least squares estimator and a simulation-based estimator based on the first two moments of the conditional response variable given the observed covariates. He shows that both estimators are consistent and asymptotically normally distributed under fairly general conditions. The author also reports Monte Carlo simulation studies showing that the proposed estimators perform satisfactorily for relatively small sample sizes. Compared to the likelihood approach, the proposed methods are computationally feasible and do not rely on the normality assumption for random effects or other variables in the model.     

A Two-Stage Estimator of the Dependence Parameter for the Clayton-Oakes Model

This paper describes the properties of a two-stage estimator of the dependence parameter in the Clayton-Oakes multivariate failure time model. The parameter is estimated from a likelihood function in which the marginal hazard functions are replaced by estimates. The method extends the approach of Shih and Louis (1995) and Genest, Ghoudi and Rivest (1995) to allow the marginal hazard for failure times to follow a stratified Cox (1972) model. The method is computationally simple and under mild regularity conditions produces a consistent, asymptotically normal estimator.     

Instrumental variable estimation in nonlinear measurement error models

This paper presents an extension of instrumental variable estimation to nonlinear regression models. For the linear model, the extended estimator is equivalent to the two-stage least squares estimator. The extended estimator is consistent for an important class of nonlinear models, including the logistic model, under relatively weak assumptions on the distribution of the measurement error. An example and simulation study are presented for the logistic regression model. The simulations suggest the estimator is reasonably efficient.     

A simulation comparison of estimators for a regression coefficient under differential non-response

We consider the estimation of a regression coefficient in a linear regression when observations are missing due to nonresponse. Response is assumed to be determined by a nonobservable variable which is linearly related to an observable variable. The values of the observable variable are assumed to be available for the whole sample but the variable is not includsd in the regression relationship of interest . Several alternative estimators have been proposed for this situation under various simplifying assumptions. A sampling theory approach provides three alternative estimatrs by considering the observatins as obtained from a sub-sample, selected on the basis of the fully observable variable , as formulated by Nathan and Holt (1980). Under an econometric approach, Heckman (1979) proposed a two-stage (probit and OLS) estimator which is consistent under specificconditions. A simulation comparison of the four estimators and the ordinary least squares estimator , under multivariate normality of all the variables involved, indicates that the econometric approach estimator is not robust to departures from the conditions underlying its derivation, while two of the other estimators exhibit a similar degree of stable performance over a wide range of conditions. Simulations for a non-normal distribution show that gains in performance can be obtained if observations on the independent variable are available for the whole population.     

Estimation of regression coefficient from survey data based on test of significance

The ordinary least squares (OLS)estimator of regression coeffecient is implicitly based on I.I.D.assumption, which is rarely satisfied by survey data. Many approaches are proposed in the literature which can be classified in two broad categories as model based and design consistent.Du Mouchel and Duncan (1983) proposed a test statistic λwhich helps in testing the ignorability of sampling weights.In this article a preliminary test estimator based on λ is proposed. The model based properties of this estimator has been invetigated theoritically where as to study the design based properties simulation approach is adopted. It has been observed that the proposed estimator is a better cimpromise between model based and randomization based inferential frame work.     

Estimating Percentiles of Time-to-Failure Distribution Obtained from a Linear Degradation Model Using Kernel Density Method

In this article, we propose a nonparametric estimator for percentiles of the time-to-failure distribution obtained from a linear degradation model using the kernel density method. The properties of the proposed kernel estimator are investigated and compared with well-known maximum likelihood and ordinary least squares estimators via a simulation technique. The mean squared error and the length of the bootstrap confidence interval are used as the basis criteria of the comparisons. The simulation study shows that the performance of the kernel estimator is acceptable as a general estimator. When the distribution of the data is assumed to be known, the maximum likelihood and ordinary least squares estimators perform better than the kernel estimator, while the kernel estimator is superior when the assumption of our knowledge of the data distribution is violated. A comparison among different estimators is achieved using a real data set.     

On multivariate quantile regression analysis

This paper investigates the estimation of parameters in a multivariate quantile regression model when the investigator wants to evaluate the associated distribution function. It proposes a new directional quantile estimator with the following properties: (1) it applies to an arbitrary number of random variables; (2) it is equivalent to estimating the distribution function allowing for non-convex distribution contours; (3) it satisfies nice equivariance properties; (4) it has desirable statistical properties (i.e., consistency and asymptotic normality); and (5) its implementation involves a modest computational burden: our proposed estimator can be obtained by solving parametric linear programming problems. As such, this paper expands the range of applications of quantile estimation for multivariate regression models.     

Penalized weighted composite quantile regression in the linear regression model with heavy-tailed autocorrelated errors

In this paper, a penalized weighted composite quantile regression estimation procedure is proposed to estimate unknown regression parameters and autoregression coefficients in the linear regression model with heavy-tailed autoregressive errors. Under some conditions, we show that the proposed estimator possesses the oracle properties. In addition, we introduce an iterative algorithm to achieve the proposed optimization problem, and use a data-driven method to choose the tuning parameters. Simulation studies demonstrate that the proposed new estimation method is robust and works much better than the least squares based method when there are outliers in the dataset or the autoregressive error distribution follows heavy-tailed distributions. Moreover, the proposed estimator works comparably to the least squares based estimator when there are no outliers and the error is normal. Finally, we apply the proposed methodology to analyze the electricity demand dataset.     

The Asymptotic Covariance Matrix of the Least Squares Estimator in the Stochastic Linear Regression Model: The Case of Elliptically Symmetric Distribution

This article considers the unconditional asymptotic covariance matrix of the least squares estimator in the linear regression model with stochastic explanatory variables. The asymptotic covariance matrix of the least squares estimator of regression parameters is evaluated relative to the standard asymptotic covariance matrix when the joint distribution of the dependent and explanatory variables is in the class of elliptically symmetric distributions. An empirical example using financial data is presented. Numerical examples and simulation experiments are given to illustrate the difference of the two asymptotic covariance matrices.     

Composite Estimation for Single‐Index Models with Responses Subject to Detection Limits

We propose a semiparametric estimator for single‐index models with censored responses due to detection limits. In the presence of left censoring, the mean function cannot be identified without any parametric distributional assumptions, but the quantile function is still identifiable at upper quantile levels. To avoid parametric distributional assumption, we propose to fit censored quantile regression and combine information across quantile levels to estimate the unknown smooth link function and the index parameter. Under some regularity conditions, we show that the estimated link function achieves the non‐parametric optimal convergence rate, and the estimated index parameter is asymptotically normal. The simulation study shows that the proposed estimator is competitive with the omniscient least squares estimator based on the latent uncensored responses for data with normal errors but much more efficient for heavy‐tailed data under light and moderate censoring. The practical value of the proposed method is demonstrated through the analysis of a human immunodeficiency virus antibody data set.     

Instrumental variable estimation of heteroskedasticity adaptive error component models

The linear panel data estimator proposed by Hausman and Taylor relaxes the hypothesis of exogenous regressors that is assumed by generalized least squares methods but, unlike the Fixed Effects estimator, it can handle endogenous time invariant explanatory variables in the regression equation. One of the assumptions underlying the estimator is the homoskedasticity of the error components. This can be restrictive in applications, and therefore in this paper the assumption is relaxed and more efficient adaptive versions of the estimator are presented.     

Rank-based ridge estimation in multiple linear regression

Multicollinearity and model misspecification are frequently encountered problems in practice that produce undesirable effects on classical ordinary least squares (OLS) regression estimator. The ridge regression estimator is an important tool to reduce the effects of multicollinearity, but it is still sensitive to a model misspecification of error distribution. Although rank-based statistical inference has desirable robustness properties compared to the OLS procedures, it can be unstable in the presence of multicollinearity. This paper introduces a rank regression estimator for regression parameters and develops tests for general linear hypotheses in a multiple linear regression model. The proposed estimator and the tests have desirable robustness features against the multicollinearity and model misspecification of error distribution. Asymptotic behaviours of the proposed estimator and the test statistics are investigated. Real and simulated data sets are used to demonstrate the feasibility and the performance of the estimator and the tests.     

A robust and efficient estimation method for nonparametric models with jump points

Nonparametric models with jump points have been considered by many researchers. However, most existing methods based on least squares or likelihood are sensitive when there are outliers or the error distribution is heavy tailed. In this article, a local piecewise-modal method is proposed to estimate the regression function with jump points in nonparametric models, and a piecewise-modal EM algorithm is introduced to estimate the proposed estimator. Under some regular conditions, the large-sample theory is established for the proposed estimators. Several simulations are presented to evaluate the performances of the proposed method, which shows that the proposed estimator is more efficient than the local piecewise-polynomial regression estimator in the presence of outliers or heavy tail error distribution. What is more, the proposed procedure is asymptotically equivalent to the local piecewise-polynomial regression estimator under the assumption that the error distribution is a Gaussian distribution. The proposed method is further illustrated via the sea-level pressures.