Projectors and linear estimation in general linear models

The paper gives a self-contained account of minimum disper­sion linear unbiased estimation of the expectation vector in a linear model with the dispersion matrix belonging to some, rather arbitrary, set of nonnegative definite matrices. The approach to linear estimation in general linear models recommended here is a direct generalization of some ideas and results presented by Rao (1973, 19 74) for the case of a general Gauss-Markov model

A new insight into the nature of some estimation problems originaly arising in the context of a general Gauss-Markov model as well as the correspondence of results known in the literature to those obtained in the present paper for general linear models are also given. As preliminary results the theory of projectors defined by Rao (1973) is extended.     

Simultaneous structure estimation and variable selection in partial linear varying coefficient models for longitudinal data

Partial linear varying coefficient models (PLVCM) are often considered for analysing longitudinal data for a good balance between flexibility and parsimony. The existing estimation and variable selection methods for this model are mainly built upon which subset of variables have linear or varying effect on the response is known in advance, or say, model structure is determined. However, in application, this is unreasonable. In this work, we propose a simultaneous structure estimation and variable selection method, which can do simultaneous coefficient estimation and three types of selections: varying and constant effects selection, relevant variable selection. It can be easily implemented in one step by employing a penalized M-type regression, which uses a general loss function to treat mean, median, quantile and robust mean regressions in a unified framework. Consistency in the three types of selections and oracle property in estimation are established as well. Simulation studies and real data analysis also confirm our method.     

Efficient estimation of general linear mixed effects models

An extension of the linear growth curve model (Biometrics 38 (1982) 963) was proposed by Stukel and Demidenko (Biometrics 53 (1997) 720) to study the effects of population covariates on one or more characteristics of the curve, when the characteristics are expressed as linear combinations of the growth curve parameters. In the present paper, this general growth curve model receives a comprehensive theoretical treatment. A two-stage estimator, consisting of a generalized least squares estimator under constraints for the population parameters and a moment estimator for the variance parameters, is developed for application in the non-Gaussian error situation. Two likelihood based estimators, global maximum likelihood and second-stage maximum likelihood, are also developed. It is shown that all three estimators are consistent, asymptotically normally distributed, and efficient, and are equivalent when the number of individuals tends to infinity. An expression for the bias in the estimator of the population parameters is derived under second stage model misspecification. We show that if parameters that are not of primary interest are incorrectly specified, bias may occur in parameters that are of interest using the standard growth curve model. The general growth curve model does not require specification of such nuisance parameters and is robust in terms of bias. The general linear growth curve model is used to study the effects of host sex on pancreatic tumor growth in rats.     

Simultaneous equivariant estimation of the parameters of linear models

Consider a family of distributions which is invariant under a group of transformations. In this paper, we define an optimality criterion with respect to an arbitrary convex loss function and we prove a characterization theorem for an equivariant estimator to be optimal. Then we consider a linear model Y=Xβ+ε, in which ε has a multivariate distribution with mean vector zero and has a density belonging to a scale family with scale parameter σ. Also we assume that the underlying family of distributions is invariant with respect to a certain group of transformations. First, we find the class of all equivariant estimators of regression parameters and the powers of σ. By using the characterization theorem we discuss the simultaneous equivariant estimation of the parameters of the linear model.     

WLAD-LASSO method for robust estimation and variable selection in partially linear models

This paper focuses on robust estimation and variable selection for partially linear models. We combine the weighted least absolute deviation (WLAD) regression with the adaptive least absolute shrinkage and selection operator (LASSO) to achieve simultaneous robust estimation and variable selection for partially linear models. Compared with the LAD-LASSO method, the WLAD-LASSO method will resist to the heavy-tailed errors and outliers in the parametric components. In addition, we estimate the unknown smooth function by a robust local linear regression. Under some regular conditions, the theoretical properties of the proposed estimators are established. We further examine finite-sample performance of the proposed procedure by simulation studies and a real data example.     

Quantile regression for robust estimation and variable selection in partially linear varying-coefficient models

In this paper, we develop a new estimation procedure based on quantile regression for semiparametric partially linear varying-coefficient models. The proposed estimation approach is empirically shown to be much more efficient than the popular least squares estimation method for non-normal error distributions, and almost not lose any efficiency for normal errors. Asymptotic normalities of the proposed estimators for both the parametric and nonparametric parts are established. To achieve sparsity when there exist irrelevant variables in the model, two variable selection procedures based on adaptive penalty are developed to select important parametric covariates as well as significant nonparametric functions. Moreover, both these two variable selection procedures are demonstrated to enjoy the oracle property under some regularity conditions. Some Monte Carlo simulations are conducted to assess the finite sample performance of the proposed estimators, and a real-data example is used to illustrate the application of the proposed methods.     

Robust signed-rank estimation and variable selection for semi-parametric additive partial linear models

A fully nonparametric model may not perform well or when the researcher wants to use a parametric model but the functional form with respect to a subset of the regressors or the density of the errors is not known. This becomes even more challenging when the data contain gross outliers or unusual observations. However, in practice the true covariates are not known in advance, nor is the smoothness of the functional form. A robust model selection approach through which we can choose the relevant covariates components and estimate the smoothing function may represent an appealing tool to the solution. A weighted signed-rank estimation and variable selection under the adaptive lasso for semi-parametric partial additive models is considered in this paper. B-spline is used to estimate the unknown additive nonparametric function. It is shown that despite using B-spline to estimate the unknown additive nonparametric function, the proposed estimator has an oracle property. The robustness of the weighted signed-rank approach for data with heavy-tail, contaminated errors, and data containing high-leverage points are validated via finite sample simulations. A practical application to an economic study is provided using an updated Canadian household gasoline consumption data.     

Admissible linear estimation in singular linear models

The admissibility results of Rao (1976), proved in the context of a nonsingular covariance matrix, are exteneded to the situation where the covariance matrix is singular. Admi.s s Lb Le linear estimators in the Gauss-Markoff model are characterized and admis-sibility of the best linear unbiased estimator is investigated.     

Variable selection for generalized linear mixed models by L 1-penalized estimation

Generalized linear mixed models are a widely used tool for modeling longitudinal data. However, their use is typically restricted to few covariates, because the presence of many predictors yields unstable estimates. The presented approach to the fitting of generalized linear mixed models includes an L ₁-penalty term that enforces variable selection and shrinkage simultaneously. A gradient ascent algorithm is proposed that allows to maximize the penalized log-likelihood yielding models with reduced complexity. In contrast to common procedures it can be used in high-dimensional settings where a large number of potentially influential explanatory variables is available. The method is investigated in simulation studies and illustrated by use of real data sets.     

Robust adaptive model selection and estimation for partial linear varying coefficient models in rank regression

Partial linear varying coefficient models are often used in real data analysis for a good balance between flexibility and parsimony. In this paper, we propose a robust adaptive model selection method based on the rank regression, which can do simultaneous coefficient estimation and three types of selections, i.e., varying and constant effects selection, relevant variable selection. The new method has superiority in robustness and efficiency by inheriting the advantage of the rank regression approach. Furthermore, consistency in the three types of selections and oracle property in estimation are established as well. Simulation studies also confirm our method.     

Bootstrap variance and bias estimation in linear models

Let θ be a nonlinear function of the regression parameters and θ be its estimator based on the least-squares method. This paper studies the bootstrap estimators of the variance and bias of θ. The bootstrap estimators are shown to be consistent and asymptotically unbiased under some conditions. Asymptotic orders of the mean squared errors of the bootstrap estimators are also obtained. The bootstrap and the classical linearization method are compared in a simulation study. Discussions about when to use the bootstrap are given.     

Simultaneous estimation for semi-parametric multi-index models

Estimation of a general multi-index model comprises determining the number of linear combinations of predictors (structural dimension) that are related to the response, estimating the loadings of each index vector, selecting the active predictors and estimating the underlying link function. These objectives are often achieved sequentially at different stages of the estimation process. In this study, we propose a unified estimation approach under a semi-parametric model framework to attain these estimation goals simultaneously. The proposed estimation method is more efficient and stable than many existing methods where the estimation error in the structural dimension may propagate to the estimation of the index vectors and variable selection stages. A detailed algorithm is provided to implement the proposed method. Comprehensive simulations and a real data analysis illustrate the effectiveness of the proposed method.     

Adaptive estimation in partially linear autoregressive models

The authors consider a partially linear autoregressive model and construct kernel‐based estimates for both the parametric and nonparametric components. They propose an estimation procedure for the model and illustrate it through simulated and real data. Their work shows that the proposed estimation procedure not only has good asymptotic properties but also works well numerically. It also suggests that a partially linear autoregression is more appropriate than a completely nonparametric autoregression for some sets of data.     

Robust estimation in generalized linear mixed models

Generalized linear mixed models (GLMMs) are widely used to analyse non-normal response data with extra-variation, but non-robust estimators are still routinely used. We propose robust methods for maximum quasi-likelihood and residual maximum quasi-likelihood estimation to limit the influence of outlying observations in GLMMs. The estimation procedure parallels the development of robust estimation methods in linear mixed models, but with adjustments in the dependent variable and the variance component. The methods proposed are applied to three data sets and a comparison is made with the nonparametric maximum likelihood approach. When applied to a set of epileptic seizure data, the methods proposed have the desired effect of limiting the influence of outlying observations on the parameter estimates. Simulation shows that one of the residual maximum quasi-likelihood proposals has a smaller bias than those of the other estimation methods. We further discuss the equivalence of two GLMM formulations when the response variable follows an exponential family. Their extensions to robust GLMMs and their comparative advantages in modelling are described. Some possible modifications of the robust GLMM estimation methods are given to provide further flexibility for applying the method.     

Restricted minimax estimation in semiparametric linear models

Abstract

In this article we develop the minimax estimation approach of general linear models to the semiparametric linear models when the parameters are simultaneously constrained by an ellipsoid and linear restrictions. Combining sample information and prior constraints the minimax estimator is obtained and compared with partially least square estimator by theoretical and simulation methods.     

Efficient mean estimation in log-normal linear models

Log-normal linear models are widely used in applications, and many times it is of interest to predict the response variable or to estimate the mean of the response variable at the original scale for a new set of covariate values. In this paper we consider the problem of efficient estimation of the conditional mean of the response variable at the original scale for log-normal linear models. Several existing estimators are reviewed first, including the maximum likelihood (ML) estimator, the restricted ML (REML) estimator, the uniformly minimum variance unbiased (UMVU) estimator, and a bias-corrected REML estimator. We then propose two estimators that minimize the asymptotic mean squared error and the asymptotic bias, respectively. A parametric bootstrap procedure is also described to obtain confidence intervals for the proposed estimators. Both the new estimators and the bootstrap procedure are very easy to implement. Comparisons of the estimators using simulation studies suggest that our estimators perform better than the existing ones, and the bootstrap procedure yields confidence intervals with good coverage properties. A real application of estimating the mean sediment discharge is used to illustrate the methodology.