NEW EFFICIENT ESTIMATION AND VARIABLE SELECTION METHODS FOR SEMIPARAMETRIC VARYING-COEFFICIENT PARTIALLY LINEAR MODELS

The complexity of semiparametric models poses new challenges to statistical inference and model selection that frequently arise from real applications. In this work, we propose new estimation and variable selection procedures for the semiparametric varying-coefficient partially linear model. We first study quantile regression estimates for the nonparametric varying-coefficient functions and the parametric regression coefficients. To achieve nice efficiency properties, we further develop a semiparametric composite quantile regression procedure. We establish the asymptotic normality of proposed estimators for both the parametric and nonparametric parts and show that the estimators achieve the best convergence rate. Moreover, we show that the proposed method is much more efficient than the least-squares-based method for many non-normal errors and that it only loses a small amount of efficiency for normal errors. In addition, it is shown that the loss in efficiency is at most 11.1% for estimating varying coefficient functions and is no greater than 13.6% for estimating parametric components. To achieve sparsity with high-dimensional covariates, we propose adaptive penalization methods for variable selection in the semiparametric varying-coefficient partially linear model and prove that the methods possess the oracle property. Extensive Monte Carlo simulation studies are conducted to examine the finite-sample performance of the proposed procedures. Finally, we apply the new methods to analyze the plasma beta-carotene level data.     

Bayesian spectral analysis models for quantile regression with Dirichlet process mixtures

This paper presents a Bayesian analysis of partially linear additive models for quantile regression. We develop a semiparametric Bayesian approach to quantile regression models using a spectral representation of the nonparametric regression functions and the Dirichlet process (DP) mixture for error distribution. We also consider Bayesian variable selection procedures for both parametric and nonparametric components in a partially linear additive model structure based on the Bayesian shrinkage priors via a stochastic search algorithm. Based on the proposed Bayesian semiparametric additive quantile regression model referred to as BSAQ, the Bayesian inference is considered for estimation and model selection. For the posterior computation, we design a simple and efficient Gibbs sampler based on a location-scale mixture of exponential and normal distributions for an asymmetric Laplace distribution, which facilitates the commonly used collapsed Gibbs sampling algorithms for the DP mixture models. Additionally, we discuss the asymptotic property of the sempiparametric quantile regression model in terms of consistency of posterior distribution. Simulation studies and real data application examples illustrate the proposed method and compare it with Bayesian quantile regression methods in the literature.     

Adaptive estimation in partially linear autoregressive models

The authors consider a partially linear autoregressive model and construct kernel‐based estimates for both the parametric and nonparametric components. They propose an estimation procedure for the model and illustrate it through simulated and real data. Their work shows that the proposed estimation procedure not only has good asymptotic properties but also works well numerically. It also suggests that a partially linear autoregression is more appropriate than a completely nonparametric autoregression for some sets of data.     

Semiparametric statistical inferences for longitudinal data with nonparametric covariance modelling

In this paper, semiparametric modelling for longitudinal data with an unstructured error process is considered. We propose a partially linear additive regression model for longitudinal data in which within-subject variances and covariances of the error process are described by unknown univariate and bivariate functions, respectively. We provide an estimating approach in which polynomial splines are used to approximate the additive nonparametric components and the within-subject variance and covariance functions are estimated nonparametrically. Both the asymptotic normality of the resulting parametric component estimators and optimal convergence rate of the resulting nonparametric component estimators are established. In addition, we develop a variable selection procedure to identify significant parametric and nonparametric components simultaneously. We show that the proposed SCAD penalty-based estimators of non-zero components have an oracle property. Some simulation studies are conducted to examine the finite-sample performance of the proposed estimation and variable selection procedures. A real data set is also analysed to demonstrate the usefulness of the proposed method.     

Outlier Resistant Estimation in Difference-Based Semiparametric Partially Linear Models

Partially linear models are extensions of linear models that include a nonparametric function of some covariate allowing an adequate and more flexible handling of explanatory variables than in linear models. The difference-based estimation in partially linear models is an approach designed to estimate parametric component by using the ordinary least squares estimator after removing the nonparametric component from the model by differencing. However, it is known that least squares estimates do not provide useful information for the majority of data when the error distribution is not normal, particularly when the errors are heavy-tailed and when outliers are present in the dataset. This paper aims to find an outlier-resistant fit that represents the information in the majority of the data by robustly estimating the parametric and the nonparametric components of the partially linear model. Simulations and a real data example are used to illustrate the feasibility of the proposed methodology and to compare it with the classical difference-based estimator when outliers exist.     

Robust estimation and wavelet thresholding in partially linear models

This paper is concerned with a semiparametric partially linear regression model with unknown regression coefficients, an unknown nonparametric function for the non-linear component, and unobservable Gaussian distributed random errors. We present a wavelet thresholding based estimation procedure to estimate the components of the partial linear model by establishing a connection between an l ₁-penalty based wavelet estimator of the nonparametric component and Huber’s M-estimation of a standard linear model with outliers. Some general results on the large sample properties of the estimates of both the parametric and the nonparametric part of the model are established. Simulations are used to illustrate the general results and to compare the proposed methodology with other methods available in the recent literature.     

A class of partially linear single‐index survival models

The authors define a class of “partially linear single‐index” survival models that are more flexible than the classical proportional hazards regression models in their treatment of covariates. The latter enter the proposed model either via a parametric linear form or a nonparametric single‐index form. It is then possible to model both linear and functional effects of covariates on the logarithm of the hazard function and if necessary, to reduce the dimensionality of multiple covariates via the single‐index component. The partially linear hazards model and the single‐index hazards model are special cases of the proposed model. The authors develop a likelihood‐based inference to estimate the model components via an iterative algorithm. They establish an asymptotic distribution theory for the proposed estimators, examine their finite‐sample behaviour through simulation, and use a set of real data to illustrate their approach.     

S-estimator in partially linear regression models

In this paper, a robust estimator is proposed for partially linear regression models. We first estimate the nonparametric component using the penalized regression spline, then we construct an estimator of parametric component by using robust S-estimator. We propose an iterative algorithm to solve the proposed optimization problem, and introduce a robust generalized cross-validation to select the penalized parameter. Simulation studies and a real data analysis illustrate that the our proposed method is robust against outliers in the dataset or errors with heavy tails.     

Estimation and variable selection in single-index composite quantile regression

In this article, a new composite quantile regression estimation approach is proposed for estimating the parametric part of single-index model. We use local linear composite quantile regression (CQR) for estimating the nonparametric part of single-index model (SIM) when the error distribution is symmetrical. The weighted local linear CQR is proposed for estimating the nonparametric part of SIM when the error distribution is asymmetrical. Moreover, a new variable selection procedure is proposed for SIM. Under some regularity conditions, we establish the large sample properties of the proposed estimators. Simulation studies and a real data analysis are presented to illustrate the behavior of the proposed estimators.     

A new orthogonality-based estimation for varying-coefficient partially linear models

Varying coefficient partially linear models are usually used for longitudinal data analysis, and an interest is mainly to improve efficiency of regression coefficients. By the orthogonality estimation technology and the quadratic inference function method, we propose a new orthogonality-based estimation method to estimate parameter and nonparametric components in varying coefficient partially linear models with longitudinal data. The proposed procedure can separately estimate the parametric and nonparametric components, and the resulting estimators do not affect each other. Under some mild conditions, we establish some asymptotic properties of the resulting estimators. Furthermore, the finite sample performance of the proposed procedure is assessed by some simulation experiments.     

Restricted Estimation in Partially Linear Errors-in-Variables Models

As a compromise between parametric regression and nonparametric regression, partially linear models are frequently used in statistical modelling. This article considers statistical inference for this semiparametric model when the linear covariate is measured with additive error and some additional linear restrictions on the parametric component are assumed to hold. We propose a restricted corrected profile least-squares estimator for the parametric component, and study the asymptotic normality of the estimator. To test hypothesis on the parametric component, we construct a Wald test statistic and obtain its limiting distribution. Some simulation studies are conducted to illustrate our approaches.     

An overview of model-robust regression

The proper combination of parametric and nonparametric regression procedures can improve upon the shortcomings of each when used individually. Considered is the situation where the researcher has an idea of which parametric model should explain the behavior of the data, but this model is not adequate throughout the entire range of the data. An extension of partial linear regression and two other methods of model-robust regression are developed and compared in this context. The model-robust procedures each involve the proportional mixing of a parametric fit to the data and a nonparametric fit to either the data or residuals. The emphasis of this work is on fitting in the small-sample situation, where nonparametric regression alone has well-known inadequacies. Performance is based on bias and variance considerations, and theoretical mean squared error formulas are developed for each procedure. An example is given that uses generated data from an underlying model with defined misspecification to provide graphical comparisons of the fits and to show the theoretical benefits of the model-robust procedures. Simulation results are presented which establish the accuracy of the theoretical formulas and illustrate the potential benefits of the model-robust procedures. Simulations are also used to illustrate the advantageous properties of a data-driven selector developed in this work for choosing the smoothing and mixing parameters. It is seen that the model-robust procedures (the final proposed method, in particular) give much improved fits over the individual parametric and nonparametric fits.     

Quantile regression for robust estimation and variable selection in partially linear varying-coefficient models

In this paper, we develop a new estimation procedure based on quantile regression for semiparametric partially linear varying-coefficient models. The proposed estimation approach is empirically shown to be much more efficient than the popular least squares estimation method for non-normal error distributions, and almost not lose any efficiency for normal errors. Asymptotic normalities of the proposed estimators for both the parametric and nonparametric parts are established. To achieve sparsity when there exist irrelevant variables in the model, two variable selection procedures based on adaptive penalty are developed to select important parametric covariates as well as significant nonparametric functions. Moreover, both these two variable selection procedures are demonstrated to enjoy the oracle property under some regularity conditions. Some Monte Carlo simulations are conducted to assess the finite sample performance of the proposed estimators, and a real-data example is used to illustrate the application of the proposed methods.     

Partially Linear Additive Models with Unknown Link Functions

In this paper, we consider partially linear additive models with an unknown link function, which include single‐index models and additive models as special cases. We use polynomial spline method for estimating the unknown link function as well as the component functions in the additive part. We establish that convergence rates for all nonparametric functions are the same as in one‐dimensional nonparametric regression. For a faster rate of the parametric part, we need to define appropriate ‘projection’ that is more complicated than that defined previously for partially linear additive models. Compared to previous approaches, a distinct advantage of our estimation approach in implementation is that estimation directly reduces estimation in the single‐index model and can thus deal with much larger dimensional problems than previous approaches for additive models with unknown link functions. Simulations and a real dataset are used to illustrate the proposed model.     

Quantile regression estimation of partially linear additive models

In this paper, we consider the estimation of partially linear additive quantile regression models where the conditional quantile function comprises a linear parametric component and a nonparametric additive component. We propose a two-step estimation approach: in the first step, we approximate the conditional quantile function using a series estimation method. In the second step, the nonparametric additive component is recovered using either a local polynomial estimator or a weighted Nadaraya–Watson estimator. Both consistency and asymptotic normality of the proposed estimators are established. Particularly, we show that the first-stage estimator for the finite-dimensional parameters attains the semiparametric efficiency bound under homoskedasticity, and that the second-stage estimators for the nonparametric additive component have an oracle efficiency property. Monte Carlo experiments are conducted to assess the finite sample performance of the proposed estimators. An application to a real data set is also illustrated.     

Estimation of partially linear single-index additive hazards model with current status data

In this paper, we introduce a partially linear single-index additive hazards model with current status data. Both the unknown link function of the single-index term and the cumulative baseline hazard function are approximated by B-splines under a monotonicity constraint on the latter. The sieve method is applied to estimate the nonparametric and parametric components simultaneously. We show that, when the nonparametric link function is an exact B-spline, the resultant estimator of regression parameter vector is asymptotically normal and achieves the semiparametric information bound and the rate of convergence of the estimator for the cumulative baseline hazard function is optimal. Simulation studies are presented to examine the finite sample performance of the proposed estimation method. For illustration, we apply the method to a clinical dataset with current status outcome.     

Variable selection for semiparametric varying coefficient partially linear model based on modal regression with missing data

Abstract

In this article, we focus on the variable selection for semiparametric varying coefficient partially linear model with response missing at random. Variable selection is proposed based on modal regression, where the non parametric functions are approximated by B-spline basis. The proposed procedure uses SCAD penalty to realize variable selection of parametric and nonparametric components simultaneously. Furthermore, we establish the consistency, the sparse property and asymptotic normality of the resulting estimators. The penalty estimation parameters value of the proposed method is calculated by EM algorithm. Simulation studies are carried out to assess the finite sample performance of the proposed variable selection procedure.     

ESTIMATION AND TESTING FOR PARTIALLY LINEAR SINGLE-INDEX MODELS

In partially linear single-index models, we obtain the semiparametrically efficient profile least-squares estimators of regression coefficients. We also employ the smoothly clipped absolute deviation penalty (SCAD) approach to simultaneously select variables and estimate regression coefficients. We show that the resulting SCAD estimators are consistent and possess the oracle property. Subsequently, we demonstrate that a proposed tuning parameter selector, BIC, identifies the true model consistently. Finally, we develop a linear hypothesis test for the parametric coefficients and a goodness-of-fit test for the nonparametric component, respectively. Monte Carlo studies are also presented.     

Principal components regression estimator of the parameters in partially linear models

ABSTRACT

As a compromise between parametric regression and non-parametric regression models, partially linear models are frequently used in statistical modelling. This paper is concerned with the estimation of partially linear regression model in the presence of multicollinearity. Based on the profile least-squares approach, we propose a novel principal components regression (PCR) estimator for the parametric component. When some additional linear restrictions on the parametric component are available, we construct a corresponding restricted PCR estimator. Some simulations are conducted to examine the performance of our proposed estimators and the results are satisfactory. Finally, a real data example is analysed.     

Statistical inference for multivariate partially linear regression models

In this paper we study a class of multivariate partially linear regression models. Various estimators for the parametric component and the nonparametric component are constructed and their asymptotic normality established. In particular, we propose an estimator of the contemporaneous correlation among the multiple responses and develop a test for detecting the existence of such contemporaneous correlation without using any nonparametric estimation. The performance of the proposed estimators and test is evaluated through some simulation studies and an analysis of a real data set is used to illustrate the developed methodology. The Canadian Journal of Statistics 41: 1–22; 2013 © 2013 Statistical Society of Canada