A Comparative Study of Semiparametric Estimation in Partially Linear Single-index Models

We consider a semiparametric method based on partial splines for estimating the unknown function and partially linear regression parameters in partially linear single-index models. Three methods—project pursuit regression (PPR), average derivative estimation (ADE), and a boosting method—are considered for estimating the single-index parameters. Simulations revealed that PPR with partial splines was superior in estimating single-index parameters, while the boosting method with partial splines performed no better than PPR and ADE. All three methods performed similarly in estimating the partially linear regression parameters. The relative performances of the methods are also illustrated using a real-world data example.     

Estimation of general semi-parametric quantile regression

Quantile regression introduced by Koenker and Bassett (1978) produces a comprehensive picture of a response variable on predictors. In this paper, we propose a general semi-parametric model of which part of predictors are presented with a single-index, to model the relationship of conditional quantiles of the response on predictors. Special cases are single-index models, partially linear single-index models and varying coefficient single-index models. We propose the qOPG, a quantile regression version of outer-product gradient estimation method (OPG, Xia et al., 2002) to estimate the single-index. Large-sample properties, simulation results and a real-data analysis are provided to examine the performance of the qOPG.     

Empirical Likelihood for Generalized Partially Linear Single-index Models

Empirical-likelihood based inference for the parameters in a generalized partially linear single-index models (GPLSIM) is investigated. Based on the local linear estimators of the nonparametric parts of the GPLSIM, an estimated empirical likelihood-based statistic of the parametric components is proposed. We show that the resulting statistic is asymptotically standard chi-squared distributed, the confidence regions for the parametric components are constructed. Some simulations are conducted to illustrate the proposed method.     

Smoothing Spline Estimation for Partially Linear Single-index Models

In this article, the partially linear single-index models are discussed based on smoothing spline and average derivative estimation method. This proposed technique consists of two stages: one is to estimate the vector parameter in the linear part using the smoothing cubic spline method, simultaneously, obtaining the estimator of unknown single-index function; the other is to estimate the single-index coefficients in the single-index part by the using average derivative estimator procedure. Some simulated and real examples are presented to illustrate the performance of this method.     

Composite quantile regression for varying-coefficient single-index models

ABSTRACT

The varying-coefficient single-index model (VCSIM) is a very general and flexible tool for exploring the relationship between a response variable and a set of predictors. Popular special cases include single-index models and varying-coefficient models. In order to estimate the index-coefficient and the non parametric varying-coefficients in the VCSIM, we propose a two-stage composite quantile regression estimation procedure, which integrates the local linear smoothing method and the information of quantile regressions at a number of conditional quantiles of the response variable. We establish the asymptotic properties of the proposed estimators for the index-coefficient and varying-coefficients when the error is heterogeneous. When compared with the existing mean-regression-based estimation method, our simulation results indicate that our proposed method has comparable performance for normal error and is more robust for error with outliers or heavy tail. We illustrate our methodologies with a real example.     

Statistical estimation for a partially linear single-index model with errors in all variables

This article considers partially linear single-index models with errors in all variables. By using the Pseudo ? θ method (Liang, Härdle, and Carroll 1999), local linear regression and simulation-extrapolation (SIMEX) technique (Cook and Stefanski 1994), we propose an efficient methodology to estimate the current model. Under certain conditions the asymptotic properties of proposed estimators are obtained. Some simulation experiments and an application are conducted to illustrate our proposed method.     

Partially linear single-index model with missing responses at random

This paper considers semiparametric partially linear single-index model with missing responses at random. Imputation approach is developed to estimate the regression coefficients, single-index coefficients and the nonparametric function, respectively. The imputation estimators for the regression coefficients and single-index coefficients are obtained by a stepwise approach. These estimators are shown to be asymptotically normal, and the estimator for the nonparametric function is proved to be asymptotically normal at any fixed point. The bandwidth problem is also considered in this paper, a delete-one cross validation method is used to select the optimal bandwidth. A simulation study is conducted to evaluate the proposed methods.     

Statistical inferences for partially linear single-index models with error-prone linear covariates

We consider statistical inference for partially linear single-index models (PLSIM) when some linear covariates are not observed, but ancillary variables are available. Based on the profile least-squared estimators of the unknowns, we study the testing problems for parametric components in the proposed models. It is to see whether the generalized likelihood ratio (GLR) tests proposed by Fan et al. (2001) are applicable to testing for the parametric components. We show that under the null hypothesis the proposed GLR statistics follow asymptotically the χ²-distributions with the scale constants and the degrees of freedom being independent of the nuisance parameters or functions, which is called the Wilks phenomenon. Simulated experiments are conducted to illustrate our proposed methodology.     

ESTIMATION AND TESTING FOR PARTIALLY LINEAR SINGLE-INDEX MODELS

In partially linear single-index models, we obtain the semiparametrically efficient profile least-squares estimators of regression coefficients. We also employ the smoothly clipped absolute deviation penalty (SCAD) approach to simultaneously select variables and estimate regression coefficients. We show that the resulting SCAD estimators are consistent and possess the oracle property. Subsequently, we demonstrate that a proposed tuning parameter selector, BIC, identifies the true model consistently. Finally, we develop a linear hypothesis test for the parametric coefficients and a goodness-of-fit test for the nonparametric component, respectively. Monte Carlo studies are also presented.     

Results concerning the generalized partially linear single-index model

The central topic of this article is the estimation of parameters of the generalized partially linear single-index model (GPLSIM). Two numerical optimization procedures are presented and an S-plus program based on these procedures is compared to a program by Wand in a simulation setting. The results from these simulations indicate that the estimates for the new procedures are as good, if not better, than Wand's. Also, this program is much more flexible than Wand's since it can handle more general models. Other simulations are also conducted. The first compares the effects of using linear interpolation versus spline interpolation in an optimization procedure. The results indicate that by using spline interpolation one gets more stable estimates at a cost of increased computational time. A second simulation was conducted to assess the performance of a method for estimating the variance of alpha. A third set of simulations is carried out to determine the best criterion for testing that one of the elements of alpha is equal to zero. The GPLSIM is applied to a water quality data set and the results indicate an interesting relationship between gastrointestinal illness and turbidity (cloudiness) of drinking water.     

Partial least squares estimator for single-index models

The partial least squares (PLS) approach first constructs new explanatory variables, known as factors (or components), which are linear combinations of available predictor variables. A small subset of these factors is then chosen and retained for prediction. We study the performance of PLS in estimating single-index models, especially when the predictor variables exhibit high collinearity. We show that PLS estimates are consistent up to a constant of proportionality. We present three simulation studies that compare the performance of PLS in estimating single-index models with that of sliced inverse regression (SIR). In the first two studies, we find that PLS performs better than SIR when collinearity exists. In the third study, we learn that PLS performs well even when there are multiple dependent variables, the link function is non-linear and the shape of the functional form is not known.     

Functional single-index quantile regression models

It is known that functional single-index regression models can achieve better prediction accuracy than functional linear models or fully nonparametric models, when the target is to predict a scalar response using a function-valued covariate. However, the performance of these models may be adversely affected by extremely large values or skewness in the response. In addition, they are not able to offer a full picture of the conditional distribution of the response. Motivated by using trajectories of $$\hbox {PM}_{{10}}$$ concentrations of last day to predict the maximum $$\hbox {PM}_{{10}}$$ concentration of the current day, a functional single-index quantile regression model is proposed to address those issues. A generalized profiling method is employed to estimate the model. Simulation studies are conducted to investigate the finite sample performance of the proposed estimator. We apply the proposed framework to predict the maximal value of $$\hbox {PM}_{{10}}$$ concentrations based on the intraday $$\hbox {PM}_{{10}}$$ concentrations of the previous day.     

The single-index support vector regression model to address the problem of high dimensionality

ABSTRACT

The last few years, the applications of Support Vector Machine (SVM) for solving classification and regression problems have been increasing, due to its high performance and ability to transform the non-linear relationships among variables to linear form by employing the kernel idea (kernel function). In this work, we develop a semi-parametric approach to fit single-index models to deal with high-dimensional problems. To achieve this goal, we use support vector regression (SVR) for estimating the unknown nonparametric link function, while the single-index is determined by using the semi-parametric least squares method (Ichimura 1993). This development enhances the ability of SVR to solve high-dimensional problem. We design a three simulation examples with high-dimensional problems (linear and nonlinear). The simulations demonstrate the superior performance of the proposed method versus the standard SVR method. This is further illustrated by applying the real data.     

Estimation and variable selection for generalised partially linear single-index models

In this paper, we study the problem of estimation and variable selection for generalised partially linear single-index models based on quasi-likelihood, extending existing studies on variable selection for partially linear single-index models to binary and count responses. To take into account the unit norm constraint of the index parameter, we use the ‘delete-one-component’ approach. The asymptotic normality of the estimates is demonstrated. Furthermore, the smoothly clipped absolute deviation penalty is added for variable selection of parameters both in the nonparametric part and the parametric part, and the oracle property of the variable selection procedure is shown. Finally, some simulation studies are carried out to illustrate the finite sample performance.     

Estimation and variable selection in single-index composite quantile regression

In this article, a new composite quantile regression estimation approach is proposed for estimating the parametric part of single-index model. We use local linear composite quantile regression (CQR) for estimating the nonparametric part of single-index model (SIM) when the error distribution is symmetrical. The weighted local linear CQR is proposed for estimating the nonparametric part of SIM when the error distribution is asymmetrical. Moreover, a new variable selection procedure is proposed for SIM. Under some regularity conditions, we establish the large sample properties of the proposed estimators. Simulation studies and a real data analysis are presented to illustrate the behavior of the proposed estimators.     

Generalized Analysis-of-variance-type Test for the Single-index Quantile Model

In this paper, we are concerned with a test for the index parameter and index function in the single-index model. Based on the estimates obtained by the quantile regression, we extend the generalized analysis-of-variance-type test to the single-index model. We investigate the asymptotic behavior of the proposed test and demonstrate that its limiting null distribution follows an asymptotically χ²-distribution. The simulation studies and real data applications are conducted to illustrate the finite sample performance of the proposed methods.     

Estimation for Partially Linear Single-index Instrumental Variables Models

In this article, we generalize the partially linear single-index models to the scenario with some endogenous covariates variables. It is well known that the estimators based on the existing methods are often inconsistent because of the endogeneity of covariates. To deal with the endogenous variables, we introduce some auxiliary instrumental variables. A three-stage estimation procedure is proposed for partially linear single-index instrumental variables models. The first stage is to obtain a linear projection of endogenous variables on a set of instrumental variables, the second stage is to estimate the link function by using local linear smoother for given constant parameters, and the last stage is to obtain the estimators of constant parameters based on the estimating equation. Asymptotic normality is established for the proposed estimators. Some simulation studies are undertaken to assess the finite sample performance of the proposed estimation procedure.     

Single-index regression models with right-censored responses

In this article, we propose some new generalizations of M-estimation procedures for single-index regression models in presence of randomly right-censored responses. We derive consistency and asymptotic normality of our estimates. The results are proved in order to be adapted to a wide range of techniques used in a censored regression framework (e.g. synthetic data or weighted least squares). As in the uncensored case, the estimator of the single-index parameter is seen to have the same asymptotic behavior as in a fully parametric scheme. We compare these new estimators with those based on the average derivative technique of Lu and Burke [2005. Censored multiple regression by the method of average derivatives. J. Multivariate Anal. 95, 182–205] through a simulation study.     

Single-index modelling of conditional probabilities in two-way contingency tables

When analysing a contingency table, it is often worth relating the probabilities that a given individual falls into different cells from a set of predictors. These conditional probabilities are usually estimated using appropriate regression techniques. In particular, in this paper, a semiparametric model is developed. Essentially, it is only assumed that the effect of the vector of covariates on the probabilities can entirely be captured by a single index, which is a linear combination of the initial covariates. The estimation is then twofold: the coefficients of the linear combination and the functions linking this index to the related conditional probabilities have to be estimated. Inspired by the estimation procedures already proposed in the literature for single-index regression models, four estimators of the index coefficients are proposed and compared, from a theoretical point-of-view, but also practically, with the aid of simulations. Estimation of the link functions is also addressed.     

Robust estimation for partially linear single-index model

In this article, we investigate a new estimation approach for the partially linear single-index model based on modal regression method, where the non parametric function is estimated by penalized spline method. Moreover, we develop an expection maximum (EM)-type algorithm and establish the large sample properties of the proposed estimation method. A distinguishing characteristic of the newly proposed estimation is robust against outliers through introducing an additional tuning parameter which can be automatically selected using the observed data. Simulation studies and real data example are used to evaluate the finite-sample performance, and the results show that the newly proposed method works very well.