Shrinkage ridge regression in partial linear models

ABSTRACT

In this paper, shrinkage ridge estimator and its positive part are defined for the regression coefficient vector in a partial linear model. The differencing approach is used to enjoy the ease of parameter estimation after removing the non parametric part of the model. The exact risk expressions in addition to biases are derived for the estimators under study and the region of optimality of each estimator is exactly determined. The performance of the estimators is evaluated by simulated as well as real data sets.     

Robust estimation and wavelet thresholding in partially linear models

This paper is concerned with a semiparametric partially linear regression model with unknown regression coefficients, an unknown nonparametric function for the non-linear component, and unobservable Gaussian distributed random errors. We present a wavelet thresholding based estimation procedure to estimate the components of the partial linear model by establishing a connection between an l ₁-penalty based wavelet estimator of the nonparametric component and Huber’s M-estimation of a standard linear model with outliers. Some general results on the large sample properties of the estimates of both the parametric and the nonparametric part of the model are established. Simulations are used to illustrate the general results and to compare the proposed methodology with other methods available in the recent literature.     

Semiparametric Ridge Regression Approach in Partially Linear Models

In this article, we introduce a semiparametric ridge regression estimator for the vector-parameter in a partial linear model. It is also assumed that some additional artificial linear restrictions are imposed to the whole parameter space and the errors are dependent. This estimator is a generalization of the well-known restricted least-squares estimator and is confined to the (affine) subspace which is generated by the restrictions. Asymptotic distributional bias and risk are also derived and the comparison result is then given.     

Efficiency of the generalized difference-based Liu estimators in semiparametric regression models with correlated errors

In this paper, a generalized difference-based estimator is introduced for the vector parameter β in the semiparametric regression model when the errors are correlated. A generalized difference-based Liu estimator is defined for the vector parameter β in the semiparametric regression model. Under the linear nonstochastic constraint Rβ=r, the generalized restricted difference-based Liu estimator is given. The risk function for the β?_GRD(η) associated with weighted balanced loss function is presented. The performance of the proposed estimators is evaluated by a simulated data set.     

Semiparametric Efficient Estimation of the Mean of a Time Series in the Presence of Conditional Heterogeneity of Unknown Form

Abstract

We obtain semiparametric efficiency bounds for estimation of a location parameter in a time series model where the innovations are stationary and ergodic conditionally symmetric martingale differences but otherwise possess general dependence and distributions of unknown form. We then describe an iterative estimator that achieves this bound when the conditional density functions of the sample are known. Finally, we develop a “semi-adaptive” estimator that achieves the bound when these densities are unknown by the investigator. This estimator employs nonparametric kernel estimates of the densities. Monte Carlo results are reported.     

Oracally efficient spline-backfitted kernel smoothing of additive partial linear measurement error model

Abstract

We consider statistical inference for additive partial linear models when the linear covariate is measured with error. A bias-corrected spline-backfitted kernel smoothing method is proposed. Under mild assumptions, the proposed component function and parameter estimator are oracally efficient and fast to compute. The nonparametric function estimator’s pointwise distribution is asymptotically equivalent to an function estimator in partial linear model. Finite-sample performance of the proposed estimators is assessed by simulation experiments. The proposed methods are applied to Boston house data set.     

Restricted Ridge Estimators of the Parameters in Semiparametric Regression Model

In this article, we introduce a ridge estimator for the vector of parameters β in a semiparametric model when additional linear restrictions on the parameter vector are assumed to hold. We also obtain the semiparametric restricted ridge estimator for the parametric component in the semiparametric regression model. The ideas in this article are illustrated with a data set consisting of housing prices and through a comparison of the performances of the proposed and related estimators via a Monte Carlo simulation.     

Feasible robust estimator in restricted semiparametric regression models based on the LTS approach

Under some nonstochastic linear restrictions based on either additional information or prior knowledge in a semiparametric regression model, a family of feasible generalized robust estimators for the regression parameter is proposed. The least trimmed squares (LTS) method proposed by Rousseeuw as a highly robust regression estimator is a statistical technique for fitting a regression model based on the subset of h observations (out of n) whose least-square fit possesses the smallest sum of squared residuals. The coverage h may be set between n/2 and n. The LTS estimator involves computing the hyperplane that minimizes the sum of the smallest h squared residuals. For practical purpose, it is assumed that the covariance matrix of the error term is unknown and thus feasible estimators are replaced. Then, we develop an algorithm for the LTS estimator based on feasible methods. Through the Monte Carlo simulation studies and a real data example, performance of the feasible type of robust estimators is compared with the classical ones in restricted semiparametric regression models.     

Positive-rule stein-type almost unbiased ridge estimator in linear regression model

Abstract

In this article, when it is suspected that regression coefficients may be restricted to a subspace, we discuss the parameter estimation of regression coefficients in a multiple regression model. Then, in order to improve the preliminary test almost ridge estimator, we study the positive-rule Stein-type almost unbiased ridge estimator based on the positive-rule stein-type shrinkage estimator and almost unbiased ridge estimator. After that, quadratic bias and quadratic risk values of the new estimator are derived and compared with some relative estimators. And we also discuss the option of parameter k. Finally, we perform a real data example and a Monte Carlo study to illustrate theoretical results.     

SHRINKAGE, PRETEST AND ABSOLUTE PENALTY ESTIMATORS IN PARTIALLY LINEAR MODELS

We consider a partially linear model in which the vector of coefficients β in the linear part can be partitioned as ( β ₁, β ₂) , where β ₁ is the coefficient vector for main effects (e.g. treatment effect, genetic effects) and β ₂ is a vector for ‘nuisance’ effects (e.g. age, laboratory). In this situation, inference about β ₁ may benefit from moving the least squares estimate for the full model in the direction of the least squares estimate without the nuisance variables (Steinian shrinkage), or from dropping the nuisance variables if there is evidence that they do not provide useful information (pretesting). We investigate the asymptotic properties of Stein‐type and pretest semiparametric estimators under quadratic loss and show that, under general conditions, a Stein‐type semiparametric estimator improves on the full model conventional semiparametric least squares estimator. The relative performance of the estimators is examined using asymptotic analysis of quadratic risk functions and it is found that the Stein‐type estimator outperforms the full model estimator uniformly. By contrast, the pretest estimator dominates the least squares estimator only in a small part of the parameter space, which is consistent with the theory. We also consider an absolute penalty‐type estimator for partially linear models and give a Monte Carlo simulation comparison of shrinkage, pretest and the absolute penalty‐type estimators. The comparison shows that the shrinkage method performs better than the absolute penalty‐type estimation method when the dimension of the β ₂ parameter space is large.     

On Semiparametric Mode Regression Estimation

It has been found that, for a variety of probability distributions, there is a surprising linear relation between mode, mean, and median. In this article, the relation between mode, mean, and median regression functions is assumed to follow a simple parametric model. We propose a semiparametric conditional mode (mode regression) estimation for an unknown (unimodal) conditional distribution function in the context of regression model, so that any m-step-ahead mean and median forecasts can then be substituted into the resultant model to deliver m-step-ahead mode prediction. In the semiparametric model, Least Squared Estimator (LSEs) for the model parameters and the simultaneous estimation of the unknown mean and median regression functions by the local linear kernel method are combined to infer about the parametric and nonparametric components of the proposed model. The asymptotic normality of these estimators is derived, and the asymptotic distribution of the parameter estimates is also given and is shown to follow usual parametric rates in spite of the presence of the nonparametric component in the model. These results are applied to obtain a data-based test for the dependence of mode regression over mean and median regression under a regression model.     

A note on the asymptotic relative efficiency of estimators from the additive hazards model

In this note, the asymptotic variance formulas are explicitly derived and compared between the parametric and semiparametric estimators of a regression parameter and survival probability under the additive hazards model. To obtain explicit formulas, it is assumed that the covariate term including a regression coefficient follows a gamma distribution and the baseline hazard function is constant. The results show that the semiparametric estimator of the regression coefficient parameter is fully efficient relative to the parametric counterpart when the survival time and a covariate are independent, as in the proportional hazards model. Relative to a more realistic case of the parametric additive hazards model with a Weibull baseline, the loss of efficiency of the semiparametric estimator of survival probability is moderate.     

Semiparametric multiple kernel estimators and model diagnostics for count regression functions

Abstract

This study concerns semiparametric approaches to estimate discrete multivariate count regression functions. The semiparametric approaches investigated consist of combining discrete multivariate nonparametric kernel and parametric estimations such that (i) a prior knowledge of the conditional distribution of model response may be incorporated and (ii) the bias of the traditional nonparametric kernel regression estimator of Nadaraya-Watson may be reduced. We are precisely interested in combination of the two estimations approaches with some asymptotic properties of the resulting estimators. Asymptotic normality results were showed for nonparametric correction terms of parametric start function of the estimators. The performance of discrete semiparametric multivariate kernel estimators studied is illustrated using simulations and real count data. In addition, diagnostic checks are performed to test the adequacy of the parametric start model to the true discrete regression model. Finally, using discrete semiparametric multivariate kernel estimators provides a bias reduction when the parametric multivariate regression model used as start regression function belongs to a neighborhood of the true regression model.     

New Ridge Regression Estimator in Semiparametric Regression Models

In the context of ridge regression, the estimation of shrinkage parameter plays an important role in analyzing data. Many efforts have been put to develop the computation of risk function in different full-parametric ridge regression approaches using eigenvalues and then bringing an efficient estimator of shrinkage parameter based on them. In this respect, the estimation of shrinkage parameter is neglected for semiparametric regression model. Not restricted, but the main focus of this approach is to develop necessary tools for computing the risk function of regression coefficient based on the eigenvalues of design matrix in semiparametric regression. For this purpose the differencing methodology is applied. We also propose a new estimator for shrinkage parameter which is of harmonic type mean of ridge estimators. It is shown that this estimator performs better than all the existing ones for the regression coefficient. For our proposal, a Monte Carlo simulation study and a real dataset analysis related to housing attributes are conducted to illustrate the efficiency of shrinkage estimators based on the minimum risk and mean squared error criteria.     

Lack-of-Fit Tests for Generalized Linear Models via Splines

Cubic B-splines are used to estimate the nonparametric component of a semiparametric generalized linear model. A penalized log-likelihood ratio test statistic is constructed for the null hypothesis of the linearity of the nonparametric function. When the number of knots is fixed, its limiting null distribution is the distribution of a linear combination of independent chi-squared random variables, each with one df. The smoothing parameter is determined by giving a specified value for its asymptotically expected value under the null hypothesis. A simulation study is conducted to evaluate its power performance; a real-life dataset is used to illustrate its practical use.     

Cross validation of ridge regression estimator in autocorrelated linear regression models

ABSTRACT

In this paper, we investigated the cross validation measures, namely OCV, GCV and Cp under the linear regression models when the error structure is autocorrelated and regressor data are correlated. The best performed ridge regression estimator is obtained by getting the optimal ridge parameter so as to minimize these measures. A Monte Carlo simulation study is given to see how the optimal ridge parameter is affected by autocorrelation and the strength of multicollinearity.     

Restricted minimax estimation in semiparametric linear models

Abstract

In this article we develop the minimax estimation approach of general linear models to the semiparametric linear models when the parameters are simultaneously constrained by an ellipsoid and linear restrictions. Combining sample information and prior constraints the minimax estimator is obtained and compared with partially least square estimator by theoretical and simulation methods.     

Semiparametric Analysis of Truncated Data

Randomly truncated data are frequently encountered in many studies where truncation arises as a result of the sampling design. In the literature, nonparametric and semiparametric methods have been proposed to estimate parameters in one-sample models. This paper considers a semiparametric model and develops an efficient method for the estimation of unknown parameters. The model assumes that K populations have a common probability distribution but the populations are observed subject to different truncation mechanisms. Semiparametric likelihood estimation is studied and the corresponding inferences are derived for both parametric and nonparametric components in the model. The method can also be applied to two-sample problems to test the difference of lifetime distributions. Simulation results and a real data analysis are presented to illustrate the methods.     

Quantile regression in functional linear semiparametric model

This paper proposes nonparametric estimation methods for functional linear semiparametric quantile regression, where the conditional quantile of the scalar responses is modelled by both scalar and functional covariates and an additional unknown nonparametric function term. The slope function is estimated using the functional principal component basis and the nonparametric function is approximated by a piecewise polynomial function. The asymptotic distribution of the estimators of slope parameters is derived and the global convergence rate of the quantile estimator of unknown slope function is established under suitable norm. The asymptotic distribution of the estimator of the unknown nonparametric function is also established. Simulation studies are conducted to investigate the finite-sample performance of the proposed estimators. The proposed methodology is demonstrated by analysing a real data from ADHD-200 sample.