On shrinkage r-estimation in a multiple regression model

For a multiple regression model, bearing the plausibility of a subset of the regression parameters being close to a pivot, for the complementary subset, based on the usual James-Stein rule, a general formulation of shrinkage R-estimation is considered. In the light of asymptotic distributional risks of estimators, performance characteristics ( under local alternatives) of the classical R-est-imator and its preliminary test and shrinkage versions (all based on the common score function ) are studied. These shed light on the relative dominance picture in a meaningful asymptotic setup.     

Tests of homogeneity of intercepts after a preliminary test

For the multl-sample regression model , the problem of testing equality of intercepts following a preliminarytest on the equality o f slopes is considered. The t e s t s are based on the conventional least squares estimators (without necessarily assuming the normality of the underlying error distributions ) , and the impact of a

preliminary test on the asymptotic size and power of the final test is studied.     

LASSO and shrinkage estimation in Weibull censored regression models

In this paper we address the problem of estimating a vector of regression parameters in the Weibull censored regression model. Our main objective is to provide natural adaptive estimators that significantly improve upon the classical procedures in the situation where some of the predictors may or may not be associated with the response. In the context of two competing Weibull censored regression models (full model and candidate submodel), we consider an adaptive shrinkage estimation strategy that shrinks the full model maximum likelihood estimate in the direction of the submodel maximum likelihood estimate. We develop the properties of these estimators using the notion of asymptotic distributional risk. The shrinkage estimators are shown to have higher efficiency than the classical estimators for a wide class of models. Further, we consider a LASSO type estimation strategy and compare the relative performance with the shrinkage estimators. Monte Carlo simulations reveal that when the true model is close to the candidate submodel, the shrinkage strategy performs better than the LASSO strategy when, and only when, there are many inactive predictors in the model. Shrinkage and LASSO strategies are applied to a real data set from Veteran's administration (VA) lung cancer study to illustrate the usefulness of the procedures in practice.     

Relative performance of stein-rule and preliminary test estimators in linear models least squares theory

In the classical (univariare) linear model, bearing the plausibility of a subset of the regression parameters being close to a pivot, shrinkage least squares estimation of the complementary subset is considered. Based on the usual James-Stein rule, shrinkage least squares estimators are constructed, and under an asymptotic setup (allowing the shrinkage parameters to be 'close to ' the pivot), the relative performance of such estimators and the prcliminary test estimators is studied. In this context, the normality of the errors is also avoided under the same asymptotic setup. None of the shrinkage and preliminary test estimators may dominate the other (in the light of the asymptotic distributional risk criterion, as has been developed here), though each of them fares well relative to the classical least squeres estimator. The chice of the shrinkage factor is also examined properly.     

A note on the limiting properties of the least squares estimation for the random coefficient autoregressive model

This note is concerned with the limiting properties of the least squares estimation for the random coefficient autoregressive model. In contrast with existing results, ours is applicable to a wide range of models under more general assumptions.     

On finite-sample properties of adaptive least squares regression estimates

This paper is mainly concerned with deriving finite-sample properties of least squares estimators for the regression function in a nonparametric regression situation under some simplifying assumptions such as normally distributed errors with a common known variance. The selection of basis functions to be used for the construction of an estimator may be regarded as a smoothing problem, and will usually be done in a data-dependent way, A straightforward application of a result by P. J. Kernpthorne yields that, under a squared error loss, all selection procedures are admissible. Furthermore, the minimax approach provides an interpolating estimator, which is often impractical, Therefore, within a certain class of selection procedures an optimal one is determined using the minimax regret principle. It can be seen to behave similarly to the procedure minimizing either an unbiased risk estimator or, equivalently, the C_p-criterion.     

Merging data from multiple sources: pretest and shrinkage perspectives

In this article, we have developed asymptotic theory for the simultaneous estimation of the k means of arbitrary populations under the common mean hypothesis and further assuming that corresponding population variances are unknown and unequal. The unrestricted estimator, the Graybill-Deal-type restricted estimator, the preliminary test, and the Stein-type shrinkage estimators are suggested. A large sample test statistic is also proposed as a pretest for testing the common mean hypothesis. Under the sequence of local alternatives and squared error loss, we have compared the asymptotic properties of the estimators by means of asymptotic distributional quadratic bias and risk. Comprehensive Monte-Carlo simulation experiments were conducted to study the relative risk performance of the estimators with reference to the unrestricted estimator in finite samples. Two real-data examples are also furnished to illustrate the application of the suggested estimation strategies.     

Model selection and post estimation based on a pretest for logistic regression models

ABSTRACT

This article addresses the problem of parameter estimation of the logistic regression model under subspace information via linear shrinkage, pretest, and shrinkage pretest estimators along with the traditional unrestricted maximum likelihood estimator and restricted estimator. We developed an asymptotic theory for the linear shrinkage and pretest estimators and compared their relative performance using the notion of asymptotic distributional bias and asymptotic quadratic risk. The analytical results demonstrated that the proposed estimation strategies outperformed the classical estimation strategies in a meaningful parameter space. Detailed Monte-Carlo simulation studies were conducted for different combinations and the performance of each estimation method was evaluated in terms of simulated relative efficiency. The results of the simulation study were in strong agreement with the asymptotic analytical findings. Two real-data examples are also given to appraise the performance of the estimators.     

Second-order least squares estimation of censored regression models

This paper proposes the second-order least squares estimation, which is an extension of the ordinary least squares method, for censored regression models where the error term has a general parametric distribution (not necessarily normal). The strong consistency and asymptotic normality of the estimator are derived under fairly general regularity conditions. We also propose a computationally simpler estimator which is consistent and asymptotically normal under the same regularity conditions. Finite sample behavior of the proposed estimators under both correctly and misspecified models are investigated through Monte Carlo simulations. The simulation results show that the proposed estimator using optimal weighting matrix performs very similar to the maximum likelihood estimator, and the estimator with the identity weight is more robust against the misspecification.     

On least squares estimation for categorical data

We consider a multinomial distribution in which the cell probabilities are known arbitrary functions of a vector parameter θ. It is desired to estimate θ by least squares. Three variations of the least squares approach are investigated, and each is found to be equivalent, in the very strong sense of being algebraically identical, to one of the following estimation procedures: maximum likelihood, minimum χ² and minimum modified χ². Two of these results also apply to the multiple hypergeometric distribution.     

On shrinkage m-estimators of location parameters

For a general class of continuous ( and marginally symmetric ) inultivariate distributions, based on suitable M-statistics ( involving bounded but possibly discontinuous score generating functions), shrinkage estimators of location are considered. These estimators are based on the James-Stein type rule and incorporates the idea of preliminary test estimation too. The main emphasis is laid on the study of asymptotic tdistributional ) risk properties of these est-innators, and asymptotic tin-) adraissibility results are also studied under fairly general regularity conditions.     

Shrinkage and penalized estimators in weighted least absolute deviations regression models

In this paper, we consider the estimation problem of the weighted least absolute deviation (WLAD) regression parameter vector when there are some outliers or heavy-tailed errors in the response and the leverage points in the predictors. We propose the pretest and James–Stein shrinkage WLAD estimators when some of the parameters may be subject to certain restrictions. We derive the asymptotic risk of the pretest and shrinkage WLAD estimators and show that if the shrinkage dimension exceeds two, the asymptotic risk of the shrinkage WLAD estimator is strictly less than the unrestricted WLAD estimator. On the other hand, the risk of the pretest WLAD estimator depends on the validity of the restrictions on the parameters. Furthermore, we study the WLAD absolute shrinkage and selection operator (WLAD-LASSO) and compare its relative performance with the pretest and shrinkage WLAD estimators. A simulation study is conducted to evaluate the performance of the proposed estimators relative to that of the unrestricted WLAD estimator. A real-life data example using body fat study is used to illustrate the performance of the suggested estimators.     

On distribution-weighted partial least squares with diverging number of highly correlated predictors

Summary.  Because highly correlated data arise from many scientific fields, we investigate parameter estimation in a semiparametric regression model with diverging number of predictors that are highly correlated. For this, we first develop a distribution-weighted least squares estimator that can recover directions in the central subspace, then use the distribution-weighted least squares estimator as a seed vector and project it onto a Krylov space by partial least squares to avoid computing the inverse of the covariance of predictors. Thus, distrbution-weighted partial least squares can handle the cases with high dimensional and highly correlated predictors. Furthermore, we also suggest an iterative algorithm for obtaining a better initial value before implementing partial least squares. For theoretical investigation, we obtain strong consistency and asymptotic normality when the dimension p of predictors is of convergence rate O { n ^1/2/ log ( n )} and o ( n ^1/3) respectively where n is the sample size. When there are no other constraints on the covariance of predictors, the rates n ^1/2 and n ^1/3 are optimal. We also propose a Bayesian information criterion type of criterion to estimate the dimension of the Krylov space in the partial least squares procedure. Illustrative examples with a real data set and comprehensive simulations demonstrate that the method is robust to non-ellipticity and works well even in 'small n –large p ' problems.     

An asymptotic theory for semiparametric generalized least squares estimation in partially linear regression models

Consider a partially linear regression model with an unknown vector parameter β, an unknown functiong(·), and unknown heteroscedastic error variances. In this paper we develop an asymptotic semiparametric generalized least squares estimation theory under some weak moment conditions. These moment conditions are satisfied by many of the error distributions encountered in practice, and our theory does not require the number of replications to go to infinity.     

Application of shrinkage estimation in linear regression models with autoregressive errors

In this paper, we consider the shrinkage and penalty estimation procedures in the linear regression model with autoregressive errors of order p when it is conjectured that some of the regression parameters are inactive. We develop the statistical properties of the shrinkage estimation method including asymptotic distributional biases and risks. We show that the shrinkage estimators have a significantly higher relative efficiency than the classical estimator. Furthermore, we consider the two penalty estimators: least absolute shrinkage and selection operator (LASSO) and adaptive LASSO estimators, and numerically compare their relative performance with that of the shrinkage estimators. A Monte Carlo simulation experiment is conducted for different combinations of inactive predictors and the performance of each estimator is evaluated in terms of the simulated mean-squared error. This study shows that the shrinkage estimators are comparable to the penalty estimators when the number of inactive predictors in the model is relatively large. The shrinkage and penalty methods are applied to a real data set to illustrate the usefulness of the procedures in practice.     

Estimation of parameters in a generalized GMANOVA model based on an outer product analogy and least squares

This paper investigates estimation of parameters in a combination of the multivariate linear model and growth curve model, called a generalized GMANOVA model. Making analogy between the outer product of data vectors and covariance yields an approach to directly do least squares to covariance. An outer product least squares estimator of covariance (COPLS estimator) is obtained and its distribution is presented if a normal assumption is imposed on the error matrix. Based on the COPLS estimator, two-stage generalized least squares estimators of the regression coefficients are derived. In addition, asymptotic normalities of these estimators are investigated. Simulation studies have shown that the COPLS estimator and two-stage GLS estimators are alternative competitors with more efficiency in the sense of sample mean, standard deviations and mean of the variance estimates to the existing ML estimator in finite samples. An example of application is also illustrated.     

Model selection and parameter estimation of a multinomial logistic regression model

In the multinomial regression model, we consider the methodology for simultaneous model selection and parameter estimation by using the shrinkage and LASSO (least absolute shrinkage and selection operation) [R. Tibshirani, Regression shrinkage and selection via the LASSO, J. R. Statist. Soc. Ser. B 58 (1996), pp. 267–288] strategies. The shrinkage estimators (SEs) provide significant improvement over their classical counterparts in the case where some of the predictors may or may not be active for the response of interest. The asymptotic properties of the SEs are developed using the notion of asymptotic distributional risk. We then compare the relative performance of the LASSO estimator with two SEs in terms of simulated relative efficiency. A simulation study shows that the shrinkage and LASSO estimators dominate the full model estimator. Further, both SEs perform better than the LASSO estimators when there are many inactive predictors in the model. A real-life data set is used to illustrate the suggested shrinkage and LASSO estimators.     

Shrinkage estimation in lognormal regression model for censored data

We introduce in this paper, the shrinkage estimation method in the lognormal regression model for censored data involving many predictors, some of which may not have any influence on the response of interest. We develop the asymptotic properties of the shrinkage estimators (SEs) using the notion of asymptotic distributional biases and risks. We show that if the shrinkage dimension exceeds two, the asymptotic risk of the SEs is strictly less than the corresponding classical estimators. Furthermore, we study the penalty (LASSO and adaptive LASSO) estimation methods and compare their relative performance with the SEs. A simulation study for various combinations of the inactive predictors and censoring percentages shows that the SEs perform better than the penalty estimators in certain parts of the parameter space, especially when there are many inactive predictors in the model. It also shows that the shrinkage and penalty estimators outperform the classical estimators. A real-life data example using Worcester heart attack study is used to illustrate the performance of the suggested estimators.     

Semiparametric Ridge Regression Approach in Partially Linear Models

In this article, we introduce a semiparametric ridge regression estimator for the vector-parameter in a partial linear model. It is also assumed that some additional artificial linear restrictions are imposed to the whole parameter space and the errors are dependent. This estimator is a generalization of the well-known restricted least-squares estimator and is confined to the (affine) subspace which is generated by the restrictions. Asymptotic distributional bias and risk are also derived and the comparison result is then given.     

On the estimation of reliabilty function in a Weibull lifetime distribution

The estimation of the reliability function of the Weibull lifetime model is considered in the presence of uncertain prior information (not in the form of prior distribution) on the parameter of interest. This information is assumed to be available in some sort of a realistic conjecture. In this article, we focus on how to combine sample and non-sample information together in order to achieve improved estimation performance. Three classes of point estimatiors, namely, the unrestricted estimator, the shrinkage estimator and shrinkage preliminary test estimator (SPTE) are proposed. Their asymptotic biases and mean-squared errors are derived and compared. The relative dominance picture of the estimators is presented. Interestingly, the proposed SPTE dominates the unrestricted estimator in a range that is wider than that of the usual preliminary test estimator. A small-scale simulation experiment is used to examine the small sample properties of the proposed estimators. Our simulation investigations have provided strong evidence that corroborates with asymptotic theory. The suggested estimation methods are applied to a published data set to illustrate the performance of the estimators in a real-life situation.