Two Stochastic Restricted Principal Components Regression Estimator in Linear Regression

In this article, we propose two stochastic restricted principal components regression estimator by combining the approach followed in obtaining the ordinary mixed estimator and the principal components regression estimator in linear regression model. The performance of the two new estimators in terms of matrix MSE criterion is studied. We also give an example and a Monte Carlo simulation to show the theoretical results.     

Comparisons of the <Emphasis Type="Italic">r</Emphasis> − <Emphasis Type="Italic">k</Emphasis> class estimator to the ordinary least squares estimator under the Pitman’s closeness criterion

In the presence of multicollinearity, the r − k class estimator is proposed as an alternative to the ordinary least squares (OLS) estimator which is a general estimator including the ordinary ridge regression (ORR), the principal components regression (PCR) and the OLS estimators. Comparison of competing estimators of a parameter in the sense of mean square error (MSE) criterion is of central interest. An alternative criterion to the MSE criterion is the Pitman’s (1937) closeness (PC) criterion. In this paper, we compare the r − k class estimator to the OLS estimator in terms of PC criterion so that we can get the comparison of the ORR estimator to the OLS estimator under the PC criterion which was done by Mason et al. (1990) and also the comparison of the PCR estimator to the OLS estimator by means of the PC criterion which was done by Lin and Wei (2002).     

Comparison of estimators of the location parameter using pitman's closeness criterion

Necessary and sufficient conditions for a linear estimator to dominate another linear estimator of a location parameter under the Pitman's criterion of comparison are discussed. Consequently it is demonstrated that a linear biased estimator can not dominate a linear unbiased estimator under Pitman's criterion and that the sample mean is the Closest Linear Unbiased Estimator (CLUE). It is also shown that the ridge regression estimator with a known biasing constant can not dominate the ordinary least squares estimator. If an estimator δdominates an estimator δin the average loss sense then sufficient conditions are obtained under which δis also preferred over δunder Pitman's criterion. Further we obtain sufficient conditions under which preference under the Pitman's criterion will lead to preference under the mean squared error sense.     

Superiority of the r–k Class Estimator Over Some Estimators In A Linear Model

In regression analysis, to overcome the problem of multicollinearity, the r ? k class estimator is proposed as an alternative to the ordinary least squares estimator which is a general estimator including the ordinary ridge regression estimator, the principal components regression estimator and the ordinary least squares estimator. In this article, we derive the necessary and sufficient conditions for the superiority of the r ? k class estimator over each of these estimators under the Mahalanobis loss function by the average loss criterion. Then, we compare these estimators with each other using the same criterion. Also, we suggest to test to verify if these conditions are indeed satisfied. Finally, a numerical example and a Monte Carlo simulation are done to illustrate the theoretical results.     

A new estimator combining the ridge regression and the restricted least squares methods of estimation

It is well-known in the literature on multicollinearity that one of the major consequences of multicollinearity on the ordinary least squares estimator is that the estimator produces large sampling variances, which in turn might inappropriately lead to exclusion of otherwise significant coefficients from the model. To circumvent this problem, two accepted estimation procedures which are often suggested are the restricted least squares method and the ridge regression method. While the former leads to a reduction in the sampling variance of the estimator, the later ensures a smaller mean square error value for the estimator. In this paper we have proposed a new estimator which is based on a criterion that combines the ideas underlying these two estimators. The standard properties of this new estimator have been studied in the paper. It has also been shown that this estimator is superior to both the restricted least squares as well as the ordinary ridge regression estimators by the criterion of mean sauare error of the estimator of the regression coefficients when the restrictions are indeed correct. The conditions for superiority of this estimator over the other two have also been derived for the situation when the restrictions are not correct.     

Principal components regression estimator and a test for the restrictions

In this article, we introduce restricted principal components regression (RPCR) estimator by combining the approaches followed in obtaining the restricted least squares estimator and the principal components regression estimator. The performance of the RPCR estimator with respect to the matrix and the generalized mean square error are examined. We also suggest a testing procedure for linear restrictions in principal components regression by using singly and doubly non-central F distribution.     

An improved and efficient biased estimation technique in logistic regression model

Abstract

In this article, we propose a new improved and efficient biased estimation method which is a modified restricted Liu-type estimator satisfying some sub-space linear restrictions in the binary logistic regression model. We study the properties of the new estimator under the mean squared error matrix criterion and our results show that under certain conditions the new estimator is superior to some other estimators. Moreover, a Monte Carlo simulation study is conducted to show the performance of the new estimator in the simulated mean squared error and predictive median squared errors sense. Finally, a real application is considered.     

Modified and Restricted r-k Class Estimators

In this article, we introduce the modified r-k class estimator and the restricted r-k class estimator. We compare the performances of the new estimators to the r-k class estimator with respect to the matrix mean square error (MSE) criterion. As a special case of the restricted r-k class estimator, we obtain the restricted principal components regression (RPCR) estimator. Finally, we conduct a Monte Carlo simulation study and a numerical example to investigate the performances of the proposed estimators by the scalar mean square error (mse) criterion.     

Ridge regression versus ols by pitman's closeness under quadratic and fisher's loss

The existence of values of the ridge parameter such that ridge regression is preferable to OLS by the Pitman nearness criterion under both the quadratic and the Fisher's loss is shown. Preference regions of the two estimators under the above loss functions are found. An upper bound for the value of the Pitman's measure of closeness, independent of a deterministic or stochastic choice of the ridge parameter, is given.     

Efficiency of an almost unbiased two-parameter estimator in linear regression model

This paper deals with parameter estimation in the linear regression model and an almost unbiased two-parameter estimator is introduced. The performance of this new estimator over the ordinary least-squares estimator and the two-parameter estimator [M.R. Özkale and S. Kaçiranlar, The restricted and unrestricted two-parameter estimator, Comm. Statist. Theory Methods 36 (2007), pp. 2707–2725] in terms of scalar mean-squared error criterion is investigated and a simulation study is done.     

The relative efficiency of the restricted estimators in linear regression models

This paper deals with the problem of multicollinearity in a multiple linear regression model with linear equality restrictions. The restricted two parameter estimator which was proposed in case of multicollinearity satisfies the restrictions. The performance of the restricted two parameter estimator over the restricted least squares (RLS) estimator and the ordinary least squares (OLS) estimator is examined under the mean square error (MSE) matrix criterion when the restrictions are correct and not correct. The necessary and sufficient conditions for the restricted ridge regression, restricted Liu and restricted shrunken estimators, which are the special cases of the restricted two parameter estimator, to have a smaller MSE matrix than the RLS and the OLS estimators are derived when the restrictions hold true and do not hold true. Theoretical results are illustrated with numerical examples based on Webster, Gunst and Mason data and Gorman and Toman data. We conduct a final demonstration of the performance of the estimators by running a Monte Carlo simulation which shows that when the variance of the error term and the correlation between the explanatory variables are large, the restricted two parameter estimator performs better than the RLS estimator and the OLS estimator under the configurations examined.     

Some conditions for optimality of the almost unbiased estimators of regression coefficients

The purpose of this paper is to examine the asymptotic properties of the operational almost unbiased estimator of regression coefficients which includes almost unbiased ordinary ridge estimator a s a special case. The small distrubance approximations for the bias and mean square error matrix of the estimator are derived. As a consequence, it is proved that, under certain conditions, the estimator is more efficient than a general class of estimators given by Vinod and Ullah (1981). Also it is shown that, if the ordinary ridge estimator (ORE) dominates the ordinary least squares estimator then the almost unbiased ordinary ridge estimator does not dominate ORE under the mean square error criterion.     

On the predictive performance of the almost unbiased Liu estimator

ABSTRACT

Regression models are usually used in forecasting (predicting) unknown values of the response variable y. This article considers the predictive performance of the almost unbiased Liu estimator compared to the ordinary least-squares estimator, principal component regression estimator, and Liu estimator. Finally, we present a numerical example to explain the theoretical results and we obtain a region where the almost unbiased Liu estimator is uniformly superior to the ordinary least-squares estimator, principal component regression estimator, and Liu estimator.     

Approaches to regression analysis with multiple measurements from individual sampling units

Application of ordinary least-squares regression to data sets which contain multiple measurements from individual sampling units produces an unbiased estimator of the parameters but a biased estimator of the covariance matrix of the parameter estimates. The present work considers a random coefficient, linear model to deal with such data sets: this model permits many senses in which multiple measurements are taken from a sampling unit, not just when it is measured at several times. Three procedures to estimate the covariance matrix of the error term of the model are considered. Given these, three procedures to estimate the parameters of the model and their covariance matrix are considered; these are ordinary least-squares, generalized least-squares, and an adjusted ordinary least-squares procedure which produces an unbiased estimator of the covariance matrix of the parameters with small samples. These various procedures are compared in simulation studies using three examples from the biological literature. The possibility of testing hypotheses about the vector of parameters is also considered. It is found that all three procedures for regression estimation produce estimators of the parameters with bias of no practical consequence, Both generalized least-squares and adjusted ordinary least-squares generally produce estimators of the covariance matrix of the parameter estimates with bias of no practical consequence, while ordinary least-squares produces a negatively biased estimator. Neither ordinary nor generalized least-squares provide satisfactory hypothesis tests of the vector of parameter estimates. It is concluded that adjusted ordinary least-squares, when applied with either of two of the procedures used to estimate the error coveriance matrix, shows promise for practical application with data sets of the nature considered here.     

The feasible generalized restricted ridge regression estimator

The presence of autocorrelation in errors and multicollinearity among the regressors have undesirable effects on the least-squares regression. There are a wide range of methods which are proposed to overcome the usefulness of the ordinary least-squares estimator or the generalized least-squares estimator, such as the Stein-rule, restricted least-squares or ridge estimator. Therefore, we introduce a new feasible generalized restricted ridge regression (FGRR) estimator to examine multicollinearity and autocorrelation problems simultaneously for the general linear regression model. We also derive some statistical properties of the FGRR estimator and comparisons have been conducted using matrix mean-square error. Moreover, a Monte Carlo simulation experiment is performed to investigate the performance of the proposed estimator over the others.     

A comparison of biased regression estimators using a pitman nearness criterion

Biased regression estimators have traditionally benn studied using the Mean Square Error (MSE) criterion. Usually these comparisons have been based on the sum of the MSE's of each of the individual parameters, i.e., a scaler valued measure that is the trace of the MSE matrix. However, since this summed MSE does not consider the covariance structure of the estimators, we propose the use of a Pitman Measure of Closeness (PMC) criterion (Keating and Gupta, 1984; Keating and Mason, 1985). In this paper we consider two versions of PMC. One of these compares the estimates and the other compares the resultant predicted values for 12 different regression estimators. These estimators represent three classes of estimators, namely, ridge, shrunken, and principal component estimators. The comparisons of these estimators using the PMC criteria are contrasted with the usual MSE criteria as well as the prediction mean square error. Included in the estimators is a relatively new estimator termed the generalized principal component estimator proposed by Jolliffe. This estimator has previously received little attention in the literature.     

Improved Liu estimator in a linear regression model

In this paper, we mainly aim to introduce the notion of improved Liu estimator (ILE) in the linear regression model y=Xβ+e. The selection of the biasing parameters is investigated under the PRESS criterion and the optimal selection is successfully derived. We make a simulation study to show the performance of ILE compared to the ordinary least squares estimator and the Liu estimator. Finally, the main results are applied to the Hald data.     

On the restricted almost unbiased estimators in linear regression

In this paper, the restricted almost unbiased ridge regression estimator and restricted almost unbiased Liu estimator are introduced for the vector of parameters in a multiple linear regression model with linear restrictions. The bias, variance matrices and mean square error (MSE) of the proposed estimators are derived and compared. It is shown that the proposed estimators will have smaller quadratic bias but larger variance than the corresponding competitors in literatures. However, they will respectively outperform the latter according to the MSE criterion under certain conditions. Finally, a simulation study and a numerical example are given to illustrate some of the theoretical results.     

THE SMALL SAMPLE PROPERTIES OF THE PRINCIPAL COMPONENTS ESTIMATOR FOR REGRESSION COEFFICIENTS

ABSTRACT

Under the mean square error matrix (MSEM) criterion and Pitman closeness (PC) criterion, the principal components estimator (PRCE) is compared with the least square estimator (LSE) and the superiority of PRCE over LSE is achieved respectively. Finally, we examine the superiority of PRCE predictor over the LSE predictor based on these two criteria.     

A note on generalized ridge estimators

It is shown that a necessary and sufficient condition derived by Farebrother (1984)for a generalized ridge estimator to dominate the ordinary least-squares estimator with respect to the mean-square-error-matrix criterion in the linear regression model admits a similar interpretation as the well known criterion of Toro-Viz-carrondo and Wallace (1968)for the dominance of a restricted least-squares estimator over the ordinary least-squares estimator. Two other properties of the generalized ridge estimators, referring to the concept of admissibility, are also pointed out.