Linearized Ridge Regression Estimator Under the Mean Squared Error Criterion in a Linear Regression Model

This article mainly aims to study the superiority of the notion of linearized ridge regression estimator (LRRE) under the mean squared error criterion in a linear regression model. Firstly, we derive uniform lower bound of MSE for the class of the generalized shrinkage estimator (GSE), based on which it is shown that the optimal LRRE is the best estimator in the class of GSE's. Secondly, we propose the notion of the almost unbiased completeness and show that LRRE possesses such a property. Thirdly, the simulation study is given, from which it indicates that the LRRE performs desirably. Finally, the main results are applied to the well known Hald data.     

On the restricted almost unbiased estimators in linear regression

In this paper, the restricted almost unbiased ridge regression estimator and restricted almost unbiased Liu estimator are introduced for the vector of parameters in a multiple linear regression model with linear restrictions. The bias, variance matrices and mean square error (MSE) of the proposed estimators are derived and compared. It is shown that the proposed estimators will have smaller quadratic bias but larger variance than the corresponding competitors in literatures. However, they will respectively outperform the latter according to the MSE criterion under certain conditions. Finally, a simulation study and a numerical example are given to illustrate some of the theoretical results.     

Recovery of inter-block information: extensions in a two variance component model

For a two variance component mixed linear model, it is shown that under suitable conditions there exists a nonlinear unbiased estimator that is better than a best linear unbiased estimator defined with respect to a given singular covariance matrix. It is also shown how this result applies to improving on intra-block estimators and on estimators like the unweighted means estimator in a random one-way model.     

General ridge predictors in a mixed linear model

This paper mainly aims to put forward two estimators for the linear combination of fixed effects and random effects, and to investigate their properties in a general mixed linear model. First, we define the notion of a Type-I general ridge predictor (GRP) and obtain two sufficient conditions for a Type-I GRP to be superior over the best linear unbiased predictor (BLUP). Second, we establish the relationship between a Type-I GRP and linear admissibility, which results in the notion of Type-II GRP. We show that a linear predictor is linearly admissible if and only if it is a Type-II GRP. The superiority of a Type-II GRP over the BLUP is also obtained. Third, the problem of confidence ellipsoids based on the BLUP and Type-II GRP is investigated.     

Estimation of regression vectors in linear mixed models with Dirichlet process random effects

The Dirichlet process has been used extensively in Bayesian non parametric modeling, and has proven to be very useful. In particular, mixed models with Dirichlet process random effects have been used in modeling many types of data and can often outperform their normal random effect counterparts. Here we examine the linear mixed model with Dirichlet process random effects from a classical view, and derive the best linear unbiased estimator (BLUE) of the fixed effects. We are also able to calculate the resulting covariance matrix and find that the covariance is directly related to the precision parameter of the Dirichlet process, giving a new interpretation of this parameter. We also characterize the relationship between the BLUE and the ordinary least-squares (OLS) estimator and show how confidence intervals can be approximated.     

On Prediction for Correlated Domains in Longitudinal Surveys

In the survey sampling estimation or prediction of both population’s and subopulation’s (domain’s) characteristics is one of the key issues. In the case of the estimation or prediction of domain’s characteristics one of the problems is looking for additional sources of information that can be used to increase the accuracy of estimators or predictors. One of these sources may be spatial and temporal autocorrelation. Due to the mean squared error (MSE) estimation, the standard assumption is that random variables are independent for population elements from different domains. If the assumption is taken into account, spatial correlation may be assumed only inside domains. In the paper, we assume some special case of the linear mixed model with two random components that obey assumptions of the first-order spatial autoregressive model SAR(1) (but inside groups of domains instead of domains) and first-order temporal autoregressive model AR(1). Based on the model, the empirical best linear unbiased predictor will be proposed together with an estimator of its MSE taking the spatial correlation between domains into account.     

Prediction in a class of mixed models with two variance components

A best unbiased predictor (BUP) of an arbitrary linear combination of fixed and random effects in mixed linear models is available when the true values of the variance ratios are known. When the true values are unknown, a two-stage predictor, obtained from the BUP by replacing the true values by estimated values, can be used. In this article, exact mean squared errors of two-stage predictors are obtained for a class of mixed models with two variance components that includes the balanced one-way random model and other analysis-of-variance models with proportional frequencies and one balanced random factor.     

A pseudo‐empirical best linear unbiased prediction approach to small area estimation using survey weights

The authors develop a small area estimation method using a nested error linear regression model and survey weights. In particular, they propose a pseudo‐empirical best linear unbiased prediction (pseudo‐EBLUP) estimator to estimate small area means. This estimator borrows strength across areas through the model and makes use of the survey weights to preserve the design consistency as the area sample size increases. The proposed estimator also has a nice self‐benchmarking property. The authors also obtain an approximation to the model mean squared error (MSE) of the proposed estimator and a nearly unbiased estimator of MSE. Finally, they compare the proposed estimator with the EBLUP estimator and the pseudo‐EBLUP estimator proposed by Prasad & Rao (1999), using data analyzed earlier by Battese, Harter & Fuller (1988).     

Good ridge estimators based on prior information

Ridge regression is re-examined and ridge estimators based on prior information are introduced. A necessary and sufficient condition is given for such ridge estimators to yield estimators of every nonnull linear combination of the regression coefficients with smaller mean square error than that of the Gauss-Markov best linear unbiased estimator.     

On the natural restrictions in the singular Gauss–Markov model

A Gauss–Markov model is said to be singular if the covariance matrix of the observable random vector in the model is singular. In such a case, there exist some natural restrictions associated with the observable random vector and the unknown parameter vector in the model. In this paper, we derive through the matrix rank method a necessary and sufficient condition for a vector of parametric functions to be estimable, and necessary and sufficient conditions for a linear estimator to be unbiased in the singular Gauss–Markov model. In addition, we give some necessary and sufficient conditions for the ordinary least-square estimator (OLSE) and the best linear unbiased estimator (BLUE) under the model to satisfy the natural restrictions.      

Mean squared error estimators of small area means using survey weights

Using survey weights, You & Rao [You and Rao, The Canadian Journal of Statistics 2002; 30, 431–439] proposed a pseudo‐empirical best linear unbiased prediction (pseudo‐EBLUP) estimator of a small area mean under a nested error linear regression model. This estimator borrows strength across areas through a linking model, and makes use of survey weights to ensure design consistency and preserve benchmarking property in the sense that the estimators add up to a reliable direct estimator of the mean of a large area covering the small areas. In this article, a second‐order approximation to the mean squared error (MSE) of the pseudo‐EBLUP estimator of a small area mean is derived. Using this approximation, an estimator of MSE that is nearly unbiased is derived; the MSE estimator of You & Rao [You and Rao, The Canadian Journal of Statistics 2002; 30, 431–439] ignored cross‐product terms in the MSE and hence it is biased. Empirical results on the performance of the proposed MSE estimator are also presented. The Canadian Journal of Statistics 38: 598–608; 2010 © 2010 Statistical Society of Canada     

Small-area estimation by combining time-series and cross-sectional data

A model involving autocorrelated random effects and sampling errors is proposed for small-area estimation, using both time-series and cross-sectional data. The sampling errors are assumed to have a known block-diagonal covariance matrix. This model is an extension of a well-known model, due to Fay and Herriot (1979), for cross-sectional data. A two-stage estimator of a small-area mean for the current period is obtained under the proposed model with known autocorrelation, by first deriving the best linear unbiased prediction estimator assuming known variance components, and then replacing them with their consistent estimators. Extending the approach of Prasad and Rao (1986, 1990) for the Fay-Herriot model, an estimator of mean squared error (MSE) of the two-stage estimator, correct to a second-order approximation for a small or moderate number of time points, T, and a large number of small areas, m, is obtained. The case of unknown autocorrelation is also considered. Limited simulation results on the efficiency of two-stage estimators and the accuracy of the proposed estimator of MSE are présentés.     

Small Area Estimation for Zero-Inflated Data

The commonly used method of small area estimation (SAE) under a linear mixed model may not be efficient if data contain substantial proportion of zeros than would be expected under standard model assumptions (hereafter zero-inflated data). The authors discuss the SAE for zero-inflated data under a two-part random effects model that account for excess zeros in the data. Empirical results show that proposed method for SAE works well and produces an efficient set of small area estimates. An application to real survey data from the National Sample Survey Office of India demonstrates the satisfactory performance of the method. The authors describe a parametric bootstrap method to estimate the mean squared error (MSE) of the proposed estimator of small areas. The bootstrap estimates of the MSE are compared to the true MSE in simulation study.     

Predicting random effects with an expanded finite population mixed model

Prediction of random effects is an important problem with expanding applications. In the simplest context, the problem corresponds to prediction of the latent value (the mean) of a realized cluster selected via two-stage sampling. Recently, Stanek and Singer [Predicting random effects from finite population clustered samples with response error. J. Amer. Statist. Assoc. 99, 119–130] developed best linear unbiased predictors (BLUP) under a finite population mixed model that outperform BLUPs from mixed models and superpopulation models. Their setup, however, does not allow for unequally sized clusters. To overcome this drawback, we consider an expanded finite population mixed model based on a larger set of random variables that span a higher dimensional space than those typically applied to such problems. We show that BLUPs for linear combinations of the realized cluster means derived under such a model have considerably smaller mean squared error (MSE) than those obtained from mixed models, superpopulation models, and finite population mixed models. We motivate our general approach by an example developed for two-stage cluster sampling and show that it faithfully captures the stochastic aspects of sampling in the problem. We also consider simulation studies to illustrate the increased accuracy of the BLUP obtained under the expanded finite population mixed model.     

Stein rule prediction of the composite target function in a general linear regression model

This paper considers the problem of simultaneous prediction of the actual and average values of the dependent variable in a general linear regression model. Utilizing the philosophy of Stein rule procedure, a family of improved predictors for a linear function of the actual and expected value of the dependent variable for the forecast period has been proposed. An unbiased estimator for the mean squared error (MSE) matrix of the proposed family of predictors has been obtained and dominance of the family of Stein rule predictors over the best linear unbiased predictor (BLUP) has been established under a quadratic loss function.     

Design-based random permutation models with auxiliary information

We extend the random permutation model to obtain the best linear unbiased estimator of a finite population mean accounting for auxiliary variables under simple random sampling without replacement (SRS) or stratified SRS. The proposed method provides a systematic design-based justification for well-known results involving common estimators derived under minimal assumptions that do not require specification of a functional relationship between the response and the auxiliary variables.     

Choice of the ridge factor from the correlation matrix determinant

Ridge regression is the alternative method to ordinary least squares, which is mostly applied when a multiple linear regression model presents a worrying degree of collinearity. A relevant topic in ridge regression is the selection of the ridge parameter, and different proposals have been presented in the scientific literature. Since the ridge estimator is biased, its estimation is normally based on the calculation of the mean square error (MSE) without considering (to the best of our knowledge) whether the proposed value for the ridge parameter really mitigates the collinearity. With this goal and different simulations, this paper proposes to estimate the ridge parameter from the determinant of the matrix of correlation of the data, which verifies that the variance inflation factor (VIF) is lower than the traditionally established threshold. The possible relation between the VIF and the determinant of the matrix of correlation is also analysed. Finally, the contribution is illustrated with three real examples.     

Generalized preliminary test stochastic restricted estimator in the linear regression model

ABSTRACT

In this paper, we propose three generalized estimators, namely, generalized unrestricted estimator (GURE), generalized stochastic restricted estimator (GSRE), and generalized preliminary test stochastic restricted estimator (GPTSRE). The GURE can be used to represent the ridge estimator, almost unbiased ridge estimator (AURE), Liu estimator, and almost unbiased Liu estimator. When stochastic restrictions are available in addition to the sample information, the GSRE can be used to represent stochastic mixed ridge estimator, stochastic restricted Liu estimator, stochastic restricted almost unbiased ridge estimator, and stochastic restricted almost unbiased Liu estimator. The GPTSRE can be used to represent the preliminary test estimators based on mixed estimator. Using the GPTSRE, the properties of three other preliminary test estimators, namely preliminary test stochastic mixed ridge estimator, preliminary test stochastic restricted almost unbiased Liu estimator, and preliminary test stochastic restricted almost unbiased ridge estimator can also be discussed. The mean square error matrix criterion is used to obtain the superiority conditions to compare the estimators based on GPTSRE with some biased estimators for the two cases for which the stochastic restrictions are correct, and are not correct. Finally, a numerical example and a Monte Carlo simulation study are done to illustrate the theoretical findings of the proposed estimators.     

Comparison of estimators of the location parameter using pitman's closeness criterion

Necessary and sufficient conditions for a linear estimator to dominate another linear estimator of a location parameter under the Pitman's criterion of comparison are discussed. Consequently it is demonstrated that a linear biased estimator can not dominate a linear unbiased estimator under Pitman's criterion and that the sample mean is the Closest Linear Unbiased Estimator (CLUE). It is also shown that the ridge regression estimator with a known biasing constant can not dominate the ordinary least squares estimator. If an estimator δdominates an estimator δin the average loss sense then sufficient conditions are obtained under which δis also preferred over δunder Pitman's criterion. Further we obtain sufficient conditions under which preference under the Pitman's criterion will lead to preference under the mean squared error sense.