An extension of a rank criterion for the least squares estimator to be the best linear unbiased estimator

Among criteria for the least squares estimator in a linear model (y, Xβ, V) to be simultaneously the best linear unbiased estimator, one convenient for applications is that of Anderson (1971, 1972). His result, however, has been developed under assumptions of full column rank for X and nonsingularity for V. Subsequently, this result has been extended by Styan (1973) to the case when the restriction on X is removed. In this note, it is shown that also the restriction on V can be relaxed and, consequently, that Anderson's criterion is applicable to the general linear model without any rank assumptions at all.     

Monte Carlo Simulation Study of Biased Estimators in the Linear Regression Models with Correlated or Heteroscedastic Errors

In this study, the performance of the estimators proposed in the presence of multicollinearity in the linear regression model with heteroscedastic or correlated or both error terms is investigated under the matrix mean square error criterion. Structures of the autocorrelated error terms are given and a Monte Carlo simulation study is conducted to examine the relative efficiency of the estimators against each other.     

The Restricted and Unrestricted Two-Parameter Estimators

In this article, we introduce a new two-parameter estimator by grafting the contraction estimator into the modified ridge estimator proposed by Swindel (1976 Swindel , B. F. ( 1976 ). Good ridge estimators based on prior information . Commun. Statist. Theor. Meth. A5 : 1065 – 1075 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]). This new two-parameter estimator is a general estimator which includes the ordinary least squares, the ridge, the Liu, and the contraction estimators as special cases. Furthermore, by setting restrictions Rβ = r on the parameter values we introduce a new restricted two-parameter estimator which includes the well-known restricted least squares, the restricted ridge proposed by Groß (2003 Groß , J. ( 2003 ). Restricted ridge estimation . Statist. Probab. Lett. 65 : 57 – 64 .[Crossref], [Web of Science ®] , [Google Scholar]), the restricted contraction estimators, and a new restricted Liu estimator which we call the modified restricted Liu estimator different from the restricted Liu estimator proposed by Kaç?ranlar et al. (1999 Kaç?ranlar , S. , Sakall?o?lu , S. , Akdeniz , F. , Styan , G. P. H. , Werner , H. J. ( 1999 ). A new biased estimator in linear regression and a detailed analysis of the widely-analysed dataset on Portland cement . Sankhya Ser. B., Ind. J. Statist. 61 : 443 – 459 . [Google Scholar]). We also obtain necessary and sufficient condition for the superiority of the new two-parameter estimator over the ordinary least squares estimator and the comparison of the new restricted two-parameter estimator to the new two-parameter estimator is done by the criterion of matrix mean square error. The estimators of the biasing parameters are given and a simulation study is done for the comparison as well as the determination of the biasing parameters.     

MSE Performance of a Heterogeneous Pre-Test Ridge Regression Estimator

This article describes how diagnostic procedures were derived for symmetrical nonlinear regression models, continuing the work carried out by Cysneiros and Vanegas (2008 Cysneiros , F. J. A. , Vanegas , L. H. ( 2008 ). Residuals and their statistical properties in symmetrical nonlinear models . Statist. Probab. Lett. 78 : 3269 – 3273 .[Crossref], [Web of Science ®] , [Google Scholar]) and Vanegas and Cysneiros (2010 Vanegas , L. H. , Cysneiros , F. J. A. ( 2010 ). Assesment of diagnostic procedures in symmetrical nonlinear regression models . Computat. Statist. Data Anal. 54 : 1002 – 1016 .[Crossref], [Web of Science ®] , [Google Scholar]), who showed that the parameters estimates in nonlinear models are more robust with heavy-tailed than with normal errors. In this article, we focus on assessing if the robustness of this kind of models is also observed in the inference process (i.e., partial F-test). Symmetrical nonlinear regression models includes all symmetric continuous distributions for errors covering both light- and heavy-tailed distributions such as Student-t, logistic-I and -II, power exponential, generalized Student-t, generalized logistic, and contaminated normal. Firstly, a statistical test is shown to evaluating the assumption that the error terms all have equal variance. The results of simulation studies which describe the behavior of the test for heteroscedasticity proposed in the presence of outliers are then given. To assess the robustness of inference process, we present the results of a simulation study which described the behavior of partial F-test in the presence of outliers. Also, some diagnostic procedures are derived to identify influential observations on the partial F-test. As ilustration, a dataset described in Venables and Ripley (2002 Venables , W. N. , Ripley , B. D. ( 2002 ). Modern Applied with S. , 4th ed. New York : Springer .[Crossref] , [Google Scholar]), is also analyzed.     

Evaluation and comparison of estimations in the generalized exponential-Poisson distribution

The uniformly minimum variance unbiased, maximum-likelihood, percentile and least-squares estimators of the probability density function and the cumulative distribution function are derived for the generalized exponential-Poisson distribution. This model has shown to be useful in reliability and lifetime data modelling, especially when the hazard rate function has a bathtub shape. Simulation studies are also carried out to show that the maximum-likelihood estimator is better than the uniformly minimum variance unbiased estimator (UMVUE) and that the UMVUE is better than others.     

Comparison of the estimators of the intra-cluster correlation for the nested error regression model

The intra-cluster correlation is insisted on nested error regression model that, in practice, is rarely known. This article demonstrates the size in generalized least squares (GLS) F-test using Fuller–Battese transformation and modification F-test. For the balanced case, the former using strictly positive, analysis of covariance (ANCOVA) and analysis of variance (ANOVA) estimators of intra-cluster correlation can control the size for moderate intra-cluster correlations. For small intra-cluster correlation, they perform well when the numbers of cluster are large. The latter using the ANOVA estimator performs well except for small numbers of cluster. When intra-cluster correlation is large, it cannot control the size. For the unbalanced case, the GLS F-test using the Fuller–Battese transformation and the modification F-test using the strictly positive, the ANCOVA and the ANOVA estimators maintain the significance level for small total sample size and small intra-cluster correlations when there is a large variation in cluster sizes, but they perform well in controlling the size for large total sample size and small different variation in cluster sizes. Besides, Henderson’s method 3 estimator maintains the significance level for a few situations.     

An efficiency comparison of dual ratio and product estimatiors

The present paper investigates the efficiency of the dual ratio estimator under the super population model with uncorrelated errors and a gamma-distributed auxiliary variabble. It is found that the dual ratio estimator is more efficient than the product estimator when the auxiliary variable has a gamma distribution with parameter greater than or equal to one, in the case when the regression is through the origin or when the product of intercept and slope is positive.     

A Class of s–K Type Principal Components Estimators in the Linear Model

In this article, we introduce a new class of estimators called the s–K type principal components estimators to combat multicollinearity, which include the principal components regression (PCR) estimator, the r–k estimator and the s–K estimator as special cases. Necessary and sufficient conditions for the superiority of the new estimator over the PCR estimator, the r–k estimator and the s–K estimator are derived in the sense of the mean squared error matrix criterion. A Monte Carlo simulation study and a numerical example are given to illustrate the performance of the proposed estimator.     

Two-stage james-stein estimators of the mean based on prior knowledge

We suggest five types of two-stage James-Stein type estimators of the mean vector μ based on prior knowledge about μ and two-stage sampling scheme proposed by Waikar and Katti(1971) Their risks are evaluated and calculated to compare with two-stage estimator suggested by Waikar and Katti(1971) when the prior form of an initial estimate of μ is 0. We find that the five estimators suggested here all have high efficiencies in large dimensions and/or in large value of ratio of two sample sizes at each stage.     

Performance of the stein-type two-parameter estimator in multiple linear regression model

In this article, we consider the Stein-type approach to the estimation of the regression parameter in a multiple regression model under a multicollinearity situation. The Stein-type two-parameter estimator is proposed when it is suspected that the regression parameter may be restricted to a subspace. The bias and the quadratic risk of the proposed estimator are derived and compared with the two-parameter estimator (TPE), the restricted TPE and the preliminary test TPE. The conditions of superiority of the proposed estimator are obtained. Finally, a real data example is provided to illustrate some of the theoretical results.     

MSE performance of the weighted average estimators consisting of shrinkage estimators

In this paper, we consider a regression model and propose estimators which are the weighted averages of two estimators among three estimators; the Stein-rule (SR), the minimum mean squared error (MMSE), and the adjusted minimum mean-squared error (AMMSE) estimators. It is shown that one of the proposed estimators has smaller mean-squared error (MSE) than the positive-part Stein-rule (PSR) estimator over a moderate region of parameter space when the number of the regression coefficients is small (i.e., 3), and its MSE performance is comparable to the PSR estimator even when the number of the regression coefficients is not so small.     

Comparisons of the alternative biased estimators for the distributed lag models

The finite distributed lag models include highly correlated variables as well as lagged and unlagged values of the same variables. Some problems are faced for this model when applying the ordinary least squares (OLS) method or econometric models such as Almon and Koyck models. The primary aim of this study is to compare performances of alternative estimators to the OLS estimator defined by combining the Almon estimator with some estimators using Almon (1965) data. A simulation study with different model parameters is performed and the estimators are compared according to the root mean square error (RMSE) and prediction mean square error (PMSE).     

Optimal determination of the parameters of some biased estimators using genetic algorithm

In this paper, some new algorithms for estimating the biasing parameters of the ridge, Liu and two-parameter estimators are introduced with the help of genetic algorithm (GA). The proposed algorithms are based on minimizing some statistical measures such as mean square error (MSE), mean absolute error (MAE) and mean absolute prediction error (MAPE). At the same time, the new algorithms allow one to keep the condition number and variance inflation factors to be less than or equal to ten by means of the GA. A numerical example is presented to show the utility of the new algorithms. In addition, an extensive Monte Carlo experiment is conducted. The numerical findings prove that the proposed algorithms enable to eliminate the problem of multicollinearity and minimize the MSE, MAE and MAPE.     

MORE ON THE PRE-TEST ESTIMATOR IN RIDGE REGRESSION

ABSTRACT

The problem of estimation of the regression coefficients in a multiple regression model is considered under a multicollinearity situation when it is suspected that the regression coefficients may be restricted to a subspace. The objective of this paper is to compare the usual preliminary test estimator and the preliminary test ridge regression estimator in the sense of the dispersion matrix of one dominating that of the other. In particular we proved two results giving necessary and sufficient conditions for the superiority of the preliminary test ridge regression estimator over the preliminary test estimator associated with the δ = 0 (or Δ = 0) and δ ≠ 0 (or Δ ≠ 0).     

Improvement of the Liu estimator in linear regression model

In the presence of stochastic prior information, in addition to the sample, Theil and Goldberger (1961) introduced a Mixed Estimator for the parameter vector β in the standard multiple linear regression model (T,Xβ,σ² I). Recently, the Liu estimator which is an alternative biased estimator for β has been proposed by Liu (1993). In this paper we introduce another new Liu type biased estimator called Stochastic restricted Liu estimator for β, and discuss its efficiency. The necessary and sufficient conditions for mean squared error matrix of the Stochastic restricted Liu estimator to exceed the mean squared error matrix of the mixed estimator will be derived for the two cases in which the parametric restrictions are correct and are not correct. In particular we show that this new biased estimator is superior in the mean squared error matrix sense to both the Mixed estimator and to the biased estimator introduced by Liu (1993).     

A sufficient condition for the MSE dominance of the positive-part shrinkage estimator when each individual regression coefficient is estimated in a misspecified linear regression model

In this paper, assuming that there exist omitted explanatory variables in the specified model, we derive the exact formula for the mean squared error (MSE) of a general family of shrinkage estimators for each individual regression coefficient. It is shown analytically that when our concern is to estimate each individual regression coefficient, the positive-part shrinkage estimators have smaller MSE than the original shrinkage estimators under some conditions even when the relevant regressors are omitted. Also, by numerical evaluations, we showed the effects of our theorem for several specific cases. It is shown that the positive-part shrinkage estimators have smaller MSE than the original shrinkage estimators for wide region of parameter space even when there exist omitted variables in the specified model.     

Efficient estimation of the PDF and the CDF of the inverse Rayleigh distribution

In this paper, we consider the estimation of the probability density function and the cumulative distribution function of the inverse Rayleigh distribution. In this regard, the following estimators are considered: uniformly minimum variance unbiased estimator, maximum likelihood (ML) estimator, percentile estimator, least squares estimator and weighted least squares estimator. To do so, analytical expressions are derived for the mean integrated squared error. As the result of simulation studies and real data applications indicate, when the sample size is not very small the ML estimator performs better than the others.     

On some pre-test and Stein-rule phi-divergence test estimators in the independence model of categorical data

For the model of independence in a two way contingency table, shrinkage estimators based on minimum φ$\phi$-divergence estimators and φ$\phi$-divergence statistics are considered. These estimators are based on the James–Stein-type rule and incorporate the idea of preliminary test estimator. The asymptotic bias and risk are obtained under contiguous alternative hypotheses, and on the basis of them a comparison study is carried out.     

Efficient Estimation of the PDF and the CDF of the Exponentiated Gumbel Distribution

The exponentiated Gumbel model has been shown to be useful in climate modeling including global warming problem, flood frequency analysis, offshore modeling, rainfall modeling, and wind speed modeling. Here, we consider estimation of the probability density function (PDF) and the cumulative distribution function (CDF) of the exponentiated Gumbel distribution. The following estimators are considered: uniformly minimum variance unbiased (UMVU) estimator, maximum likelihood (ML) estimator, percentile (PC) estimator, least-square (LS) estimator, and weighted least-square (WLS) estimator. Analytical expressions are derived for the bias and the mean squared error. Simulation studies and real data applications show that the ML estimator performs better than others.     

Linearized Ridge Regression Estimator in Linear Regression

In this article, we aim to study the linearized ridge regression (LRR) estimator in a linear regression model motivated by the work of Liu (1993). The LRR estimator and the two types of generalized Liu estimators are investigated under the PRESS criterion. The method of obtaining the optimal generalized ridge regression (GRR) estimator is derived from the optimal LRR estimator. We apply the Hald data as a numerical example and then make a simulation study to show the main results. It is concluded that the idea of transforming the GRR estimator as a complicated function of the biasing parameters to a linearized version should be paid more attention in the future.