MSE performance of the weighted average estimators consisting of shrinkage estimators when each individual regression coefficient is estimated

In this paper, we analytically derive the exact formula for the mean squared error (MSE) of two weighted average (WA) estimators for each individual regression coefficient. Further, we execute numerical evaluations to investigate small sample properties of the WA estimators, and compare the MSE performance of the WA estimators with the other shrinkage estimators and the usual OLS estimator. Our numerical results show that (1) the WA estimators have smaller MSE than the other shrinkage estimators and the OLS estimator over a wide region of parameter space; (2) the range where the relative MSE of the WA estimator is smaller than that of the OLS estimator gets narrower as the number of explanatory variables k increases.     

Inference on the Common Variance of Correlated Normal Random Variables

ABSTRACT

Scale equivariant estimators of the common variance σ², of correlated normal random variables, have mean squared errors (MSE) which depend on the unknown correlations. For this reason, a scale equivariant estimator of σ² which uniformly minimizes the MSE does not exist. For the equi-correlated case, we have developed three equivariant estimators of σ²: a Bayesian estimator for invariant prior as well as two non-Bayesian estimators. We then generalized these three estimators for the case of several variables with multiple unknown correlations. In addition, we developed a system of confidence intervals which produce the desired coverage probability while being efficient in terms of expected length.     

A two-parameter estimator in the negative binomial regression model

In this article, a two-parameter estimator is proposed to combat multicollinearity in the negative binomial regression model. The proposed two-parameter estimator is a general estimator which includes the maximum likelihood (ML) estimator, the ridge estimator (RE) and the Liu estimator as special cases. Some properties on the asymptotic mean-squared error (MSE) are derived and necessary and sufficient conditions for the superiority of the two-parameter estimator over the ML estimator and sufficient conditions for the superiority of the two-parameter estimator over the RE and the Liu estimator in the asymptotic mean-squared error (MSE) matrix sense are obtained. Furthermore, several methods and three rules for choosing appropriate shrinkage parameters are proposed. Finally, a Monte Carlo simulation study is given to illustrate some of the theoretical results.     

Finite sample properties of an HPT estimator when each individual regression coefficient is estimated in a misspecified linear regression model

ABSTRACT

In this paper, assuming that there exist omitted variables in the specified model, we analytically derive the exact formula for the mean squared error (MSE) of a heterogeneous pre-test (HPT) estimator whose components are the ordinary least squares (OLS) and feasible ridge regression (FRR) estimators. Since we cannot examine the MSE performance analytically, we execute numerical evaluations to investigate small sample properties of the HPT estimator, and compare the MSE performance of the HPT estimator with those of the FRR estimator and the usual OLS estimator. Our numerical results show that (1) the HPT estimator is more efficient when the model misspecification is severe; (2) the HPT estimator with the optimal critical value obtained under the correctly specified model can be safely used even when there exist omitted variables in the specified model.     

A Bayesian approach with generalized ridge estimation for high-dimensional regression and testing

This paper adopts a Bayesian strategy for generalized ridge estimation for high-dimensional regression. We also consider significance testing based on the proposed estimator, which is useful for selecting regressors. Both theoretical and simulation studies show that the proposed estimator can simultaneously outperform the ordinary ridge estimator and the LSE in terms of the mean square error (MSE) criterion. The simulation study also demonstrates the competitive MSE performance of our proposal with the Lasso under sparse models. We demonstrate the method using the lung cancer data involving high-dimensional microarrays.     

Using Liu-Type Estimator to Combat Collinearity

Abstract

Linear regression model and least squares method are widely used in many fields of natural and social sciences. In the presence of collinearity, the least squares estimator is unstable and often gives misleading information. Ridge regression is the most common method to overcome this problem. We find that when there exists severe collinearity, the shrinkage parameter selected by existing methods for ridge regression may not fully address the ill conditioning problem. To solve this problem, we propose a new two-parameter estimator. We show using both theoretic results and simulation that our new estimator has two advantages over ridge regression. First, our estimator has less mean squared error (MSE). Second, our estimator can fully address the ill conditioning problem. A numerical example from literature is used to illustrate the results.     

Extreme value index estimator using maximum likelihood and moment estimation

ABSTRACT

When a distribution function is in the max domain of attraction of an extreme value distribution, its tail can be well approximated by a generalized Pareto distribution. Based on this fact we use a moment estimation idea to propose an adapted maximum likelihood estimator for the extreme value index, which can be understood as a combination of the maximum likelihood estimation and moment estimation. Under certain regularity conditions, we derive the asymptotic normality of the new estimator and investigate its finite sample behavior by comparing with several classical or competitive estimators. A simulation study shows that the new estimator is competitive with other estimators in view of average bias, average MSE, and coefficient of variance of the new device for the optimal selection of the threshold.     

New difference-based estimator of parameters in semiparametric regression models

The paper introduces a new difference-based Liu estimator β?_Ldiff=([Xtilde]′[Xtilde]+I)^?1([Xtilde]′[ytilde]+η β?_diff) of the regression parameters β in the semiparametric regression model, y=Xβ+f+?. Difference-based estimator, β?_diff=([Xtilde]′[Xtilde])^?1[Xtilde]′[ytilde] and difference-based Liu estimator are analysed and compared with respect to mean-squared error (mse) criterion. Finally, the performance of the new estimator is evaluated for a real data set. Monte Carlo simulation is given to show the improvement in the scalar mse of the estimator.     

General procedure for estimating the population mean using ranked set sampling

In this paper, we suggest a class of estimators for estimating the population mean ? of the study variable Y using information on X?, the population mean of the auxiliary variable X using ranked set sampling envisaged by McIntyre [A method of unbiased selective sampling using ranked sets, Aust. J. Agric. Res. 3 (1952), pp. 385–390] and developed by Takahasi and Wakimoto [On unbiased estimates of the population mean based on the sample stratified by means of ordering, Ann. Inst. Statist. Math. 20 (1968), pp. 1–31]. The estimator reported by Kadilar et al. [Ratio estimator for the population mean using ranked set sampling, Statist. Papers 50 (2009), pp. 301–309] is identified as a member of the proposed class of estimators. The bias and the mean-squared error (MSE) of the proposed class of estimators are obtained. An asymptotically optimum estimator in the class is identified with its MSE formulae. To judge the merits of the suggested class of estimators over others, an empirical study is carried out.     

A New Estimator for Cox Proportional Hazard Regression Model in Presence of Collinearity

We propose a new approach to estimate the parameters of the Cox proportional hazards model in the presence of collinearity. Generally, a maximum partial likelihood estimator is used to estimate parameters for the Cox proportional hazards model. However, the maximum partial likelihood estimators can be seriously affected by the presence of collinearity since the parameter estimates result in large variances.

In this study, we develop a Liu-type estimator for Cox proportional hazards model parameters and compare it with a ridge regression estimator based on the scalar mean squared error (MSE). Finally, we evaluate its performance through a simulation study.     

Two Stochastic Restricted Principal Components Regression Estimator in Linear Regression

In this article, we propose two stochastic restricted principal components regression estimator by combining the approach followed in obtaining the ordinary mixed estimator and the principal components regression estimator in linear regression model. The performance of the two new estimators in terms of matrix MSE criterion is studied. We also give an example and a Monte Carlo simulation to show the theoretical results.     

Small-area estimation by combining time-series and cross-sectional data

A model involving autocorrelated random effects and sampling errors is proposed for small-area estimation, using both time-series and cross-sectional data. The sampling errors are assumed to have a known block-diagonal covariance matrix. This model is an extension of a well-known model, due to Fay and Herriot (1979), for cross-sectional data. A two-stage estimator of a small-area mean for the current period is obtained under the proposed model with known autocorrelation, by first deriving the best linear unbiased prediction estimator assuming known variance components, and then replacing them with their consistent estimators. Extending the approach of Prasad and Rao (1986, 1990) for the Fay-Herriot model, an estimator of mean squared error (MSE) of the two-stage estimator, correct to a second-order approximation for a small or moderate number of time points, T, and a large number of small areas, m, is obtained. The case of unknown autocorrelation is also considered. Limited simulation results on the efficiency of two-stage estimators and the accuracy of the proposed estimator of MSE are présentés.     

Comparisons of the <Emphasis Type="Italic">r</Emphasis> − <Emphasis Type="Italic">k</Emphasis> class estimator to the ordinary least squares estimator under the Pitman’s closeness criterion

In the presence of multicollinearity, the r − k class estimator is proposed as an alternative to the ordinary least squares (OLS) estimator which is a general estimator including the ordinary ridge regression (ORR), the principal components regression (PCR) and the OLS estimators. Comparison of competing estimators of a parameter in the sense of mean square error (MSE) criterion is of central interest. An alternative criterion to the MSE criterion is the Pitman’s (1937) closeness (PC) criterion. In this paper, we compare the r − k class estimator to the OLS estimator in terms of PC criterion so that we can get the comparison of the ORR estimator to the OLS estimator under the PC criterion which was done by Mason et al. (1990) and also the comparison of the PCR estimator to the OLS estimator by means of the PC criterion which was done by Lin and Wei (2002).     

On the performance of some new Liu parameters for the gamma regression model

The maximum likelihood (ML) method is used to estimate the unknown Gamma regression (GR) coefficients. In the presence of multicollinearity, the variance of the ML method becomes overstated and the inference based on the ML method may not be trustworthy. To combat multicollinearity, the Liu estimator has been used. In this estimator, estimation of the Liu parameter d is an important problem. A few estimation methods are available in the literature for estimating such a parameter. This study has considered some of these methods and also proposed some new methods for estimation of the d. The Monte Carlo simulation study has been conducted to assess the performance of the proposed methods where the mean squared error (MSE) is considered as a performance criterion. Based on the Monte Carlo simulation and application results, it is shown that the Liu estimator is always superior to the ML and recommendation about which best Liu parameter should be used in the Liu estimator for the GR model is given.     

Asymptotic mean squared error of constrained James–Stein estimators

In this paper, we consider the asymptotic expansion of the MSE of constrained James–Stein estimators. We provide an estimator of the MSE which is asymptotically valid upto O(m⁻¹). A simulation study is undertaken to evaluate the performance of these estimators.     

A restricted Liu estimator for binary regression models and its application to an applied demand system

In this article, we propose a restricted Liu regression estimator (RLRE) for estimating the parameter vector, β, in the presence of multicollinearity, when the dependent variable is binary and it is suspected that β may belong to a linear subspace defined by Rβ?=?r. First, we investigate the mean squared error (MSE) properties of the new estimator and compare them with those of the restricted maximum likelihood estimator (RMLE). Then we suggest some estimators of the shrinkage parameter, and a simulation study is conducted to compare the performance of the different estimators. Finally, we show the benefit of using RLRE instead of RMLE when estimating how changes in price affect consumer demand for a specific product.     

Modified and Restricted r-k Class Estimators

In this article, we introduce the modified r-k class estimator and the restricted r-k class estimator. We compare the performances of the new estimators to the r-k class estimator with respect to the matrix mean square error (MSE) criterion. As a special case of the restricted r-k class estimator, we obtain the restricted principal components regression (RPCR) estimator. Finally, we conduct a Monte Carlo simulation study and a numerical example to investigate the performances of the proposed estimators by the scalar mean square error (mse) criterion.     

Estimation of p{y p> max(y1, y2, …, yp-1)}in theexponential case

This paper deals with the derivation of (i) the MLE (ii) the MVUE (iii) a Bayes estimator of the probability in the title, for the case p = 2. Simulation studies are carried out to compare these estimators. The results suggest that the MLE and the Bayes estimator are biased and the Bayes estimator have the smallest MSE. In the general case, explicit expression for the probability in the title is derived and the MLE and Bayes estimator are obtained. A general method of deriving the MVUE is pointed out. Because of the simulation studies for p = 2 it is recommended that the Bayes or predictive estimator should be used.     

Bias Correction in the Type I Generalized Logistic Distribution

Four strategies for bias correction of the maximum likelihood estimator of the parameters in the Type I generalized logistic distribution are studied. First, we consider an analytic bias-corrected estimator, which is obtained by deriving an analytic expression for the bias to order n ^?1; second, a method based on modifying the likelihood equations; third, we consider the jackknife bias-corrected estimator; and fourth, we consider two bootstrap bias-corrected estimators. All bias correction estimators are compared by simulation. Finally, an example with a real data set is also presented.     

Computational comparison of the general weighted moments estimators of the scale parameter of a Pareto distribution with a known shape parameter with other estimators based on a multiply type II-censored sample

Wu et al. [Computational comparison for weighted moments estimators and BLUE of the scale parameter of a Pareto distribution with known shape parameter under type II multiply censored sample, Appl. Math. Comput. 181 (2006), pp. 1462–1470] proposed the weighted moments estimators (WMEs) of the scale parameter of a Pareto distribution with known shape parameter on a multiply type II-censored sample. They claimed that some WMEs are better than the best linear unbiased estimator (BLUE) based on the exact mean-squared error (MSE). In this paper, the general WME (GWME) is proposed and the computational comparison of the proposed estimator with the WMEs and BLUE is done on the basis of the exact MSE for given sample sizes and different censoring schemes. As a result, the GWME is performing better than the best estimator among 12 WMEs and BLUE for all cases. Therefore, GWME is recommended for use. At last, one example is given to demonstrate the proposed GWME.