Exact small sample properties of an operational variant of the minimum mean squared error estimator

In this paper, we show a sufficient condition for an operational variant of the minimum mean squared error estimator (simply, the minimum MSE estimator) to dominate the ordinary least squares (OLS) estimator. It is also shown numerically that the minimum MSE estimator dominates the OLS estimator if the number of regression coefficients is larger than or equal to three, even if the sufficient condition is not satisfied. When the number of regression coefficients is smaller than three, our numerical results show that the gain in MSE of using the minimum MSE estimator is larger than the loss.     

Risk of a homoscedasticity pre-test estimator of the regression scale under LINEX loss

In this paper we consider the risk of an estimator of the error variance after a pre-test for homoscedasticity of the variances in the two-sample heteroscedastic linear regression model. This particular pre-test problem has been well investigated but always under the restrictive assumption of a squared error loss function. We consider an asymmetric loss function — the LINEX loss function — and derive the exact risks of various estimators of the error variance.     

A comparison of estimators for the mean in the inverse gaussian distribution with a known coefficient of variation

In this paper, we discuss an estimation problem of the mean in the inverse Gaussian distribution with a known coefficient of variation. Two types of linear estimators for the mean, the linear minimum variance unbiased estimator and the linear minimum mean squared error estimator, are constructed by using the squared error loss function and their properties are examined. It is observed that, for small samples the performance of the proposed estimators is better than that of the maximum likelihood estimator, when the coefficient of variation is large.     

The Bayes rule of the variance parameter of the hierarchical normal and inverse gamma model under Stein's loss

For the variance parameter of the hierarchical normal and inverse gamma model, we analytically calculate the Bayes rule (estimator) with respect to a prior distribution IG (alpha, beta) under Stein's loss function. This estimator minimizes the posterior expected Stein's loss (PESL). We also analytically calculate the Bayes rule and the PESL under the squared error loss. Finally, the numerical simulations exemplify that the PESLs depend only on alpha and the number of observations. The Bayes rules and PESLs under Stein's loss are unanimously smaller than those under the squared error loss.     

Risk Comparison of Inequality Constrained Estimators in the Heteroscedastic Linear Model

There exist many studies which treat the inequality and/or interval constraints on coefficients in the homoscedastic linear regression model. However, the sampling performance of the inequality constrained estimators in the heteroscedastic linear model has not been examined. This paper considers the inequality constrained estimators in the heteroscedastic linear regression model and derives their risks under a quadratic loss function. Furthermore, using the inequality constrained estimators, we introduce a pre-test estimator which might be employed after the test for homoscedasticity and derive its risk. In addition, the risk performance of these estimators is evaluated numerically.     

Developing a restricted two-parameter Liu-type estimator: A comparison of restricted estimators in the binary logistic regression model

In the context of estimating regression coefficients of an ill-conditioned binary logistic regression model, we develop a new biased estimator having two parameters for estimating the regression vector parameter β when it is subjected to lie in the linear subspace restriction Hβ = h. The matrix mean squared error and mean squared error (MSE) functions of these newly defined estimators are derived. Moreover, a method to choose the two parameters is proposed. Then, the performance of the proposed estimator is compared to that of the restricted maximum likelihood estimator and some other existing estimators in the sense of MSE via a Monte Carlo simulation study. According to the simulation results, the performance of the estimators depends on the sample size, number of explanatory variables, and degree of correlation. The superiority region of our proposed estimator is identified based on the biasing parameters, numerically. It is concluded that the new estimator is superior to the others in most of the situations considered and it is recommended to the researchers.     

Semiparametric Stein estimators

Stein [Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In Proc. 3rd Berkeley symp. math. statist. and pro. (pp. 197–206). University of California Press], in his seminal paper, came up with the surprising discovery that the sample mean is an inadmissible estimator of the population mean in three or higher dimensions under squared error loss. The past five decades have witnessed multiple extensions and variations of Stein’s results. In this paper we develop Stein-type estimators in a semiparametric framework and prove their coordinatewise asymptotic dominance over the sample mean in terms of Bayes risks.     

On a class of shrinkage estimators of vector of regression coefficients the

This paper studies a class of shrinkage estimators of the vector of regression coefficients. The small disturbance approximations for the bias and the mean squared error matrix of the estimator are derived. In the sense of mean squared error, these estimators dominate the least squares estimator and the generalized Stein estimator developed by Hosmane (1988).     

Modified ridge-type for the Poisson regression model: simulation and application

The Poisson regression model (PRM) is employed in modelling the relationship between a count variable (y) and one or more explanatory variables. The parameters of PRM are popularly estimated using the Poisson maximum likelihood estimator (PMLE). There is a tendency that the explanatory variables grow together, which results in the problem of multicollinearity. The variance of the PMLE becomes inflated in the presence of multicollinearity. The Poisson ridge regression (PRRE) and Liu estimator (PLE) have been suggested as an alternative to the PMLE. However, in this study, we propose a new estimator to estimate the regression coefficients for the PRM when multicollinearity is a challenge. We perform a simulation study under different specifications to assess the performance of the new estimator and the existing ones. The performance was evaluated using the scalar mean square error criterion and the mean squared error prediction error. The aircraft damage data was adopted for the application study and the estimators’ performance judged by the SMSE and the mean squared prediction error. The theoretical comparison shows that the proposed estimator outperforms other estimators. This is further supported by the simulation study and the application result.KEYWORDS: Poisson regression model, Poisson maximum likelihood estimator, multicollinearity, Poisson ridge regression, Liu estimator, simulation     

PMSE dominance of the positive-part shrinkage estimator in a regression model with proxy variables

Consider a linear regression model with some relevant regressors are unobservable. In such a situation, we estimate the model by using the proxy variables as regressors or by simply omitting the relevant regressors. In this paper, we derive the explicit formula of predictive mean squared error (PMSE) of a general family of shrinkage estimators of regression coefficients. It is shown analytically that the positive-part shrinkage estimator dominates the ordinary shrinkage estimator even when proxy variables are used in place of the unobserved variables. Also, as an example, our result is applied to the double k-class estimator proposed by Ullah and Ullah (Double k-class estimators of coefficients in linear regression. Econometrica. 1978;46:705–722). Our numerical results show that the positive-part double k-class estimator with proxy variables has preferable PMSE performance.     

MSE performance of the weighted average estimators consisting of shrinkage estimators

In this paper, we consider a regression model and propose estimators which are the weighted averages of two estimators among three estimators; the Stein-rule (SR), the minimum mean squared error (MMSE), and the adjusted minimum mean-squared error (AMMSE) estimators. It is shown that one of the proposed estimators has smaller mean-squared error (MSE) than the positive-part Stein-rule (PSR) estimator over a moderate region of parameter space when the number of the regression coefficients is small (i.e., 3), and its MSE performance is comparable to the PSR estimator even when the number of the regression coefficients is not so small.     

Loss functions in restricted parameter spaces and their Bayesian applications

ABSTRACT

Squared error loss remains the most commonly used loss function for constructing a Bayes estimator of the parameter of interest. However, it can lead to suboptimal solutions when a parameter is defined on a restricted space. It can also be an inappropriate choice in the context when an extreme overestimation and/or underestimation results in severe consequences and a more conservative estimator is preferred. We advocate a class of loss functions for parameters defined on restricted spaces which infinitely penalize boundary decisions like the squared error loss does on the real line. We also recall several properties of loss functions such as symmetry, convexity and invariance. We propose generalizations of the squared error loss function for parameters defined on the positive real line and on an interval. We provide explicit solutions for corresponding Bayes estimators and discuss multivariate extensions. Four well-known Bayesian estimation problems are used to demonstrate inferential benefits the novel Bayes estimators can provide in the context of restricted estimation.     

Bayesian Estimation of Regression Coefficients Under Extended Balanced Loss Function

Appreciating the desirability of simultaneously using both the criteria of goodness of fitted model and clustering of estimates around true parameter values, an extended version of the balanced loss function is presented and the Bayesian estimation of regression coefficients is discussed. The thus obtained optimal estimator is then compared with the least squares estimator and posterior mean vector with respect to the criteria like posterior expected loss, Bayes risk, bias vector, mean squared error matrix and risk function.     

Asymptotic Properties and Variance Estimators of the M-quantile Regression Coefficients Estimators

M-quantile regression is defined as a “quantile-like” generalization of robust regression based on influence functions. This article outlines asymptotic properties for the M-quantile regression coefficients estimators in the case of i.i.d. data with stochastic regressors, paying attention to adjustments due to the first-step scale estimation. A variance estimator of the M-quantile regression coefficients based on the sandwich approach is proposed. Empirical results show that this estimator appears to perform well under different simulated scenarios. The sandwich estimator is applied in the small area estimation context for the estimation of the mean squared error of an estimator for the small area means. The results obtained improve previous findings, especially in the case of heteroskedastic data.     

ESTIMATION OF EXPONENTIATED WEIBULL SHAPE PARAMETERS UNDER LINEX LOSS FUNCTION

ABSTRACT

The paper deals with Bayes estimation of the exponentiated Weibull shape parameters under linex loss function when independent non-informative type of priors are available for the parameters. Generalized maximum likelihood estimators have also been obtained. Performances of the proposed Bayes estimator, generalized maximum likelihood estimators, posterior mean (i.e., Bayes estimator under squared error loss function) and maximum likelihood estimators have been studied on the basis of their risks under linex loss function. The comparison is based on a simulation study because the expressions for risk functions of these estimators cannot be obtained in nice closed forms.     

Estimating the error variance after a pre-test for an inequality restriction on the coefficients

Estimation of the regression error variance after a preliminary test of an inequality constraint on the coefficient vector is considered. We derive the exact finite sample risk of several inequality restricted and pre-test estimators of σ². These estimators are associated with the maximum likelihood, least squares and minimum mean squared error component estimators. Optimal critical values for the pre-test according to a mini-max regret criterion are numerically calculated. Furthermore, we examine the robustness of the optimal choice of critical values and the risk properties of the estimators of σ² to model mis-specification through the exclusion of relevant regressors.     

Improved Estimation of the Pareto Location Parameter Under Squared Error Loss

Suppose two independent observations are drawn from Pareto distributions with known shape parameters and an order restriction on the unknown location parameters. An isotonic regression estimator of the smaller location parameter dominates a preferred marginal estimator under squared error loss, but fails to dominate under stochastic domination. The results expressed herein advance the theory of order restricted inference.     

Second-order asymptotic comparison of estimators under φ-divergence loss

Our main concern is on the second-order asymptotic optimality problem of estimators. The φ-divergence loss is used as a criterion for evaluating the performance of estimators. In the comparison problem of any two estimators, the condition that one estimator dominates another estimator under the φ-divergence risk is given by evaluating the second-order term in the difference between the risks. As a result, it is proved that the condition is characterized by a peculiar value of the φ-divergence loss, which is called the divergence-loss coefficient. Furthermore, it is shown that the comparison based on the φ-divergence loss does not correspond with that based on any standard loss functions including the mean squared error, the absolute loss and the 0-1 loss. In addition, a necessary and sufficient condition for an estimator to be second-order admissible is derived.     

Estimation Under Inequality Constraints: Semiparametric Estimation of Conditional Duration Models

This article proposes a semiparametric estimator of the parameter in a conditional duration model when there are inequality constraints on some parameters and the error distribution may be unknown. We propose to estimate the parameter by a constrained version of an unrestricted semiparametrically efficient estimator. The main requirement for applying this method is that the initial unrestricted estimator converges in distribution. Apart from this, additional regularity conditions on the data generating process or the likelihood function, are not required. Hence the method is applicable to a broad range of models where the parameter space is constrained by inequality constraints, such as the conditional duration models. In a simulation study involving conditional duration models, the overall performance of the constrained estimator was better than its competitors, in terms of mean squared error. A data example is used to illustrate the method.     

The Bayes rule of the parameter in (0,1) under Zhang’s loss function with an application to the beta-binomial model

Abstract

For the restricted parameter space (0,1), we propose Zhang’s loss function which satisfies all the 7 properties for a good loss function on (0,1). We then calculate the Bayes rule (estimator), the posterior expectation, the integrated risk, and the Bayes risk of the parameter in (0,1) under Zhang’s loss function. We also calculate the usual Bayes estimator under the squared error loss function, and the Bayes estimator has been proved to underestimate the Bayes estimator under Zhang’s loss function. Finally, the numerical simulations and a real data example of some monthly magazine exposure data exemplify our theoretical studies of two size relationships about the Bayes estimators and the Posterior Expected Zhang’s Losses (PEZLs).