Risk Comparison of Inequality Constrained Estimators in the Heteroscedastic Linear Model

There exist many studies which treat the inequality and/or interval constraints on coefficients in the homoscedastic linear regression model. However, the sampling performance of the inequality constrained estimators in the heteroscedastic linear model has not been examined. This paper considers the inequality constrained estimators in the heteroscedastic linear regression model and derives their risks under a quadratic loss function. Furthermore, using the inequality constrained estimators, we introduce a pre-test estimator which might be employed after the test for homoscedasticity and derive its risk. In addition, the risk performance of these estimators is evaluated numerically.     

ON THE USE OF THE STEIN VARIANCE ESTIMATOR IN THE DOUBLE k-CLASS ESTIMATOR IN REGRESSION

This paper investigates the predictive mean squared error performance of a modified double k-class estimator by incorporating the Stein variance estimator. Recent studies show that the performance of the Stein rule estimator can be improved by using the Stein variance estimator. However, as we demonstrate below, this conclusion does not hold in general for all members of the double k-class estimators. On the other hand, an estimator is found to have smaller predictive mean squared error than the Stein variance-Stein rule estimator, over quite large parts of the parameter space.     

The exact mean squared error risks of preliminary test type estimators for the multivariate normal mean

The exact mean squared error risks of the preliminary test estimtor and the Sclove modified Stein rule estimator (Sclove, Morris and Radhakrishnan, 1972) for the multivariate normal mean are computed and their risks are compared with the risks of Stein estimators.     

An explicit formula for the risk of James-Stein estimators

Expressions for the risk of a Stein estimator and its principal derivatives involve a well-defined. but nonevaluated. expectation term. Stein (1966) has suggested a second-order approximation. In this paper we present an alternative exact expression.     

A formal bayes multiple shrinkage estimator

A new minimax multiple shrinkage estimator is constructed. This estimator which can adaptively shrink towards many subspace targets, is formal Bayes with respect to a mixture of harmonic priors. Unbiased estimates of risk and simulation results suggest that the risk properties of this estimator are very similar to those of the multiple shrinkage Stein estimator proposed by George (1986a). A special case is seen to be admissible.     

Estimation Under Inequality Constraints: Semiparametric Estimation of Conditional Duration Models

This article proposes a semiparametric estimator of the parameter in a conditional duration model when there are inequality constraints on some parameters and the error distribution may be unknown. We propose to estimate the parameter by a constrained version of an unrestricted semiparametrically efficient estimator. The main requirement for applying this method is that the initial unrestricted estimator converges in distribution. Apart from this, additional regularity conditions on the data generating process or the likelihood function, are not required. Hence the method is applicable to a broad range of models where the parameter space is constrained by inequality constraints, such as the conditional duration models. In a simulation study involving conditional duration models, the overall performance of the constrained estimator was better than its competitors, in terms of mean squared error. A data example is used to illustrate the method.     

Estimating risk reduction in stein estimation

It is shown that the unbiased estimator of the risk reduction in Stein estimation is unsatisfactory from a mean-squared-error point of view. A truncated form of the unbiased estimator and various empirical Bayes estimators of the risk reduction are shown to perform much better than the unbiased estimator. A simple practical estimator is proposed whose performance is a compromise between that of the truncated and empirical Bayes estimators.     

Asymptotic mean squared error of constrained James–Stein estimators

In this paper, we consider the asymptotic expansion of the MSE of constrained James–Stein estimators. We provide an estimator of the MSE which is asymptotically valid upto O(m⁻¹). A simulation study is undertaken to evaluate the performance of these estimators.     

On the sampling performance of an inequality pre-test estimator of the regression error variance under LINEX loss

We consider the estimation of the error variance of a linear regression model where prior information is available in the form of an (uncertain) inequality constraint on the coefficients. Previous studies on this and other related problems use the squared error loss in comparing estimator’s performance. Here, by adopting the asymmetric LINEX loss function, we derive and numerically evaluate the exact risks of the inequality constrained estimator and the inequality pre-test estimator which results after a preliminary test for an inequality constraint on the coefficients. The risks based on squared error loss are special cases of our results, and we draw appropriate comparisons.     

On the bias and mean square error of the least square estimator in a regression model with two inequality constraints and multivariate t error terms

We derive and numerically evaluate the bias and mean square error of the inequality constrained least squares estimator in a model with two inequality constraints and multivariate terror terms. Our results suggest that qualitatively, the estimator properties found for models with normal errors carry over to the case of multivariate terrors.     

Estimation of the MSE matrix of the stein estimator

Uniformly minimum-variance unbiased (UMVU) estimators of the total risk and the mean-squared-error (MSE) matrix of the Stein estimator for the multivariate normal mean with unknown covariance matrix are proposed. The estimated MSE matrix is helpful in identifying the components which contribute most to the total risk. It also contains information about the performance of the shrinkage estimator with respect to other quadratic loss functions.     

A closed form solution for tre least squares regression problem with linear inequality constraints

In this paper we consider a linear model Y = Xβ+e with linear inequality constraints R'β≥r, where X and R are known and full column rank matrices. The closed form of the inequality constrained least squares (ICLS) estimator is given. We provide two examples which illustrate the use of this closed form in the computation of estimates.     

SHRINKAGE, PRETEST AND ABSOLUTE PENALTY ESTIMATORS IN PARTIALLY LINEAR MODELS

We consider a partially linear model in which the vector of coefficients β in the linear part can be partitioned as ( β ₁, β ₂) , where β ₁ is the coefficient vector for main effects (e.g. treatment effect, genetic effects) and β ₂ is a vector for ‘nuisance’ effects (e.g. age, laboratory). In this situation, inference about β ₁ may benefit from moving the least squares estimate for the full model in the direction of the least squares estimate without the nuisance variables (Steinian shrinkage), or from dropping the nuisance variables if there is evidence that they do not provide useful information (pretesting). We investigate the asymptotic properties of Stein‐type and pretest semiparametric estimators under quadratic loss and show that, under general conditions, a Stein‐type semiparametric estimator improves on the full model conventional semiparametric least squares estimator. The relative performance of the estimators is examined using asymptotic analysis of quadratic risk functions and it is found that the Stein‐type estimator outperforms the full model estimator uniformly. By contrast, the pretest estimator dominates the least squares estimator only in a small part of the parameter space, which is consistent with the theory. We also consider an absolute penalty‐type estimator for partially linear models and give a Monte Carlo simulation comparison of shrinkage, pretest and the absolute penalty‐type estimators. The comparison shows that the shrinkage method performs better than the absolute penalty‐type estimation method when the dimension of the β ₂ parameter space is large.     

On the use of the Stein variance estimator in the double <Emphasis Type="Italic">k</Emphasis>-class estimator when each individual regression coefficient is estimated

In this paper we consider the double k-class estimator which incorporates the Stein variance estimator. This estimator is called the SVKK estimator. We derive the explicit formula for the mean squared error (MSE) of the SVKK estimator for each individual regression coefficient. It is shown analytically that the MSE performance of the Stein-rule estimator for each individual regression coefficient can be improved by utilizing the Stein variance estimator. Also, MSE’s of several estimators included in a family of the SVKK estimators are compared by numerical evaluations.     

Whither jackknifing in stein-rule estimation

In multi-parameter ( multivariate ) estimation, the Stein rule provides minimax and admissible estimators , compromising generally on their unbiasedness. On the other hand, the primary aim of jack-knifing is to reduce the bias of an estimator ( without necessarily compromising on its efficacy ), and, at the same time, jackknifing provides an estimator of the sampling variance of the estimator as well. In shrinkage estimation ( where minimization of a suitably defined risk function is the basic goal ), one may wonder how far the bias-reduction objective of jackknifing incorporates the dual objective of minimaxity ( or admissibility ) and estimating the risk of the estimator ? A critical appraisal of this basic role of jackknifing in shrinkage estimation is made here. Restricted, semi-restricted and the usual versions of jackknifed shrinkage estimates are considered and their performance characteristics are studied . It is shown that for Pitman-type ( local ) alternatives, usually, jackkntfing fails to provide a consistent estimator of the ( asymptotic ) risk of the shrinkage estimator, and a degenerate asymptotic situation arises for the usual fixed alternative case.     

Estimating a multivariate treatment effect under a biased allocation rule

In a clinical trial with a biased allocation rule whereby all and only those patients at risk are given the new treatment, Robbins and Zhang (1989) derived an asymptotically normal and efficient estimator of the mean difference between the new and old treatments on those at risk. This paper with the use of a well known identity of Stein (1981) generalizes the result to the multivariate situation.     

Alternative estimators of the common regression matrix in two GMANOVA models under weighted quadratic losses

We consider the problem of estimating the common regression matrix of two GMANOVA models with different unknown covariance matrices under certain type of loss functions which include a weighted quadratic loss function as a special case. We consider a class of estimators, which contains the Graybill–Deal-type estimator proposed by Sugiura and Kubokawa (Ann. Inst. Statist. Math. 40 (1988) 119), and we give its risk representation via Kubokawa and Srivastava's (Ann. Statist. 27 (1999) 600; J. Multivariate Anal. 76 (2001) 138) identities when the error matrices follow the elliptically contoured distributions. Using the method similar to an approximate minimization of the unbiased risk estimate due to Stein (Studies in the Statistical Theory of Estimation, vol. 74, Nauka, Leningrad, 1977, p. 4), we obtain an alternative estimator to the Graybill–Deal-type estimator which was given under the normality assumption. However, it seems difficult to evaluate the risk of our proposed estimator analytically because of complex nature of its risk function. Instead, we conduct a Monte-Carlo simulation to evaluate the performance of our proposed estimator. The results indicate that our proposed estimator compares favorably with the Graybill–Deal-type estimator.     

Semiparametric Stein estimators

Stein [Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In Proc. 3rd Berkeley symp. math. statist. and pro. (pp. 197–206). University of California Press], in his seminal paper, came up with the surprising discovery that the sample mean is an inadmissible estimator of the population mean in three or higher dimensions under squared error loss. The past five decades have witnessed multiple extensions and variations of Stein’s results. In this paper we develop Stein-type estimators in a semiparametric framework and prove their coordinatewise asymptotic dominance over the sample mean in terms of Bayes risks.     

On unbiased and improved loss estimation for the mean of a multivariate normal distribution with unknown variance

There is now a sizeable literature dealing with point estimation using Stein-type estimators. As discussed in Rukhin (In: Gupta, S.S., Berger, J.O. (Eds.), Statistical Decision Theory and Related Topics, Vol. IV, Springer, New York, pp. 409–418), instances arise in practice in which an estimation rule is to be accompanied by an estimate of its loss, which is unobservable. In the context of estimating the mean vector of a multi-normal distribution assuming a known population variance, Johnstone (In: Gupta, S.S., Berger, J.O. (Eds.), Statistical Decision Theory and Related Topics, Vol. IV, Springer, New York, pp. 361–379) derived an estimator that dominates the unbiased estimator of the quadratic loss incurred by the James–Stein estimator. By applying the Stein's lemma, this note generalizes Johnstone's (In: Gupta, S.S., Berger, J.O. (Eds.), Statistical Decision Theory and Related Topics, Vol. IV, Springer, New York, pp. 361–379) analysis to the setting of the unknown population variance. Computational evidence is provided about the risk magnitude of loss estimators associated with the James–Stein point estimator and its positive-part version.     

Shrinkage and penalized estimators in weighted least absolute deviations regression models

In this paper, we consider the estimation problem of the weighted least absolute deviation (WLAD) regression parameter vector when there are some outliers or heavy-tailed errors in the response and the leverage points in the predictors. We propose the pretest and James–Stein shrinkage WLAD estimators when some of the parameters may be subject to certain restrictions. We derive the asymptotic risk of the pretest and shrinkage WLAD estimators and show that if the shrinkage dimension exceeds two, the asymptotic risk of the shrinkage WLAD estimator is strictly less than the unrestricted WLAD estimator. On the other hand, the risk of the pretest WLAD estimator depends on the validity of the restrictions on the parameters. Furthermore, we study the WLAD absolute shrinkage and selection operator (WLAD-LASSO) and compare its relative performance with the pretest and shrinkage WLAD estimators. A simulation study is conducted to evaluate the performance of the proposed estimators relative to that of the unrestricted WLAD estimator. A real-life data example using body fat study is used to illustrate the performance of the suggested estimators.