Estimation of Standard Errors of Empirical Bayes Estimators in Capm-Type Models

This paper is concerned with the problem of estimating the standard errors of the empirical Bayes estimators in linear regression models. The problem of deriving an exact expression for the standard error of this estimator is generally intractable. We suggest a procedure based on Efron’s bootstrap method as a way of estimating the standard error. It is shown, through simulations, that the bootstrap method provides a more accurate estimate of the standard error of the empirical Bayes estimator than the traditional large sample method.     

Bootstrap adjustments for empirical bayes interval estimates of small-area proportions

Empirical Bayes approaches have often been applied to the problem of estimating small-area parameters. As a compromise between synthetic and direct survey estimators, an estimator based on an empirical Bayes procedure is not subject to the large bias that is sometimes associated with a synthetic estimator, nor is it as variable as a direct survey estimator. Although the point estimates perform very well, naïve empirical Bayes confidence intervals tend to be too short to attain the desired coverage probability, since they fail to incorporate the uncertainty which results from having to estimate the prior distribution. Several alternative methodologies for interval estimation which correct for the deficiencies associated with the naïve approach have been suggested. Laird and Louis (1987) proposed three types of bootstrap for correcting naïve empirical Bayes confidence intervals. Calling the methodology of Laird and Louis (1987) an unconditional bias-corrected naïve approach, Carlin and Gelfand (1991) suggested a modification to the Type III parametric bootstrap which corrects for bias in the naïve intervals by conditioning on the data. Here we empirically evaluate the Type II and Type III bootstrap proposed by Laird and Louis, as well as the modification suggested by Carlin and Gelfand (1991), with the objective of examining coverage properties of empirical Bayes confidence intervals for small-area proportions.     

The Statistical Properties of the Equity Estimator

In this article we consider the Equity estimator proposed by Krishnamurthi and Rangaswamy. We show that this estimator is inconsistent and does not necessarily improve on the mean squared error (MSE) of the least squares (LS) estimator. We perform a Monte Carlo experiment based on the price-promotion model used in marketing research, with marketing data, comparing the MSE of the Equity estimator to that of two empirical Bayes estimators and the LS estimator. We find that the empirical Bayes estimators have substantially smaller MSE than the Equity estimator in almost every case.     

On the small sample properties of norm-restricted maximum likelihood estimators for logistic regression models

This paper develops alternatives to maximum likelihood estimators (MLE) for logistic regression models and compares the mean squared error (MSE) of the estimators. The MLE for the vector of underlying success probabilities has low MSE only when the true probabilities are extreme (i.e., near 0 or 1). Extreme probabilities correspond to logistic regression parameter vectors which are large in norm. A competing “restricted” MLE and an empirical version of it are suggested as estimators with better performance than the MLE for central probabilities. An approximate EM-algorithm for estimating the restriction is described. As in the case of normal theory ridge estimators, the proposed estimators are shown to be formally derivable by Bayes and empirical Bayes arguments. The small sample operating characteristics of the proposed estimators are compared to the MLE via a simulation study; both the estimation of individual probabilities and of logistic parameters are considered.     

Estimation of a population size through capture-mark-recapture method: a comparison of various point and interval estimators

This article deals with the estimation of a fixed population size through capture-mark-recapture method that gives rise to hypergeometric distribution. There are a few well-known and popular point estimators available in the literature, but no good comprehensive comparison is available about their merits. Apart from the available estimators, an empirical Bayes (EB) estimator of the population size is proposed. We compare all the point estimators in terms of relative bias and relative mean squared error. Next, two new interval estimators – (a) an EB highest posterior distribution interval and (b) a frequentist interval estimator based on a parametric bootstrap method, are proposed. The comparison is then carried among the two proposed interval estimators and interval estimators derived from the currently available estimators in terms of coverage probability and average length (AL). Based on comprehensive numerical results, we rank and recommend the point estimators as well as interval estimators for practical use. Finally, a real-life data set for a green treefrog population is used as a demonstration for all the methods discussed.     

A Jackknifed estimators for the negative binomial regression model

Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators for estimating the ridge parameter k in the negative binomial (NB) regression have been proposed. The Jackknifed estimators are obtained to remedy the multicollinearity and reduce the bias. A simulation study is provided to evaluate the performance of estimators. Both mean squared error (MSE) and the percentage relative error (PRE) are considered as the performance criteria. The simulated result indicated that some of proposed Jackknifed estimators should be preferred to the ML method and ridge estimators to reduce MSE and bias.     

Extended linear empirical bayes estimation

In this paper, the linear empirical Bayes estimation method, which is based on approximation of the Bayes estimator by a linear function, is generalized to an extended linear empirical Bayes estimation technique which represents the Bayes estimator by a series of algebraic polynomials. The extended linear empirical Bayes estimators are elaborated in the case of a location or a scale parameter. The theory is illustrated by examples of its application to the normal distribution with a location parameter and the gamma distribution with a scale parameter. The linear and the extended linear empirical Bayes estimators are constructed in these two cases and, then, studied numerically via Monte Carlo simulations. The simulations show that the extended linear empirical Bayes estimators have better convergence rates than the traditional linear empirical Bayes estimators.     

Efficient mean estimation in log-normal linear models

Log-normal linear models are widely used in applications, and many times it is of interest to predict the response variable or to estimate the mean of the response variable at the original scale for a new set of covariate values. In this paper we consider the problem of efficient estimation of the conditional mean of the response variable at the original scale for log-normal linear models. Several existing estimators are reviewed first, including the maximum likelihood (ML) estimator, the restricted ML (REML) estimator, the uniformly minimum variance unbiased (UMVU) estimator, and a bias-corrected REML estimator. We then propose two estimators that minimize the asymptotic mean squared error and the asymptotic bias, respectively. A parametric bootstrap procedure is also described to obtain confidence intervals for the proposed estimators. Both the new estimators and the bootstrap procedure are very easy to implement. Comparisons of the estimators using simulation studies suggest that our estimators perform better than the existing ones, and the bootstrap procedure yields confidence intervals with good coverage properties. A real application of estimating the mean sediment discharge is used to illustrate the methodology.     

Restricted ridge estimator in generalized linear models: Monte Carlo simulation studies on Poisson and binomial distributed responses

It is known that collinearity among the explanatory variables in generalized linear models (GLMs) inflates the variance of maximum likelihood estimators. To overcome multicollinearity in GLMs, ordinary ridge estimator and restricted estimator were proposed. In this study, a restricted ridge estimator is introduced by unifying the ordinary ridge estimator and the restricted estimator in GLMs and its mean squared error (MSE) properties are discussed. The MSE comparisons are done in the context of first-order approximated estimators. The results are illustrated by a numerical example and two simulation studies are conducted with Poisson and binomial responses.     

A parametric empirical Bayes approach to the analysis of capture-recapture experiments

Empirical Bayes methods and a bootstrap bias adjustment procedure are used to estimate the size of a closed population when the individual capture probabilities are independently and identically distributed with a Beta distribution. The method is examined in simulations and applied to several well-known datasets. The simulations show the estimator performs as well as several other proposed parametric and non-parametric estimators.     

On the estimation of the true probability of a correct selection

For ranking and selection problems, the true probabiIity of a correct selection P(CS) is unknown even if a selection is made under the indifference-zone approach. Thus to estimate the true P(CS) some Bayes estimators and a bootstrap estimator are proposed for two normcal populations with common known variance. Also a bootstrap estimator and a bootstrap confidence interval are proposed for normal populations with common unknown variance. Some comparisons between proposed estimators and some other known estimators are made via Monte Carlo simulations.     

Empirical bayes estimation of several population means and variances under random sampling variances model

An extension of Kleffe–Rao model, an extended mixed model with random sampling variances, is considered. Empirical Bayes estimation is found to be very effective under such a model. The empirical Bayes estimators do not have a closed form. A second order Laplace approximation is proposed which works well for moderately large sample sizes. This approximation is specially useful when the uncertainties of the proposed empirical Bayes estimators are measured by the parametric bootstrap technique. A numerical example is considered to demonstrate the method.     

Small sample properties of a ridge regression estimator when there exist omitted variables

In this paper, we derive the exact formulae for moments of the ridge regression estimator proposed by Huang (Econ Lett 62:261–264, 1999), when there exist omitted variables. We show the conditions under which the ridge regression estimator has smaller mean squared error (MSE) than the ordinary least squares estimator. Based on the exact formulae for moments, we compare the bias and MSE performances of both estimators by numerical evaluations.     

Estimating risk reduction in stein estimation

It is shown that the unbiased estimator of the risk reduction in Stein estimation is unsatisfactory from a mean-squared-error point of view. A truncated form of the unbiased estimator and various empirical Bayes estimators of the risk reduction are shown to perform much better than the unbiased estimator. A simple practical estimator is proposed whose performance is a compromise between that of the truncated and empirical Bayes estimators.     

Improved two-parameter estimators for the negative binomial and Poisson regression models

Negative binomial regression (NBR) and Poisson regression (PR) applications have become very popular in the analysis of count data in recent years. However, if there is a high degree of relationship between the independent variables, the problem of multicollinearity arises in these models. We introduce new two-parameter estimators (TPEs) for the NBR and the PR models by unifying the two-parameter estimator (TPE) of Özkale and Kaç?ranlar [The restricted and unrestricted two-parameter estimators. Commun Stat Theory Methods. 2007;36:2707–2725]. These new estimators are general estimators which include maximum likelihood (ML) estimator, ridge estimator (RE), Liu estimator (LE) and contraction estimator (CE) as special cases. Furthermore, biasing parameters of these estimators are given and a Monte Carlo simulation is done to evaluate the performance of these estimators using mean square error (MSE) criterion. The benefits of the new TPEs are also illustrated in an empirical application. The results show that the new proposed TPEs for the NBR and the PR models are better than the ML estimator, the RE and the LE.     

An evaluation of ridge estimator in linear mixed models: an example from kidney failure data

This paper is concerned with the ridge estimation of fixed and random effects in the context of Henderson's mixed model equations in the linear mixed model. For this purpose, a penalized likelihood method is proposed. A linear combination of ridge estimator for fixed and random effects is compared to a linear combination of best linear unbiased estimator for fixed and random effects under the mean-square error (MSE) matrix criterion. Additionally, for choosing the biasing parameter, a method of MSE under the ridge estimator is given. A real data analysis is provided to illustrate the theoretical results and a simulation study is conducted to characterize the performance of ridge and best linear unbiased estimators approach in the linear mixed model.     

On Measuring Uncertainty of Benchmarked Predictors with Application to Disease Risk Estimate

Empirical Bayes (EB) estimates in general linear mixed models are useful for the small area estimation in the sense of increasing precision of estimation of small area means. However, one potential difficulty of EB is that the overall estimate for a larger geographical area based on a (weighted) sum of EB estimates is not necessarily identical to the corresponding direct estimate such as the overall sample mean. Another difficulty is that EB estimates yield over‐shrinking, which results in the sampling variance smaller than the posterior variance. One way to fix these problems is the benchmarking approach based on the constrained empirical Bayes (CEB) estimators, which satisfy the constraints that the aggregated mean and variance are identical to the requested values of mean and variance. In this paper, we treat the general mixed models, derive asymptotic approximations of the mean squared error (MSE) of CEB and provide second‐order unbiased estimators of MSE based on the parametric bootstrap method. These results are applied to natural exponential families with quadratic variance functions. As a specific example, the Poisson‐gamma model is dealt with, and it is illustrated that the CEB estimates and their MSE estimates work well through real mortality data.     

On the restricted almost unbiased estimators in linear regression

In this paper, the restricted almost unbiased ridge regression estimator and restricted almost unbiased Liu estimator are introduced for the vector of parameters in a multiple linear regression model with linear restrictions. The bias, variance matrices and mean square error (MSE) of the proposed estimators are derived and compared. It is shown that the proposed estimators will have smaller quadratic bias but larger variance than the corresponding competitors in literatures. However, they will respectively outperform the latter according to the MSE criterion under certain conditions. Finally, a simulation study and a numerical example are given to illustrate some of the theoretical results.     

Bayesian predictive modeling for Inverse Gamma-Pareto composite distribution

Inverse Gamma-Pareto composite distribution is considered as a model for heavy tailed data. The maximum likelihood (ML), smoothed empirical percentile (SM), and Bayes estimators (informative and non-informative) for the parameter θ, which is the boundary point for the supports of the two distributions are derived. A Bayesian predictive density is derived via a gamma prior for θ and the density is used to estimate risk measures. Accuracy of estimators of θ and the risk measures are assessed via simulation studies. It is shown that the informative Bayes estimator is consistently more accurate than ML, Smoothed, and the non-informative Bayes estimators.     

Confidence intervals centred on bootstrap smoothed estimators

Bootstrap smoothed (bagged) parameter estimators have been proposed as an improvement on estimators found after preliminary data‐based model selection. A result of Efron in 2014 is a very convenient and widely applicable formula for a delta method approximation to the standard deviation of the bootstrap smoothed estimator. This approximation provides an easily computed guide to the accuracy of this estimator. In addition, Efron considered a confidence interval centred on the bootstrap smoothed estimator, with width proportional to the estimate of this approximation to the standard deviation. We evaluate this confidence interval in the scenario of two nested linear regression models, the full model and a simpler model, and a preliminary test of the null hypothesis that the simpler model is correct. We derive computationally convenient expressions for the ideal bootstrap smoothed estimator and the coverage probability and expected length of this confidence interval. In terms of coverage probability, this confidence interval outperforms the post‐model‐selection confidence interval with the same nominal coverage and based on the same preliminary test. We also compare the performance of the confidence interval centred on the bootstrap smoothed estimator, in terms of expected length, to the usual confidence interval, with the same minimum coverage probability, based on the full model.