An Appreciation of Balanced Loss Functions Via Regret Loss

We examine balanced loss functions, which account for both estimation error and goodness of fit (or proximity to a “target” estimator), in terms of their regret losses, providing new insight and interpretations. This also shows a connection between quadratic balanced loss and usual quadratic loss, which easily converts frequentist and Bayesian results for quadratic loss to related results for quadratic balanced loss and vice versa. Some implications of these results for Stein-rule estimators under linear regression are discussed. We also examine regret losses corresponding to several non-quadratic balanced loss functions.