The application of generalized ridge and stein-like general minimax rules to multicollinear data

We consider the use of minimax shrinkage estimators for the linear regression mcjel under several loss functions when severe multicollinearity is present. The examples considered illustrate that little or no departure from the least squares estimates is permitted in many cases when the data is highly multicollinear and/or shrinkage is toward a point in the parameter space that does not closely agree with the sample data     

A Note on the Kuks-Olman Estimator

In the linear regression model the unknown parameter vector θ is supposed to vary in a known ellipsoid. Under this parameter constraint Kuks and Olman derived an estimator by demanding a minimax property. Since sometimes the Kuks-Olman estimator takes values outside of the ellipsoid a modification is proposed in the paper. It is shown that this modified variant is a least squares estimator in the restricted model.     

Minimax Linear Smoothers

We consider the problem of estimating the mean of a multivariate distribution. As a general alternative to penalized least squares estimators, we consider minimax estimators for squared error over a restricted parameter space where the restriction is determined by the penalization term. For a quadratic penalty term, the minimax estimator among linear estimators can be found explicitly. It is shown that all symmetric linear smoothers with eigenvalues in the unit interval can be characterized as minimax linear estimators over a certain parameter space where the bias is bounded. The minimax linear estimator depends on smoothing parameters that must be estimated in practice. Using results in Kneip (1994), this can be done using Mallows' C _L -statistic and the resulting adaptive estimator is now asymptotically minimax linear. The minimax estimator is compared to the penalized least squares estimator both in finite samples and asymptotically.     

Two-Stage Bounded-lnfluence Estimators for Simultaneous-Equations Models

This article presents a class of estimators for linear structural models that are robust to heavytailed disturbance distributions, gross errors in either the endogenous or exogenous variables, and certain other model failures. The class of estimators modifies ordinary two-stage least squares by replacing each least squares regression by a bounded-influence regression. Conditions under which the estimators are qualitatively robust, consistent, and asymptotically normal are established, and an empirical example is presented.     

Minimax estimators of regression functions under normalized quadratic loss functions and inequality restrictions

This paper deals with the estimation of a regression function f of unknown form by shrinked least squares estimates calculated on the basis of possibly replicated observa-tions, The estimation loss is chosen in a somewhat more realistic manner then the usual quadratic losses and is given by an adequately weighted sum of squared errors in estimat-ing the values of f at the design points, normalized by the squared norm of the regression function, Shrinked least squares (as special ridge estimators) have been proved by the suthors in special cases to be minimax under all estimatiors.

We investigate the shrinking of least squares estimators with the objective of minimiz-ing the least favourable risk. Here we assume a known lower bound for the magnitude of f and a known upper bound for the difference between f and some simple function approxi-mating f, e.g. we know that f is the sum of a quadratic polynomial and of some     

On exact small sample properties of the minimax generalized ridge regression estimators

The purpose of this paper is to compare sampling performance of the minimax generalized ridge regression estimators considered by Casella (1985) with that of ordinary least squares estimator by numerical calculations of exact mean squared error of these estimators.     

Minimax estimation of a probability of success under LINEX loss

Minimax estimation of a binomial probability under LINEX loss function is considered. It is shown that no equalizer estimator is available in the statistical decision problem under consideration. It is pointed out that the problem can be solved by determining the Bayes estimator with respect to a least favorable distribution having finite support. In this situation, the optimal estimator and the least favorable distribution can be determined only by using numerical methods. Some properties of the minimax estimators and the corresponding least favorable prior distributions are provided depending on the parameters of the loss function. The properties presented are exploited in computing the minimax estimators and the least favorable distributions. The results obtained can be applied to determine minimax estimators of a cumulative distribution function and minimax estimators of a survival function.     

On Necessary and Sufficient Conditions for Ordinary Least Squares Estimators to Be Best Linear Unbiased Estimators

Two often-quoted necessary and sufficient conditions for ordinary least squares estimators to be best linear unbiased estimators are described. Another necessary and sufficient condition is described, providing an additional tool for checking to see whether the covariance matrix of a given linear model is such that the ordinary least squares estimator is also the best linear unbiased estimator. The new condition is used to show that one of the two published conditions is only a sufficient condition.     

On finite-sample properties of adaptive least squares regression estimates

This paper is mainly concerned with deriving finite-sample properties of least squares estimators for the regression function in a nonparametric regression situation under some simplifying assumptions such as normally distributed errors with a common known variance. The selection of basis functions to be used for the construction of an estimator may be regarded as a smoothing problem, and will usually be done in a data-dependent way, A straightforward application of a result by P. J. Kernpthorne yields that, under a squared error loss, all selection procedures are admissible. Furthermore, the minimax approach provides an interpolating estimator, which is often impractical, Therefore, within a certain class of selection procedures an optimal one is determined using the minimax regret principle. It can be seen to behave similarly to the procedure minimizing either an unbiased risk estimator or, equivalently, the C_p-criterion.     

Estimation of a linear model in transformed variables under microaggregation by individual ranking

We consider the least squares estimation of a linear regression model in transformed variables from a data set that has been microaggregated by means of the individual ranking method. It is shown that the least squares estimators are consistent even in the case where variable transformations are carried out after microaggregation. Applying individual ranking techniques to a data set thus guarantees the analytical validity of the microaggregated data for a wide class of statistical models.     

On shrinkage least squares estimation in a parallelism problem

In a multi-sample simple regression model, generally, homogeneity of the regression slopes leads to improved estimation of the intercepts. Analogous to the preliminary test estimators, (smooth) shrinkage least squares estimators of Intercepts based on the James-Stein rule on regression slopes are considered. Relative pictures on the (asymptotic) risk of the classical, preliminary test and the shrinkage least squares estimators are also presented. None of the preliminary test and shrinkage least squares estimators may dominate over the other, though each of them fares well relative to the other estimators.     

Non-minimaxity of linear combinations of restricted location estimators and related problems

The estimation of a linear combination of several restricted location parameters is addressed from a decision-theoretic point of view. Although the corresponding linear combination of the unbiased estimators is minimax under the restricted problem, it has a drawback of taking values outside the restricted parameter space. Thus, it is reasonable to use the linear combination of the restricted estimators such as maximum likelihood or truncated estimators. In this paper, a necessary and sufficient condition for such restricted estimators to be minimax is derived, and it is shown that the restricted estimators are not minimax when the number of the location parameters is large. The condition for minimaxity is examined for some specific distributions. Finally, similar problems of estimating the product and sum of the restricted scale parameters are studied, and it is shown that analogous non-dominance properties appear when the number of the scale parameters is large.     

Another look at Huber's estimator: A new minimax estimator in regression with stochastically bounded noise

Huber's estimator has had a long lasting impact, particularly on robust statistics. It is well known that under certain conditions, Huber's estimator is asymptotically minimax. A moderate generalization in rederiving Huber's estimator shows that Huber's estimator is not the only choice. We develop an alternative asymptotic minimax estimator and name it regression with stochastically bounded noise (RSBN). Simulations demonstrate that RSBN is slightly better in performance, although it is unclear how to justify such an improvement theoretically. We propose two numerical solutions: an iterative numerical solution, which is extremely easy to implement and is based on the proximal point method; and a solution by applying state-of-the-art nonlinear optimization software packages, e.g., SNOPT. Contribution: the generalization of the variational approach is interesting and should be useful in deriving other asymptotic minimax estimators in other problems.     

Admissibility for linear estimators of regression coefficient in a general Gauss–Markoff model under balanced loss function

A generalization of Zellner's balanced loss function is proposed according to unified theory of least squares under a general Gauss–Markoff model. Admissibility of linear estimators is investigated under the balanced loss function. And necessary and sufficient conditions that linear estimators are admissible in a class of homogeneous and nonhomogeneous linear estimators are obtained, respectively.     

Estimating Mark Functions Through Spectral Analysis for Marked Point Patterns

Classical techniques for modeling numerical data associated to a regular grid have been widely developed in the literature. When a trigonometric model for the data is considered, it is possible to use the corresponding least squares (classical) estimators, but when the data are not observed on a regular grid, these estimators do not show appropriate properties. In this article we propose a novel way to model data that is not observed on a regular grid, and we establish a practical criterion, based on the mean squared error (MSE), to objectively decide which estimator should be used in each case: the inappropriate classical or the new unbiased estimator, which has greater variance. Jackknife and cross-validation techniques are used to follow a similar criterion in practice, when the MSE is not known. Finally, we present an application of the methodology to univariate and bivariate data.     

Some properties of the relative squared error approach to linear regression analysis

In order to obtain optimal estimators in a generalized linear regression model we apply the minimax principle to the relative squared error. It turns out that this approach is equivalent to the application of the minimax principle to the absolute squared error when an ellipsoidal prior information set is given. We discuss the admissibility of these minimax estimators. Furthermore, a close relation to a Bayesian approach is derived.     

Confidence regions based on inefficient estimators

We consider an unbiased estimator of a function of mean value parameters, which is not efficient. This inefficient estimator is correlated with a residual vector. Thus, if a unit dispersion is unknown, it is impossible to determine the correct confidence region for a function of mean value parameters via a standard estimator of an unknown dispersion with the exception of the case when the ordinary least squares (OLS) estimator is considered in a model with a special covariance structure such that the OLS and the generalized least squares (GLS) estimator are the same, that is the OLS estimator is efficient. Two different estimators of a unit dispersion independent of an inefficient estimator are derived in a singular linear statistical model. Their quality was verified by simulations for several types of experimental designs. Two new estimators of the unit dispersion were compared with the standard estimators based on the GLS and the OLS estimators of the function of the mean value parameters. The OLS estimator was considered in the incorrect model with a different covariance matrix such that the originally inefficient estimator became efficient. The numerical examples led to a slightly surprising result which seems to be due to data behaviour. An example from geodetic practice is presented in the paper.     

Robust designs for nearly linear regression

In this paper we seek designs and estimators which are optimal in some sense for multivariate linear regression on cubes and simplexes when the true regression function is unknown. More precisely, we assume that the unknown true regression function is the sum of a linear part plus some contamination orthogonal to the set of all linear functions in the L₂ norm with respect to Lebesgue measure. The contamination is assumed bounded in absolute value and it is shown that the usual designs for multivariate linear regression on cubes and simplices and the usual least squares estimators minimize the supremum over all possible contaminations of the expected mean square error. Additional results for extrapolation and interpolation, among other things, are discussed. For suitable loss functions optimal designs are found to have support on the extreme points of our design space.     

Recursive Estimation of GARCH Models

This article develops three recursive on-line algorithms, based on a two-stage least squares scheme for estimating generalized autoregressive conditionally heteroskedastic (GARCH) models. The first one, denoted by 2S-RLS, is an adaptation of the recursive least squares method for estimating autoregressive conditionally heteroskedastic (ARCH) models. The second and the third ones (denoted, respectively, by 2S-PLR and 2S-RML) are adapted versions of the pseudolinear regression (PLR) and the recursive maximum likelihood (RML) methods to the GARCH case. We show that the proposed algorithms give consistent estimators and that the 2S-RLS and the 2S-RML estimators are asymptotically Gaussian. These methods seem very adequate for modeling the sequential feature of financial time series, which are observed on a high-frequency basis. The performance of these algorithms is shown via a simulation study.     

Minimax properties of beta kernel estimators

In this paper, we are interested in the study of beta kernel density estimators from an asymptotic minimax point of view. These estimators allows to estimate density functions with support in [0,1]. It is well-known that beta kernel estimators are - on the contrary of classical kernel estimators - “free of boundary effect” and thus are very useful in practice. The goal of this paper is to prove that there is a price to pay: for very regular density functions or for certain losses, these estimators are not minimax. Nevertheless they are minimax for classical regularities such as regularity of order two or less than two, supposed commonly in the practice and for some classical losses.