Estimation of the mean of a spherically symmetric distribution with constraints on the norm

The authors consider the problem of estimating, under quadratic loss, the mean of a spherically symmetric distribution when its norm is supposed to be known and when a residual vector is available. They give a necessary and sufficient condition for the optimal James‐Stein estimator to dominate the usual estimator. Various examples are given that are not necessarily variance mixtures of normal distributions. Consideration is also given to an alternative class of robust James‐Stein type estimators that take into account the residual vector. A more general domination condition is given for this class.     

A Robust and Almost Fully Efficient M-Estimator

Simultaneous robust estimates of location and scale parameters are derived from a class of M-estimating equations. A coefficient p ( p > 0), which plays a role similar to that of a tuning constant in the theory of M-estimation, determines the estimating equations. These estimating equations may be obtained as the gradient of a strictly convex criterion function. This article shows that the estimators are uniquely defined, asymptotically bi-variate normal and have positive breakdown for some choices of p . When p = 0.12 and p = 0.3, the estimators are almost fully efficient for normal and exponential distributions: efficiencies with respect to the maximum likelihood estimators are 1.00 and 0.99, respectively. It is shown that the location estimator for known scale has the maximum breakdown point 0.5 independent of p , when the target model is symmetric. Also it is shown that the scale estimator has a positive breakdown point which depends on the choice of p . A simulation study finds that the proposed location estimator has smaller variance than the Hodges–Lehmann estimator, Huber's minimax and bisquare M-estimators.     

The minimum L2 distance estimator for Poisson mixture models

A robust estimator is developed for Poisson mixture models with a known number of components. The proposed estimator minimizes the L₂ distance between a sample of data and the model. When the component distributions are completely known, the estimators for the mixing proportions are in closed form. When the parameters for the component Poisson distributions are unknown, numerical methods are needed to calculate the estimators. Compared to the minimum Hellinger distance estimator, the minimum L₂ estimator can be less robust to extreme outliers, and often more robust to moderate outliers.     

Improved minimax estimation of a normal precision matrix

Let S_{p × p} have a Wishart distribution with parameter matrix Σ and n degrees of freedom. We consider here the problem of estimating the precision matrix Σ^?1 under the loss functions L₁(σ) tr (σ) - log |σ| and L₂(σ) = tr (σ). James-Stein-type estimators have been derived for an arbitrary p. We also obtain an orthogonal invariant and a diagonal invariant minimax estimator under both loss functions. A Monte-Carlo simulation study indicates that the risk improvement of the orthogonal invariant estimators over the James-Stein type estimators, the Haff (1979) estimator, and the “testimator” given by Sinha and Ghosh (1987) is substantial.     

Nonlinear Lp-norm estimation: Part II: The asymptotic distribution gf the exponent,p, as a function of the sample kurtosis.

In this paper it will be shown that the exponent p in L_p,-norm P estimation as an explicit function of the sample kurtosis is asymptotically normally distributed. The asymptotic variances of p for two sllch formulae are derived. An alternative formula which implicitly relates p to the sample kurtosis is also discussed.

An adaptive procedure for the selection of p when the underlying error distribution is unknown is also suggested. This procedure is used to verify empirically that the asymptotic distribution of p is normal.     

A MORE GENERAL CRITERION FOR SUBSET SELECTION IN MULTIPLE LINEAR REGRESSION

ABSTRACT

In this article, we propose a more general criterion called S_p -criterion, for subset selection in the multiple linear regression Model. Many subset selection methods are based on the Least Squares (LS) estimator of β, but whenever the data contain an influential observation or the distribution of the error variable deviates from normality, the LS estimator performs ‘poorly’ and hence a method based on this estimator (for example, Mallows’ C_p -criterion) tends to select a ‘wrong’ subset. The proposed method overcomes this drawback and its main feature is that it can be used with any type of estimator (either the LS estimator or any robust estimator) of β without any need for modification of the proposed criterion. Moreover, this technique is operationally simple to implement as compared to other existing criteria. The method is illustrated with examples.     

Fractional principal components regression: a general approach to biased estimators

Several biased estimators have been proposed as alternatives to the least squares estimator when multicollinearity is present in the multiple linear regression model. The ridge estimator and the principal components estimator are two techniques that have been proposed for such problems. In this paper the class of fractional principal component estimators is developed for the multiple linear regression model. This class contains many of the biased estimators commonly used to combat multicollinearity. In the fractional principal components framework, two new estimation techniques are introduced. The theoretical performances of the new estimators are evaluated and their small sample properties are compared via simulation with the ridge, generalized ridge and principal components estimators     

On least absolute values estimation

The resistance of least absolute values (L₁) estimators to outliers and their robustness to heavy-tailed distributions make these estimators useful alternatives to the usual least squares estimators. The recent development of efficient algorithms for L₁ estimation in linear models has permitted their use in practical data analysis. Although in general the L₁ estimators are not unique, there are a number of properties they all share. The set of all L₁ estimators for a given model and data set can be characterized as the convex hull of some extreme estimators. Properties of the extreme estimators and of the L₁-estimate set are considered.     

Density estimation under the koziol-green model of censoring by penalized likelihood methods

We propose two density estimators of the survival distribution in the setting of the Koziol-Green random-censoring model. The estimators are obtained by maximum-penalized-likelihood methods, and we provide an algorithm for their numerical evaluation. We establish the strong consistency of the estimators in the Hellinger metric, the L_p-norms, p= 1,2, ∞, and a Sobolev norm, under mild conditions on the underlying survival density and the censoring distribution.     

Estimating the Expectation of the Log-Likelihood with Censored Data for Estimator Selection

A criterion for choosing an estimator in a family of semi-parametric estimators from incomplete data is proposed. This criterion is the expected observed log-likelihood (ELL). Adapted versions of this criterion in case of censored data and in presence of explanatory variables are exhibited. We show that likelihood cross-validation (LCV) is an estimator of ELL and we exhibit three bootstrap estimators. A simulation study considering both families of kernel and penalized likelihood estimators of the hazard function (indexed on a smoothing parameter) demonstrates good results of LCV and a bootstrap estimator called ELL_bboot . We apply the ELL_bboot criterion to compare the kernel and penalized likelihood estimators to estimate the risk of developing dementia for women using data from a large cohort study.     

Estimation of a multivariate mean with constraints on the norm

Estimation of the mean θ of a spherical distribution with prior knowledge concerning the norm ||θ|| is considered. The best equivariant estimator is obtained for the local problem ||θ|| = λ₀, and its risk is evaluated. This yields a sharp lower bound for the risk functions of a large class of estimators. The risk functions of the best equivariant estimator and the best linear estimator are compared under departures from the assumption ||θ|| = λ₀.     

On estimation of the scale matrix of the multivariate f distribution

Let F _p×phave a multivariate F distribution with a scale p×p matrix Δ and degrees of freedom k₁ and k₂ such that k_i - p - 1 > 0, i = 1,2. The estimation of Δ under entropy and squared error loss functions are considered. In both cases a new class of orthogonally invariant estimators are obtained which dominate the best unbiased estimator.     

Meta-analysis,pretest, and shrinkage estimation of kurtosis parameters

An asymptotic theory for the improved estimation of kurtosis parameter vector is developed for multi-sample case using uncertain prior information (UPI) that several kurtosis parameters are the same. Meta-analysis is performed to obtain pooled estimator, as it is a statistical methodology for pooling quantitative evidence. Pooled estimator is a good choice when assumption of homogeneity holds but it becomes inconsistent as assumption violates, therefore pretest and Stein-type shrinkage estimators are proposed as they combine sample and nonsample information in a superior way. Asymptotic properties of suggested estimators are discussed and their risk comparisons are also mentioned.     

Lp-estimators in ARCH models

For ergodic ARCH processes, we introduce a one-parameter family of L_p-estimators. The construction is based on the concept of weighted M-estimators. Under weak assumptions on the error distribution, the consistency is established. The asymptotic normality is proved for the special cases p=1 and 2. To prove the asymptotic normality of the L₁-estimator, one needs the existence of a density of the squares of the errors, whereas for the L₂-estimator the existence of fourth moments is assumed. The asymptotic covariance matrix of the estimator depends on the unknown parameter which can be substituted by consistent estimators. For the L₁-estimator we construct a kernel estimator for the unknown density of the square of the errors.     

Stabilizing the Performance of Kurtosis Estimator of Multivariate Data

The estimation of the kurtosis parameter of the underlying distribution plays a central role in many statistical applications. The central theme of the article is to improve the estimation of the kurtosis parameter using a priori information. More specifically, we consider the problem of estimating kurtosis parameter of a multivariate population when some prior information regarding the the parameter is available. The rationale is that the sample estimator of the kurtosis parameter has a large estimation error. In this situation we consider shrinkage and pretest estimation methodologies and reappraise their statistical properties. The estimation based on these strategies yield relatively smaller estimation error in comparison with the sample estimator in the candidate subspace. A large sample theory of the suggested estimators are developed and compared. The results demonstrate that suggested estimators outperform the estimator based on the sample data only in the candidate subspace. In an effort to appreciate the relative behavior of the estimators in a finite sample scenario, a Monte-carlo simulation study is planned and performed. The result of simulation study strongly corroborates the asymptotic result. To illustrate the application of the estimators, some example are showcased based on recently published data.     

Dominating james-stein positive-part estimator for normal mean with unknown covariance matrix

Assume that we have a random sample of size n from p-variate normal population and we wish to estimate the mean vector under quadratic loss with respect to the inverse of the unknown covariance matrix, A class of superior estimators to James-Stein positive part estimator is given when n>max{9p+10,13p-7}, based on the argument by Shao and Strawderman(1994).     

Selection of Variables in Multivariate Regression Models for Large Dimensions

The Akaike information criterion, AIC, and Mallows’ C _p statistic have been proposed for selecting a smaller number of regressors in the multivariate regression models with fully unknown covariance matrix. All of these criteria are, however, based on the implicit assumption that the sample size is substantially larger than the dimension of the covariance matrix. To obtain a stable estimator of the covariance matrix, it is required that the dimension of the covariance matrix is much smaller than the sample size. When the dimension is close to the sample size, it is necessary to use ridge-type estimators for the covariance matrix. In this article, we use a ridge-type estimators for the covariance matrix and obtain the modified AIC and modified C _p statistic under the asymptotic theory that both the sample size and the dimension go to infinity. It is numerically shown that these modified procedures perform very well in the sense of selecting the true model in large dimensional cases.     

Generalized Least Squares Model Averaging

In this article, we propose a method of averaging generalized least squares estimators for linear regression models with heteroskedastic errors. The averaging weights are chosen to minimize Mallows’ C_p-like criterion. We show that the weight vector selected by our method is optimal. It is also shown that this optimality holds even when the variances of the error terms are estimated and the feasible generalized least squares estimators are averaged. The variances can be estimated parametrically or nonparametrically. Monte Carlo simulation results are encouraging. An empirical example illustrates that the proposed method is useful for predicting a measure of firms’ performance.     

A New Kernel Distribution Function Estimator Based on a Non-parametric Transformation of the Data

Abstract.  A new kernel distribution function (df) estimator based on a non-parametric transformation of the data is proposed. It is shown that the asymptotic bias and mean squared error of the estimator are considerably smaller than that of the standard kernel df estimator. For the practical implementation of the new estimator a data-based choice of the bandwidth is proposed. Two possible areas of application are the non-parametric smoothed bootstrap and survival analysis. In the latter case new estimators for the survival function and the mean residual life function are derived.     

Comparisons of the <Emphasis Type="Italic">r</Emphasis> − <Emphasis Type="Italic">k</Emphasis> class estimator to the ordinary least squares estimator under the Pitman’s closeness criterion

In the presence of multicollinearity, the r − k class estimator is proposed as an alternative to the ordinary least squares (OLS) estimator which is a general estimator including the ordinary ridge regression (ORR), the principal components regression (PCR) and the OLS estimators. Comparison of competing estimators of a parameter in the sense of mean square error (MSE) criterion is of central interest. An alternative criterion to the MSE criterion is the Pitman’s (1937) closeness (PC) criterion. In this paper, we compare the r − k class estimator to the OLS estimator in terms of PC criterion so that we can get the comparison of the ORR estimator to the OLS estimator under the PC criterion which was done by Mason et al. (1990) and also the comparison of the PCR estimator to the OLS estimator by means of the PC criterion which was done by Lin and Wei (2002).