Formulating robust regression estimation as an optimum allocation problem

The Mallows-type estimator, one of the most reasonable bounded influence estimators, often downweights leverage points regardless of the magnitude of the corresponding residual, and this could imply a loss of efficiency. In this article, we consider whether the efficiency of this bounded influence estimator could be improved by regarding both the robust x -distance and the residual size. We develop a new robust procedure based on the ideas of the Mallows-type estimator and the general robust recipe, where data been cleaned by pulling outliers towards their fitted values. Our basic idea is to formulate the robust estimation as an allocation problem, where the objective function is a Huber-type "loss" function, but the pulling resource is restricted. Using a mathematical programming technique, the pulling resource is optimally allocated to influential points <$>({x}_i, y_i)<$> with respect to residual size and given weights, <$>w({x}_i)<$>. Three previously published approaches are compared to our proposal via simulated experiments. In the case of contaminated data by regression outliers and "good" leverage points, the proposed robust estimator is a reasonable bounded influence estimator concerning both efficiency and norm of bias. In addition, the proposed approach offers the potential to establish constraints for the regression parameters and also may potentially provide insight regarding outlier detection.     

Improved approaches to interval estimation of effective doses for a logistic dose-response curve

Asymptotic confidence (delta) intervals and intervals based upon the use of Fieller's theorem are alternative methods for constructing intervals for the <$>\gamma<$>% effective doses (ED<$>_\gamma<$>). Sitter and Wu (1993) provided a comparison of the two approaches for the ED<$>_{50}<$>, for the case in which a logistic dose response curve is assumed. They showed that the Fieller intervals are generally superior. In this paper, we introduce two new families of intervals, both of which include the delta and Fieller intervals as special cases. In addition we consider interval estimation of the ED<$>_{90}<$> as well as the ED<$>_{50}<$>. We provide a comparison of the various methods for the problem of constructing a confidence interval for the ED<$>_\gamma<$>.     

Limit distributions of unbiased estimators in natural exponential families

We obtain the possible limit distributions of unbiased estimators of functions of the parameter of a natural exponential family. The limit distribution depends on <$>j<$>, the order of the first non-zero derivative at the true (but usually unknown) value of the parameter. We show that if <$>j \geq 2<$> then the umvu and the maximum likelihood estimators are not asymptotically equivalent.     

Markov Chain Markov Field dynamics: Models and statistics

This study deals with time dynamics of Markov fields defined on a finite set of sites with state space <$>E<$>, focussing on Markov Chain Markov Field (MCMF) evolution. Such a model is characterized by two families of potentials: the instantaneous interaction potentials, and the time delay potentials. Four models are specified: auto-exponential dynamics (<$>E = {\of R}^+<$>), auto-normal dynamics (<$>E = {\of R}<$>), auto-Poissonian dynamics (<$>E = {\of N}<$>) and auto-logistic dynamics ( E qualitative and finite). Sufficient conditions ensuring ergodicity and strong law of large numbers are given by using a Lyapunov criterion of stability, and the conditional pseudo-likelihood statistics are summarized. We discuss the identification procedure of the two Markovian graphs and look for validation tests using martingale central limit theorems. An application to meteorological data illustrates such a modelling.     

On the efficiencies of some common quick estimators

The quick estimators of location and scale have broad applications and are widely used. For a variety of symmetric populations we obtain the quantiles and the weights for which the asymptotic variances of the quick estimators are minimum. These optimal quick estimators are then used to obtain the asymptotic relative efficiencies of the commonly used estimators such as trimean. gastwirth. median, midrange. and interquartile range with respect to the optimal quick estimators in order to determine a choice among them and to check whether they are unacceptably poor. In the process it is seen that the interquartile range is the optimal quick estimator of scale for Cauchy populations; but the interdecile range is in general preferable. Also the optimal estimator of the location for the logistic distribution puts weights 0.3 on each of the two quartiles and 0.4 on the median. It is shown that for the symmetric distributions, such as the beta and Tukey- lambda with [d] > 0, which have finite support and short tails, i.e. the tail exponents (Parzen, 1979) satisfy [d] < 1, the midrange and the range are the optimal quick estimators of location and scale respectively if [d] < 1/2. The class of such distributions Include the distributions with high discontinuous tails, e.g. Tukey-lambda with [d] > 1, as well as some distributions with p.d.f.'s going to zero at the ends of the finite support, such as Tukey-lambda with 1/2 < [d] < 1. As a byproduct an interesting tail correspondence between beta and Tukey-lambda distributions is seen.     

Location adjustment for the minimum volume ellipsoid estimator

Estimating multivariate location and scatter with both affine equivariance and positive breakdown has always been difficult. A well-known estimator which satisfies both properties is the Minimum Volume Ellipsoid Estimator (MVE). Computing the exact MVE is often not feasible, so one usually resorts to an approximate algorithm. In the regression setup, algorithms for positive-breakdown estimators like Least Median of Squares typically recompute the intercept at each step, to improve the result. This approach is called intercept adjustment. In this paper we show that a similar technique, called location adjustment, can be applied to the MVE. For this purpose we use the Minimum Volume Ball (MVB), in order to lower the MVE objective function. An exact algorithm for calculating the MVB is presented. As an alternative to MVB location adjustment we propose L ₁ location adjustment, which does not necessarily lower the MVE objective function but yields more efficient estimates for the location part. Simulations compare the two types of location adjustment. We also obtain the maxbias curves of L ₁ and the MVB in the multivariate setting, revealing the superiority of L ₁.     

Small sample study on estimators of life distributions and of mean survival times using randomly censored data

Whereas large-sample properties of the estimators of survival distributions using censored data have been studied by many authors, exact results for small samples have been difficult to obtain. In this paper we obtain the exact expression for the ath moment (a > 0) of the Bayes estimator of survival distribution using the censored data under proportional hazard model. Using the exact expression we compute the exact mean, variance and MSE of the Bayes estimator. Also two estimators ofthe mean survival time based on the Kaplan-Meier estimator and the Bayes estimator are compared for small samples under proportional hazards.     

Estimating capability index cpk for processes with asymmetric tolerances

Pearn and Chen (1996) considered the process capability index C_pk, and investigated the statistical properties of its natural estimator under various process conditions. Their investigation, however, was restricted to processes with symmetric tolerances. Recently, Pearn and Chen (1998) considered a generalization of C_pk, referred to as C^? _pk, to cover processes with asymmetric tolerances. They investigated the statistical properties of the natural estimator of C^? _pk, and obtained the exact formulae for the expected value and variance. In this paper, we consider a new estimator of C^? _pk, assuming the knowledge on P(LI > T) = p is available, where 0 > p > 1, which can be obtained from historical information of a stable process. We obtain the exact distribution of the new estimator assuming the process characteristic follows the normal distribution. We show that the new estimator is consistent, asymptotically unbiased, which converges to a mixture of two normal distributions. We also show that by adding suitable correction factors to the new estimator, we may obtain the UMVUE and the MLE of the generalization C^? _pk.     

Estimation of the mean of the selected gamma population

Let л₁ and л₂ denote two independent gamma populations G(α₁, p) and G(α₂, p) respectively. Assume α(i=1,2)are unknown and the common shape parameter p is a known positive integer. Let Y_i denote the sample mean based on a random sample of size n from the i-th population. For selecting the population with the larger mean, we consider, the natural rule according to which the population corresponding to the larger Y_i is selected. We consider? in this paper, the estimation of M, the mean of the selected population. It is shown that the natural estimator is positively biased. We obtain the uniformly minimum variance unbiased estimator(UMVE) of M. We also consider certain subclasses of estikmators of the form c₁x₍₁₎ +c₁x₍₂₎ and derive admissible estimators in these classes. The minimazity of certain estimators of interest is investigated. Itis shown that p(p+1)^-1x₍₁₎ is minimax and dominates the UMVUE. Also UMVUE is not minimax.     

Bias reduction of the maximum-likelihood estimator for a conditional Gaussian MA(1) model

In this paper, we consider an estimation for the unknown parameters of a conditional Gaussian MA(1) model. In the majority of cases, a maximum-likelihood estimator is chosen because the estimator is consistent. However, for small sample sizes the error is large, because the estimator has a bias of O(n^{? 1}). Therefore, we provide a bias of O(n^{? 1}) for the maximum-likelihood estimator for the conditional Gaussian MA(1) model. Moreover, we propose new estimators for the unknown parameters of the conditional Gaussian MA(1) model based on the bias of O(n^{? 1}). We investigate the properties of the bias, as well as the asymptotical variance of the maximum-likelihood estimators for the unknown parameters, by performing some simulations. Finally, we demonstrate the validity of the new estimators through this simulation study.     

Minimax estimation of a lower-bounded scale parameter of a gamma distribution for scale-invariant squared-error loss

Let X have a gamma distribution with known shape parameter θr;aL and unknown scale parameter θ. Suppose it is known that θ ≥ a for some known a > 0. An admissible minimax estimator for scale-invariant squared-error loss is presented. This estimator is the pointwise limit of a sequence of Bayes estimators. Further, the class of truncated linear estimators C = {θ_ρ|θ_ρ(x) = max(a, ρ), ρ > 0} is studied. It is shown that each θ_ρ is inadmissible and that exactly one of them is minimax. Finally, it is shown that Katz's [Ann. Math. Statist., 32, 136–142 (1961)] estimator of θ is not minimax for our loss function. Some further properties of and comparisons among these estimators are also presented.     

On estimation of R=P(Y<X) for exponential distribution under progressive type-II censoring

This paper deals with the estimation of the stress–strength parameter R=P(Y<X), when X and Y are independent exponential random variables, and the data obtained from both distributions are progressively type-II censored. The uniformly minimum variance unbiased estimator and the maximum-likelihood estimator (MLE) are obtained for the stress–strength parameter. Based on the exact distribution of the MLE of R, an exact confidence interval of R has been obtained. Bayes estimate of R and the associated credible interval are also obtained under the assumption of independent inverse gamma priors. An extensive computer simulation is used to compare the performances of the proposed estimators. One data analysis has been performed for illustrative purpose.     

On estimation of the scale matrix of the multivariate f distribution

Let F _p×phave a multivariate F distribution with a scale p×p matrix Δ and degrees of freedom k₁ and k₂ such that k_i - p - 1 > 0, i = 1,2. The estimation of Δ under entropy and squared error loss functions are considered. In both cases a new class of orthogonally invariant estimators are obtained which dominate the best unbiased estimator.     

Simultaneous robust estimation of location and scale parameters: A minimum-distance approach

Simultaneous robust estimates of location and scale parameters are derived from minimizing a minimum-distance criterion function. The criterion function measures the squared distance between the pth power (p > 0) of the empirical distribution function and the pth power of the imperfectly determined model distribution function over the real line. We show that the estimator is uniquely defined, is asymptotically bivariate normal and for p > 0.3 has positive breakdown. If the scale parameter is known, when p = 0.9 the asymptotic variance (1.0436) of the location estimator for the normal model is smaller than the asymptotic variance of the Hodges-Lehmann (HL)estimator (1.0472). Efficiencies with respect to HL and maximum-likelihood estimators (MLE) are 1.0034 and 0.9582, respectively. Similarly, if the location parameter is known, when p = 0.97 the asymptotic variance (0.6158) of the scale estimator is minimum. The efficiency with respect to the MLE is 0.8119. We show that the estimator can tolerate more corrupted observations at oo than at – for p < 1, and vice versa for p > 1.     

Improved estimators for the selected location parameters

_i , i = 1, 2, ..., k be k independent exponential populations with different unknown location parameters θ _i , i = 1, 2, ..., k and common known scale parameter σ. Let Y _i denote the smallest observation based on a random sample of size n from the i-th population. Suppose a subset of the given k population is selected using the subset selection procedure according to which the population π _i is selected iff Y _i ≥Y ₍₁₎−d, where Y ₍₁₎ is the largest of the Y _i 's and d is some suitable constant. The estimation of the location parameters associated with the selected populations is considered for the squared error loss. It is observed that the natural estimator dominates the unbiased estimator. It is also shown that the natural estimator itself is inadmissible and a class of improved estimators that dominate the natural estimator is obtained. The improved estimators are consistent and their risks are shown to be O(kn ⁻²). As a special case, we obtain the coresponding results for the estimation of θ₍₁₎, the parameter associated with Y ₍₁₎. Received: January 6, 1998; revised version: July 11, 2000     

Shrinkage ridge regression in partial linear models

ABSTRACT

In this paper, shrinkage ridge estimator and its positive part are defined for the regression coefficient vector in a partial linear model. The differencing approach is used to enjoy the ease of parameter estimation after removing the non parametric part of the model. The exact risk expressions in addition to biases are derived for the estimators under study and the region of optimality of each estimator is exactly determined. The performance of the estimators is evaluated by simulated as well as real data sets.     

ROBUST RIDGE REGRESSION BASED ON AN M-ESTIMATOR

Consider the linear regression model y =β₀1 +Xβ+ in the usual notation. It is argued that the class of ordinary ridge estimators obtained by shrinking the least squares estimator by the matrix (X¹X + kI)^-1X'X is sensitive to outliers in the ^variable. To overcome this problem, we propose a new class of ridge-type M-estimators, obtained by shrinking an M-estimator (instead of the least squares estimator) by the same matrix. Since the optimal value of the ridge parameter k is unknown, we suggest a procedure for choosing it adaptively. In a reasonably large scale simulation study with a particular M-estimator, we found that if the conditions are such that the M-estimator is more efficient than the least squares estimator then the corresponding ridge-type M-estimator proposed here is better, in terms of a Mean Squared Error criteria, than the ordinary ridge estimator with k chosen suitably. An example illustrates that the estimators proposed here are less sensitive to outliers in the y-variable than ordinary ridge estimators.     

PMSE performance of two different types of preliminary test estimators under a multivariate t error term

Abstract

In this paper, assuming that the error terms follow a multivariate t distribution, we derive the exact formula for the predictive mean squared error (PMSE) of two different types of pretest estimators. It is shown analytically that one of the pretest estimator dominates the SR estimator if a critical value of the pretest is chosen appropriately. Also, we compare the PMSE of the pretest estimators with the MMSE, AMMSE, SR and PSR estimators by numerical evaluations. Our results show that the pretest estimators dominate the OLS estimator for all combinations when the degrees of freedom is not more than 5.     

The Distribution of Sums of Certain I.I.D. Pareto Variates

ABSTRACT

Though the Pareto distribution is important to actuaries and economists, an exact expression for the distribution of the sum of n i.i.d. Pareto variates has been difficult to obtain in general. This article considers Pareto random variables with common probability density function (pdf) f(x) = (α/β) (1 + x/β)^α+1 for x > 0, where α = 1,2,… and β > 0 is a scale parameter. To date, explicit expressions are known only for a few special cases: (i) α = 1 and n = 1,2,3; (ii) 0 < α < 1 and n = 1,2,…; and (iii) 1 < α < 2 and n = 1,2,…. New expressions are provided for the more general case where β > 0, and α and n are positive integers. Laplace transforms and generalized exponential integrals are used to derive these expressions, which involve integrals of real valued functions on the positive real line. An important attribute of these expressions is that the integrands involved are non oscillating.     

PORT Hill and Moment Estimators for Heavy-Tailed Models

In this article, we use the peaks over random threshold (PORT)-methodology, and consider Hill and moment PORT-classes of extreme value index estimators. These classes of estimators are invariant not only to changes in scale, like the classical Hill and moment estimators, but also to changes in location. They are based on the sample of excesses over a random threshold, the order statistic X _[np]+1:n , 0 ≤ p < 1, being p a tuning parameter, which makes them highly flexible. Under convenient restrictions on the underlying model, these classes of estimators are consistent and asymptotically normal for adequate values of k, the number of top order statistics used in the semi-parametric estimation of the extreme value index γ. In practice, there may however appear a stability around a value distant from the target γ when the minimum is chosen for the random threshold, and attention is drawn for the danger of transforming the original data through the subtraction of the minimum. A new bias-corrected moment estimator is also introduced. The exact performance of the new extreme value index PORT-estimators is compared, through a large-scale Monte-Carlo simulation study, with the original Hill and moment estimators, the bias-corrected moment estimator, and one of the minimum-variance reduced-bias (MVRB) extreme value index estimators recently introduced in the literature. As an empirical example we estimate the tail index associated to a set of real data from the field of finance.