Robust regression using iteratively reweighted least-squares

The rapid development of the theory of robust estimation (Huber, 1973) has created a need for computational procedures to produce robust estimates. We will review a number of different computational approaches for robust linear regression but focus on one—iteratively reweighted least-squares (IRLS). The weight functions that we discuss are a part of a semi-portable subroutine library called ROSEPACK (RObust Statistical Estimation PACKage) that has been developed by the authors and Virginia Klema at the Computer Research Center of the National Bureau of Economic Research, Inc. in Cambridge, Mass. with the support of the National Science Foundation. This library (Klema, 1976) makes it relatively simple to implement an IRLS regression package.     

Minimum average risk estimators for coefficients in linear models

We consider the problem of estimating the coefficient vector β of a linear regression model with quadratic loss function. Some biased estimators which utilize the prior information about β are considered. Also studied is the problem of estimating the parameters of an over-identified structural equation from undersized samples.     

A note on some negative dependence notions

A random vector X = (X ₁,…,X _n ) is negatively associated if and only if for every pair of partitions X ₁ = (X _π(1),…,X _π(k)), X ₂ = (X _π(k+1),…,X _π(n)) of X , P( X ₁ ? A, X ₂ ? B) ≤ P( X ₁ ? A)P( X ₂ ? B) whenever A and B are open upper sets and π is any permutation of {1,…,n}. In this paper, we develop some of concepts of negative dependence, which are weaker than negative association but stronger than negative orthant dependence by requiring the above inequality to hold only for some upper sets A and B and applying the arguments in Shaked.     

Probabilistic tolerance design for a subsystem under burr distribution using taguchi's loss functions

Taguchi (1986) has derived tolerances for subcomponents, subsystems, parts and materials in which the relationship between a higher-level (Y) and a lower-level (X) quality characteristic is assumed to be deterministic and linear, namely, Y=_α+βX, without an error term. Tsai (1990) developed a probabilistic tolerance design for a subsystem in which a bivariate normal distribution between the above two quality characteristics as well as Taguchi's quadratic loss function were considered together to develop a closed form solution of the tolerance design for a subsystem. The Burr family is very rich for fitting sample data, and has positive domain. A bivariate Burr distribution can describe a nonlinear relationship between two quality characteristics, hence, it is adopted instead of a bivariate normal distribution and the simple solutions of three probabilistic tolerance desings for a subsystem are obtained for three cases of “nominal-is-best”, “smaller-is-berrer”, and “larger-is-beter” quality characteristics, by using Taguchi’ los functions, respectively.     

Estimation of parameters in latin squares and graeco-latin squares with missing observations

We show that if in an additive model with p-2 MOLS if one omits up to p-1 observations from the same row, the same column or which correspond to the same letter in any of the squares all effects are estimable. On the other hand with only two missing observations not from the same row, the same column or corresponding to the same letter in any of the squares one degree of freedom is lost for each set of effects.     

On least squares estimation for categorical data

We consider a multinomial distribution in which the cell probabilities are known arbitrary functions of a vector parameter θ. It is desired to estimate θ by least squares. Three variations of the least squares approach are investigated, and each is found to be equivalent, in the very strong sense of being algebraically identical, to one of the following estimation procedures: maximum likelihood, minimum χ² and minimum modified χ². Two of these results also apply to the multiple hypergeometric distribution.     

Subset selection procedures for ΔP-superior populations

In some ranking and selection problems it is reasonable to consider any population which is inferior but sufficiently close to the best (t-th best) as acceptable. Under this assumption, this paper studies classes of procedures to meet two possible goals. A and B. Goal A is to select a subset which contains only good populations, while Goal B is of a screening nature and requires selection of a subset of size not exceeding m (1 ≤ m ≤ k) and containing at least one good population. In each case results loading to the determination of the sample size required to attain the goals above with prespecified probability are obtained. Properties of the procedures are discussed.     

Sequential unbiased estimation of parameters in a two-way contingency table

Classical analysis of contingency tables employs (i) fixed sample sizes and (ii) the maximum likelihood and weighted least squares approach to parameter estimation. It is well-known, however, that certain important parameters, such as the main effect and interaction parameters, can neverbe estimated unbiasedly when the sample size is fixed a priori We introduce a sequential unbiased estimator for the cell probabilities subject to log linear constraints. As a simple consequence, we show how parameters such as those mentioned above may. be estimated unbiasedly. Our unbiased estimator for the vector of cell probabilities is shown to be consistent in the sense of Wolfowitz (Ann. Math. Statist. (1947) 18). We give a sufficient condition on a multinomial stopping rule for the corresponding sufficient statistic to be complete. When this condition holds, we have a unique minimum variance unbiased estimator for the cell probabilities.     

On bayes-type estimators of linearly or log-linearly constrained multinomial cell probabilities

Two Bayes-type procedures for estimating a multinomial cell probabilities vector, P, in the presence of linear constraints on the parameters are proposed and illustrated by examples from contingency table analysis. Estimation under log-linear constraints is also considered.     

Confidence intervals for the largest mean from k correlated normal populations

Let X= (X₁,…, X_k)’ be a k-variate (k ≥ 2) normal random vector with unknown population mean vector μ = (μ₁ ,…, μ_k)’ and covariance matrix Σ of order k and let μ_[1] ≤ … ≤ μ_[k] be the ordered values of the μ ’ s. No prior knowledge of the pairing of the μ_[i] with the X_j. (or μ_[i] with the σ_j ²) is assumed for any i and j (1 ≤ i, j ≤ k). Based on a random sample of N independent vector observations on X, this paper considers both upper and lower (one-sided) and two-sided 100γ% (0 < γ < 1) confidence intervals for μ_[k] and μ_[1], the largest and the smallest mean, respectively, when Σ is known and when Σ is equal to σ²R with common unknown variance σ² > 0 and correlation matrix R known, respectively. An optimum two-sided confidence interval via finding the shortest length from this class is also considered. Necessary tables and computer program to actually apply these procedures are provided.     

A Two-stage minimax procedure with screening for selecting the largest normal mean

The problem of selecting the normal population with the largest population mean when the populations have a common known variance is considered. A two-stage procedure is proposed which guarantees the same probability requirement using the indifference-zone approach as does the single-stage procedure of Bechhofer (1954). The two-stage procedure has the highly desirable property that the expected total number of observations required by the procedure is always less than the total number of observations required by the corresponding single-stage procedure, regardless of the configuration of the population means. The saving in expected total number of observations can be substantial, particularly when the configuration of the population means is favorable to the experimenter. The saving is accomplished by screening out “non-contending” populations in the first stage, and concentrating sampling only on “contending” populations in the second stage.

The two-stage procedure can be regarded as a composite one which uses a screening subset-type approach (Gupta (1956), (1965)) in the first stage, and an indifference-zone approach (Bechhofer (1954)) applied to all populations retained in the selected sub-set in the second stage. Constants to implement the procedure for various k and P? are provided, as are calculations giving the saving in expected total sample size if the two-stage procedure is used in place of the corresponding single-stage procedure.     

Balanced block designs

In the present paper an attempt has been made to characterize and unify the three different concepts of balancing in incomplete block designs, namely (i) variance balance, (ii) efficiency balance and (iii) pairwise balance. Simple characterizations of variance balance and efficiency balance have been given using the P matrix. A method of constructing efficiency balanced (EB) and variance balanced designs has also been presented.     

Procedures for investigating outliers when estimating in the general univariate linear situation-nonfull rank case

This paper is a continuation of GGD = Golub, Guttman and Dutter (1973) and DG = Dutter and Guttman (1974a), and will deal with the problem of how to handle outliers in the general univariate linear model situation y = Xθ + ?, ? = N(0, σ²I).when X is not of full rank, and where interest is in the estimation of τ = Bθ, where τ is assumed to be estimable. The special case B = X, that is, interest is in η = Xθ, is discussed.     

Normal approximation in an urn model with indistinguishable balls

For the Bose-Einstein Statistics, where n indistinguishable balls are distributed in m urns such that all the arrangements are equally likely, define the random variables

M_k = number of urns containing exactly k balls each;

N_k = number of urns containing at least k balls each.

We consider the approximation of the distributions of M_k and N_k by suitable normal distributions, for large but finite m. Estimates are found for the error in the approximation to both the probability mass function and the distribution function in each case. These results apply also to the alternative model where no urn is allowed to be empty. The results are illustrated by some numerical examples.     

A new nonparametric procedure designed for simulation studies

This paper demonstrates how certain statistics, computed from a sample of size n (from almost any distribution) may be simulated by using a sequence of substantially less than n random normal variates. Many statistics, θ, including almost all maximum likelihood estimates, can be expressed in terms of the sample trigonometric moments, STM. The STM are asymptotically multivariate normal with a mean vector and variance-covariance matrix easily expressible in terms of equally spaced characteristic function evaluations. Thus one only needs to know the Fourier transform or equivalently the characteristic function associated with elements of any moderate to large i. i. d. sample and have access to a normal random number generator to generate a sequence of STM with distributional properties almost identical to those of STM computed from that sample. These STM can in turn be used to compute the desired statistic θ.     

A note on millers's empirical weights for heteroscedastic linear regression

For heteroscedastic simple linear regression when the variances are proportional to a power of the mean of the response variable, Miller (1986) recommends the following procedure: do a weighted least squares regression with the weights (empirical weights) estimated by the inverse of the appropriate power of the response variable. The practical appeal of this approach is its simplicity.

In this article some of the consequences of this simple procedure are considered. Specifically, the effect of this procedure on the bias of the point estimators of the regression coefficients and on the coverage probabilities of their corresponding confidence intervals is examined. It is found that the performance of the process of employing empirical weights in a weighted least squares regression depends on : (1) the particular regression parameter (slope or intercept) of interest, (2) the appropriate power of the mean of the response variable involved, and (3) the amount of variation in the data about the true regression line.     

Correcting for non-compliance in randomized trials using structural nested mean models

In a randomized trial designed to study the effect of a treatment of interest on the evolution of the mean of a time-dependent outcome variable, subjects are assigned to a treatment regime, or, equivalently, a treatment protocol. Unfortunately, subjects often fail to comply with their assigned regime. From a public health point of view, the causal parameter of interest will often be a function of the treatment differences that would have been observed hadcontrary to fact, all subjects remained on protocol. This paper considers the identification and estimation of these treatment differences based on a new class of structural models, the multivariate structural nested mean models, when reliable estimates of each subject's actual treatment are available. Estimates of “actual treatment” might, for example, be obtained by measuring the amount of “active drug” in the subject's blood or urine at each follow-up visit or by pill counting techniques. In addition, we discuss a natural extension of our methods to observational studies.