Linear least squares estimates and nonlinear means

The consistency and asymptotic normality of a linear least squares estimate of the form (X'X)^-X'Y when the mean is not Xβ is investigated in this paper. The least squares estimate is a consistent estimate of the best linear approximation of the true mean function for the design chosen. The asymptotic normality of the least squares estimate depends on the design and the asymptotic mean may not be the best linear approximation of the true mean function. Choices of designs which allow large sample inferences to be made about the best linear approximation of the true mean function are discussed.     

General formulae for expectations, variances and covariances of the mean squares for staggered nested designs

Staggered nested experimental designs are the most popular class of unbalanced nested designs. Using a special notation which covers the particular structure of the staggered nested design, this paper systematically derives the canonical form for the arbitrary m-factors. Under the normality assumption for every random variable, a vector comprising m canonical variables from each experimental unit is normally independently and identically distributed. Every sum of squares used in the analysis of variance (ANOVA) can be expressed as the sum of squares of the corresponding canonical variables. Hence, general formulae for the expectations, variances and covariances of the mean squares are directly obtained from the canonical form. Applying the formulae, the explicit forms of the ANOVA estimators of the variance components and unbiased estimators of the ratios of the variance components are introduced in this paper. The formulae are easily applied to obtain the variances and covariances of any linear combinations of the mean squares, especially the ANOVA estimators of the variance components. These results are eff ectively applied for the standardization of measurement methods.     

The analysis of non-orthogonal designs with many classifications

The least squares analysis of non-orthogonal designs with many classifications is considered. A unified simpler approach than the existing methods is derived and simple expressions for the various sums of squares are given. The paper also generalizes the canonical forms of Pearce Jeffers (1971) for the adjusted treatment sum of squares and the error sum of squares in block designs to designs with several non-orthogonal classifications.     

Optimal designs for calibration of orientations

This paper concerns designed experiments involving observations of orientations following the models of Prentice (1989) and Rivest &Chang (2006). The authors state minimal conditions on the designs for consistent least squares estimation of the matrix parameters in these models. The conditions are expressed in terms of the axes and rotation angles of the design orientations. The authors show that designs satisfying U₁ + … + U_n = 0 are optimal in the sense of minimizing the estimation error average angular distance. The authors give constructions of optimal n‐point designs when n ≥ 4 and they compare the performance of several designs through approximations and simulation.     

Another look at plotting positions

Probability paper was used as early as 1896, and was mentioned in the literature more than 30 times before 1950, mainly by hydrologists, most of whom used the plotting position (i-0.5)/n proposed by Hazen (1914). Gumbel (1942a) considered the modal position (i-1)/(n-1) and the mean position i/(n+1) [the latter proposed by Weibull (1939a,b)], and chose the latter. Lebedev (1952) and others proposed the use of (i-0.3)/(n+0.4), which is approximately the median position advocated by Johnson (1951). Blom (1958) sug-gested (i-α)/(n-2α+1), where a is a constant (usually 0 ≤ α ≤ 1), which includes all of the above plotting positions as special cases. Moreover, by proper choice of α, one can approximate F[E(x_i)], the position proposed by Kimball (1946), for any distri-bution of interest. Gumbel (1954) stated five postulates which plotting positions should satisfy. Chernoff & Lieberman (1954) discussed the optimum choice of plotting positions in various situ-ations. It is clear that the optimum plotting position depends on the use that is to be made of the results and may also depend on the underlying distribution. The author endeavors to formulate recommendations as to the best choice in various situations.     

Robust regression for estimating the Burr XII parameters with outliers

The Burr XII distribution offers a more flexible alternative to the lognormal, log-logistic and Weibull distributions. Outliers can occur during reliability life testing. Thus, we need an efficient method to estimate the parameters of the Burr XII distribution for censored data with outliers. The objective of this paper is to present a robust regression (RR) method called M-estimator to estimate the parameters of a two-parameter Burr XII distribution based on the probability plotting procedure for both the complete and multiply-censored data with outliers. The simulation results show that the RR method outperforms the unweighted least squares and maximum likelihood methods in most cases in terms of bias and errors in the root mean square.     

Random balanced resampling: A new method for estimating variance components in unbalanced designs

Variance components in factorial designs with balanced data are commonly estimated by equating mean squares to expected mean squares. For unbalanced data, the usual extensions of this approach are the Henderson methods, which require formulas that are rather involved. Alternatively, maximum likelihood estimation based on normality has been proposed. Although the algorithm for maximum likelihood is computationally complex, programs exist in some statistical packages. This article introduces a simpler method, that of creating a balanced data set by resampling from the original one. Revised formulas for expected mean squares are presented for the two-way case; they are easily generalized to larger factorial designs. The results of a number of simulation studies indicate that, in certain types of designs, the proposed method has performance advantages over both the Henderson Method I and maximum likelihood estimators.     

A Weighted Least Squares Approach to Levene's Test of Homogeneity of Variance

In 1960 Levene suggested a potentially robust test of homogeneity of variance based on an ordinary least squares analysis of variance of the absolute values of mean-based residuals. Levene's test has since been shown to have inflated levels of significance when based on the F-distribution, and tests a hypothesis other than homogeneity of variance when treatments are unequally replicated, but the incorrect formulation is now standard output in several statistical packages. This paper develops a weighted least squares analysis of variance of the absolute values of both mean-based and median-based residuals. It shows how to adjust the residuals so that tests using the F -statistic focus on homogeneity of variance for both balanced and unbalanced designs. It shows how to modify the F -statistics currently produced by statistical packages so that the distribution of the resultant test statistic is closer to an F-distribution than is currently the case. The weighted least squares approach also produces component mean squares that are unbiased irrespective of which variable is used in Levene's test. To complete this aspect of the investigation the paper derives exact second-order moments of the component sums of squares used in the calculation of the mean-based test statistic. It shows that, for large samples, both ordinary and weighted least squares test statistics are equivalent; however they are over-dispersed compared to an F variable.     

Robust designs for one-point extrapolation

We consider the construction of designs for the extrapolation of a regression response to one point outside of the design space. The response function is an only approximately known function of a specified linear function. As well, we allow for variance heterogeneity. We find minimax designs and corresponding optimal regression weights in the context of the following problems: (P1) for nonlinear least squares estimation with homoscedasticity, determine a design to minimize the maximum value of the mean squared extrapolation error (MSEE), with the maximum being evaluated over the possible departures from the response function; (P2) for nonlinear least squares estimation with heteroscedasticity, determine a design to minimize the maximum value of MSEE, with the maximum being evaluated over both types of departures; (P3) for nonlinear weighted least squares estimation, determine both weights and a design to minimize the maximum MSEE; (P4) choose weights and design points to minimize the maximum MSEE, subject to a side condition of unbiasedness. Solutions to (P1)–(P4) are given in complete generality. Numerical comparisons indicate that our designs and weights perform well in combining robustness and efficiency. Applications to accelerated life testing are highlighted.     

Properties of Designs in Experiments on Spatial Lattices

ABSTRACT

When spatial variation is present in experiments, it is clearly sensible to use designs with favorable properties under both generalized and ordinary least squares. This will make the statistical analysis more robust to misspecification of the spatial model than would be the case if designs were based solely on generalized least squares. In this article, treatment information is introduced as a way of studying the ordinary least squares properties of designs. The treatment information is separated into orthogonal frequency or polynomial components which are assumed to be independent under the spatial model. The well-known trend-resistant designs are those with no treatment information at the very low order frequency or polynomial components which tend to have the higher variances under the spatial model. Ideally, designs would be chosen with all the treatment information distributed at the higher-order components. However, the results in this article show that there are limits on how much trend resistance can be achieved as there are many constraints on the treatment information. In addition, appropriately chosen Williams squares designs are shown to have favorable properties under both ordinary and generalized least squares. At all times, the ordinary least squares properties of the designs are balanced against the generalized least squares objectives of optimizing neighbor balance.     

Estimating coefficients of single-index regression models by minimizing variation

This paper proposes a novel estimation of coefficients in single-index regression models. Unlike the traditional average derivative estimation [Powell JL, Stock JH, Stoker TM. Semiparametric estimation of index coefficients. Econometrica. 1989;57(6):1403–1430; Hardle W, Thomas M. Investigating smooth multiple regression by the method of average derivatives. J Amer Statist Assoc. 1989;84(408):986–995] and semiparametric least squares estimation [Ichimura H. Semiparametric least squares (sls) and weighted sls estimation of single-index models. J Econometrics. 1993;58(1):71–120; Hardle W, Hall P, Ichimura H. Optimal smoothing in single-index models. Ann Statist. 1993;21(1):157–178], the procedure developed in this paper is to estimate the coefficients directly by minimizing the mean variation function and does not involve estimating the link function nonparametrically. As a result, it avoids the selection of the bandwidth or the number of knots, and its implementation is more robust and easier. The resultant estimator is shown to be consistent. Numerical results and real data analysis also show that the proposed procedure is more applicable against model free assumptions.     

ON OPTIMIZING MULTI-LEVEL DESIGNS: POWER UNDER BUDGET CONSTRAINTS

This paper derives a procedure for efficiently allocating the number of units in multi‐level designs given prespecified power levels. The derivation of the procedure is based on a constrained optimization problem that maximizes a general form of a ratio of expected mean squares subject to a budget constraint. The procedure makes use of variance component estimates to optimize designs during the budget formulating stages. The method provides more general closed form solutions than other currently available formulae. As such, the proposed procedure allows for the determination of the optimal numbers of units for studies that involve more complex designs. A method is also described for optimizing designs when variance component estimates are not available. Case studies are provided to demonstrate the method.     

Maximum likelihood estimation of variance components in a completely random bib design

ABSTRACT

The purposes of this paper are to abstract from a number of articles variance component estimation procedures which can be used for completely random balanced incomplete block designs, to develop an iterated least squares (ITLS) computing algorithm for calculating maximum likelihood estimates, and to compare these procedures by use of simulated experiments. Based on the simulated experiments, the estimated mean square errors of the ITLS estimates are generally less than*those for previously proposed analysis of variance and symmetric sums estimators.     

The 'improved' brown and forsythe test for mean equality: some things can't be fixed

Mehrotra (1997) presented an ‘;improved’ Brown and Forsythe (1974) statistic which is designed to provide a valid test of mean equality in independent groups designs when variances are heterogeneous. In particular, the usual Brown and Fosythe procedure was modified by using a Satterthwaite approximation for numerator degrees of freedom instead of the usual value of number of groups minus one. Mehrotra then, through Monte Carlo methods, demonstrated that the ‘improved’ method resulted in a robust test of significance in cases where the usual Brown and Forsythe method did not. Accordingly, this ‘improved’ procedure was recommended. We show that under conditions likely to be encountered in applied settings, that is, conditions involving heterogeneous variances as well as nonnormal data, the ‘improved’ Brown and Forsythe procedure results in depressed or inflated rates of Type I error in unbalanced designs. Previous findings indicate, however, that one can obtain a robust test by adopting a heteroscedastic statistic with the robust estimators, rather than the usual least squares estimators, and further improvement can be expected when critical significance values are obtained through bootstrapping methods.     

A monte carlo of plotting positions

A Monte Carlo study was made of the effects of using simple linear regression, on the appropriate probability paper, to estimate parameters, quantiles and cumulative probability for several distributions. These distributions were the Normal, Weibull (shape parameters 1, 2, and 4) and the Type I largest extreme-value distributions. The specific objective was to observe differences arising from choice of plotting positions. Plotting positions used were i/(n+l), (i?3)/(n+.04), (i?.5)/n, either (i?.375)/(n+.25) or (i?.4)/(n+.2), and either F[E(Y_i)] or F[E(£n Y)]. For each combination of 4 sample sizes (n=10(10)(40)), distribution, and plotting position, regression lines were found for each of N =9999 samples. Each regression line was used to estimate: (1) quantiles of 9 specific probabilities, (2) probabilities of 9 specific quantiles, and (3) return periods corresponding to 9 specific quantiles. Compa[rgrave]ison of the mean, variances, mean square error and medians of these estimates and of the regression coefficients confirm some results of Harter [Commun. Statist. A13(13), 1984] and provide further insight.     

Confidence regions based on inefficient estimators

We consider an unbiased estimator of a function of mean value parameters, which is not efficient. This inefficient estimator is correlated with a residual vector. Thus, if a unit dispersion is unknown, it is impossible to determine the correct confidence region for a function of mean value parameters via a standard estimator of an unknown dispersion with the exception of the case when the ordinary least squares (OLS) estimator is considered in a model with a special covariance structure such that the OLS and the generalized least squares (GLS) estimator are the same, that is the OLS estimator is efficient. Two different estimators of a unit dispersion independent of an inefficient estimator are derived in a singular linear statistical model. Their quality was verified by simulations for several types of experimental designs. Two new estimators of the unit dispersion were compared with the standard estimators based on the GLS and the OLS estimators of the function of the mean value parameters. The OLS estimator was considered in the incorrect model with a different covariance matrix such that the originally inefficient estimator became efficient. The numerical examples led to a slightly surprising result which seems to be due to data behaviour. An example from geodetic practice is presented in the paper.     

Computer experiment designs for accurate prediction

Computer experiments using deterministic simulators are sometimes used to replace or supplement physical system experiments. This paper compares designs for an initial computer simulator experiment based on empirical prediction accuracy; it recommends designs for producing accurate predictions. The basis for the majority of the designs compared is the integrated mean squared prediction error (IMSPE) that is computed assuming a Gaussian process model with a Gaussian correlation function. Designs that minimize the IMSPE with respect to a fixed set of correlation parameters as well as designs that minimize a weighted IMSPE over the correlation parameters are studied. These IMSPE-based designs are compared with three widely-used space-filling designs. The designs are used to predict test surfaces representing a range of stationary and non-stationary functions. For the test conditions examined in this paper, the designs constructed under IMSPE-based criteria are shown to outperform space-filling Latin hypercube designs and maximum projection designs when predicting smooth functions of stationary appearance, while space-filling and maximum projection designs are superior for test functions that exhibit strong non-stationarity.     

Robust extrapolation designs and weights for biased regression models with heteroscedastic errors

We consider the construction of designs for the extrapolation of regression responses, allowing both for possible heteroscedasticity in the errors and for imprecision in the specification of the response function. We find minimax designs and correspondingly optimal estimation weights in the context of the following problems: (1) for ordinary least squares estimation, determine a design to minimize the maximum value of the integrated mean squared prediction error (IMSPE), with the maximum being evaluated over both types of departure; (2) for weighted least squares estimation, determine both weights and a design to minimize the maximum IMSPE; (3) choose weights and design points to minimize the maximum IMSPE, subject to a side condition of unbiasedness. Solutions to (1) and (2) are given for multiple linear regression with no interactions, a spherical design space and an annular extrapolation space. For (3) the solution is given in complete generality; as one example we consider polynomial regression. Applications to a dose-response problem for bioassays are discussed. Numerical comparisons, including a simulation study, indicate that, as well as being easily implemented, the designs and weights for (3) perform as well as those for (1) and (2) and outperform some common competitors for moderate but undetectable amounts of model bias.     

Graphical technique for comparing designs for random models

Methods for comparing designs for a random (or mixed) linear model have focused primarily on criteria based on single-valued functions. In general, these functions are difficult to use, because of their complex forms, in addition to their dependence on the model's unknown variance components. In this paper, a graphical approach is presented for comparing designs for random models. The one-way model is used for illustration. The proposed approach is based on using quantiles of an estimator of a function of the variance components. The dependence of these quantiles on the true values of the variance components is depicted by plotting the so-called quantile dispersion graphs (QDGs), which provide a comprehensive picture of the quality of estimation obtained with a given design. The QDGs can therefore be used to compare several candidate designs. Two methods of estimation of variance components are considered, namely analysis of variance and maximum-likelihood estimation.     

Nonparametric rank-based test procedures for non-additive models in the two-way layout I. no replications

One of the major unresolved problems in the area of nonparametric statistics is the need for satisfactory rank-based test procedures for non-additive models in the two-way layout, especially when there is only one observation on each combination of the levels of the experimental factors. In this paper we consider an arbitrary non-additive model for the two-way layout with n levels of each factor. We utilize both alignment and ranking of the data together with basic properties of Latin squares to develop rank tests for interaction (non-additivity). Our technique involves first aligning within one of the main effects, ranking within the other main effects (columns and rows) and then adding the resulting ranks within “interaction bands” corresponding to orthogonal partitions of the interaction for the model, as denoted by the letters of an n × n Latin square. A Friedman-type statistic is then computed on the resulting sums. This is repeated for each of (n?1) mutually orthogonal Latin squares (thus accounting for all the interaction degrees of freedom). The resulting (n?1) Friedman-type statistics are finally combined to obtain an overall test statistic. The necessary null distribution tables for applying the proposed test for non-additivity are presented and we discuss the results of a Monte Carlo simulation study of the relative powers of this new procedure and other (parametric and nonparametric) procedures designed to detect interaction in a two-way layout with one observation per cell.