Robust designs for one-point extrapolation

We consider the construction of designs for the extrapolation of a regression response to one point outside of the design space. The response function is an only approximately known function of a specified linear function. As well, we allow for variance heterogeneity. We find minimax designs and corresponding optimal regression weights in the context of the following problems: (P1) for nonlinear least squares estimation with homoscedasticity, determine a design to minimize the maximum value of the mean squared extrapolation error (MSEE), with the maximum being evaluated over the possible departures from the response function; (P2) for nonlinear least squares estimation with heteroscedasticity, determine a design to minimize the maximum value of MSEE, with the maximum being evaluated over both types of departures; (P3) for nonlinear weighted least squares estimation, determine both weights and a design to minimize the maximum MSEE; (P4) choose weights and design points to minimize the maximum MSEE, subject to a side condition of unbiasedness. Solutions to (P1)–(P4) are given in complete generality. Numerical comparisons indicate that our designs and weights perform well in combining robustness and efficiency. Applications to accelerated life testing are highlighted.     

Robust extrapolation designs and weights for biased regression models with heteroscedastic errors

We consider the construction of designs for the extrapolation of regression responses, allowing both for possible heteroscedasticity in the errors and for imprecision in the specification of the response function. We find minimax designs and correspondingly optimal estimation weights in the context of the following problems: (1) for ordinary least squares estimation, determine a design to minimize the maximum value of the integrated mean squared prediction error (IMSPE), with the maximum being evaluated over both types of departure; (2) for weighted least squares estimation, determine both weights and a design to minimize the maximum IMSPE; (3) choose weights and design points to minimize the maximum IMSPE, subject to a side condition of unbiasedness. Solutions to (1) and (2) are given for multiple linear regression with no interactions, a spherical design space and an annular extrapolation space. For (3) the solution is given in complete generality; as one example we consider polynomial regression. Applications to a dose-response problem for bioassays are discussed. Numerical comparisons, including a simulation study, indicate that, as well as being easily implemented, the designs and weights for (3) perform as well as those for (1) and (2) and outperform some common competitors for moderate but undetectable amounts of model bias.     

Designs for weighted least squares regression, with estimated weights

We study designs, optimal up to and including terms that are O(n ^?1), for weighted least squares regression, when the weights are intended to be inversely proportional to the variances but are estimated with random error. We take a finite, but arbitrarily large, design space from which the support points are to be chosen, and obtain the optimal proportions of observations to be assigned to each point. Specific examples of D- and I-optimal design for polynomial responses are studied. In some cases the same designs that are optimal under homoscedasticity remain so for a range of variance functions; in others there tend to be more support points than are required in the homoscedastic case. We also exhibit minimax designs, that minimize the maximum, over finite classes of variance functions, value of the loss. These also tend to have more support points, often resulting from the breaking down of replicates into clusters.     

Optimal designs for estimating the slope of a regression

In the common linear model with quantitative predictors we consider the problem of designing experiments for estimating the slope of the expected response in a regression. We discuss locally optimal designs, where the experimenter is only interested in the slope at a particular point, and standardized minimax optimal designs, which could be used if precise estimation of the slope over a given region is required. General results on the number of support points of locally optimal designs are derived if the regression functions form a Chebyshev system. For polynomial regression and Fourier regression models of arbitrary degree the optimal designs for estimating the slope of the regression are determined explicitly for many cases of practical interest.     

E-optimal designs for polynomial regression without intercept

We give all E-optimal designs for the mean parameter vector in polynomial regression of degree d without intercept in one real variable. The derivation is based on interplays between E-optimal design problems in the present and in certain heteroscedastic polynomial setups with intercept. Thereby the optimal supports are found to be related to the alternation points of the Chebyshev polynomials of the first kind, but the structure of optimal designs essentially depends on the regression degree being odd or even. In the former case the E-optimal designs are precisely the (infinitely many) scalar optimal designs, where the scalar parameter system refers to the Chebyshev coefficients, while for even d there is exactly one optimal design. In both cases explicit formulae for the corresponding optimal weights are obtained. Remarks on extending the results to E-optimality for subparameters of the mean vector (in heteroscdastic setups) are also given.     

A geometric characterization of optimal designs for regression models with correlated observations

We consider the problem of optimal design of experiments for random effects models, especially population models, where a small number of correlated observations can be taken on each individual, while the observations corresponding to different individuals are assumed to be uncorrelated. We focus on c-optimal design problems and show that the classical equivalence theorem and the famous geometric characterization of Elfving (1952) from the case of uncorrelated data can be adapted to the problem of selecting optimal sets of observations for the n individual patients. The theory is demonstrated by finding optimal designs for a linear model with correlated observations and a nonlinear random effects population model, which is commonly used in pharmacokinetics.     

Robust Linear Calibration

We regard the simple linear calibration problem where only the response y of the regression line y = β₀ + β₁ t is observed with errors. The experimental conditions t are observed without error. For the errors of the observations y we assume that there may be some gross errors providing outlying observations. This situation can be modeled by a conditionally contaminated regression model. In this model the classical calibration estimator based on the least squares estimator has an unbounded asymptotic bias. Therefore we introduce calibration estimators based on robust one-step-M-estimators which have a bounded asymptotic bias. For this class of estimators we discuss two problems: The optimal estimators and their corresponding optimal designs. We derive the locally optimal solutions and show that the maximin efficient designs for non-robust estimation and robust estimation coincide.     

Minimax designs for approximately linear models with AR(1) errors

We obtain designs for linear regression models under two main departures from the classical assumptions: (1) the response is taken to be only approximately linear, and (2) the errors are not assumed to be independent, but to instead follow a first-order autoregressive process. These designs have the property that they minimize (a modification of) the maximum integrated mean squared error of the estimated response, with the maximum taken over a class of departures from strict linearity and over all autoregression parameters ρ,|ρ,| < 1, of fixed sign. Specific methods of implementation are discussed. We find that an asymptotically optimal procedure for AR(1) models consists of choosing points from that design measure which is optimal for uncorrelated errors, and then implementing them in an appropriate order.     

Optimal designs for statistical analysis with Zernike polynomials

The Zernike polynomials arise in several applications such as optical metrology or image analysis on a circular domain. In the present paper, we determine optimal designs for regression models which are represented by expansions in terms of Zernike polynomials. We consider two estimation methods for the coefficients in these models and determine the corresponding optimal designs. The first one is the classical least squares method and Φ _p -optimal designs in the sense of Kiefer [Kiefer, J., 1974, General equivalence theory for optimum designs (approximate theory). Annals of Statistics, 2 849–879.] are derived, which minimize an appropriate functional of the covariance matrix of the least squares estimator. It is demonstrated that optimal designs with respect to Kiefer's Φ _p -criteria (p>?∞) are essentially unique and concentrate observations on certain circles in the experimental domain. E-optimal designs have the same structure but it is shown in several examples that these optimal designs are not necessarily uniquely determined. The second method is based on the direct estimation of the Fourier coefficients in the expansion of the expected response in terms of Zernike polynomials and optimal designs minimizing the trace of the covariance matrix of the corresponding estimator are determined. The designs are also compared with the uniform designs on a grid, which is commonly used in this context.     

Optimal Sequential Design in a Controlled Non-parametric Regression

Abstract. In a non‐parametric regression, the heteroscedasticity (dependence of the variance of the regression error on the predictor) can be a serious complication in estimation or visualization of an underlying regression function. If a controlled sampling is permitted, then the statistician can choose the design of predictors which attenuates the effect of heteroscedasticity. It is proposed to use a design which minimizes the mean integrated squared error of the regression function estimation. Then the corresponding optimal design density is proportional to the standard deviation of the regression error (the so‐called scale function). Because in general the statistician does not know an underlying scale function, the natural question is as follows: is it possible to suggest a sequential design which performs as well as an oracle that knows the underlying scale function? The answer is ‘yes’, and a corresponding sequential procedure is developed. It is proved, for the first time in the literature, that a data‐driven sequential design, together with an adaptive regression estimator, can mimic the oracle and be sharp minimax. Further, it is shown that the suggested method is feasible for small samples.     

E-optimal designs for regression models with quantitative factors— a reasonable choice?

For regression models with quantitative factors it is illustrated that the E-optimal design can be extremely inefficient in the sense that it degenerates to a design which takes all observations at only one point. This phenomenon is caused by the different size of the elements in the covariance matrix of the least-squares estimator for the unknown parameters. For these reasons we propose to replace the E-criterion by a corresponding standardized version. The advantage of this approach is demonstrated for the polynomial regression on a nonnegative interval, where the classical and standardized E-optimal designs can be found explicitly. The described phenomena are not restricted to the E-criterion but appear for nearly all optimality criteria proposed in the literature. Therefore standardization is recommended for optimal experimental design in regression models with quantitative factors. The optimal designs with respect to the new standardized criteria satisfy a similar invariance property as the famous D-optimal designs, which allows an easy calculation of standardized optimal designs on many linearly transformed design spaces.     

Optimal designs for exponential regression

This paper is concerned with the optimal design problem for the particular case of non-linear parametrisation:the parameters to be estimated are included in exponents.Some properties of locally optimal designs as functions of estimated parameters are investigated and a table of such designs in given.We consider also designs to be optimal in the sense of minimax approach.     

Applications and Implementations of Continuous Robust Designs

In the literature concerning the construction of robust optimal designs, many resulting designs turn out to have densities. In practice, an exact design should tell the experimenter what the support points are and how many subjects should be allocated to each of these points. In particular, we consider a practical situation in which the number of support points allowed is constrained. We discuss an intuitive approach, which motivates a new implementation scheme that minimizes the loss function based on the Kolmogorov and Smirnov distance between an exact design and the optimal design having a density. We present three examples to illustrate the application and implementation of a robust design constructed: one for a nonlinear dose-response experiment and the other two for general linear regression. Additionally, we perform some simulation studies to compare the efficiencies of the exact designs obtained by our optimal implementation with those by other commonly used implementation methods.     

On Exact K-optimal Designs Minimizing the Condition Number

A new design criterion based on the condition number of an information matrix is proposed to construct optimal designs for linear models, and the resulting designs are called K-optimal designs. The relationship between exact and asymptotic K-optimal designs is derived. Since it is usually hard to find exact optimal designs analytically, we apply a simulated annealing algorithm to compute K-optimal design points on continuous design spaces. Specific issues are addressed to make the algorithm effective. Through exact designs, we can examine some properties of the K-optimal designs such as symmetry and the number of support points. Examples and results are given for polynomial regression models and linear models for fractional factorial experiments. In addition, K-optimal designs are compared with A-optimal and D-optimal designs for polynomial regression models, showing that K-optimal designs are quite similar to A-optimal designs.     

Robustness properties of minimally‐supported Bayesian D‐optimal designs for heteroscedastic models

Bayesian D‐optimal designs supported on a fixed number of points were found by Dette & Wong (1998) for estimating parameters in a polynomial model when the error variance depends exponentially on the explanatory variable. The present authors provide optimal designs under a broader class of error variance structures and investigate the robustness properties of these designs to model and prior distribution assumptions. A comparison of the performance of the optimal designs relative to the popular uniform designs is also given. The authors' results suggest that Bayesian D‐optimal designs suported on a fixed number of points are more likely to be globaly optimal among all designs if the prior distribution is symmetric and is concentrated around its mean.     

Optimal designs in growth curve models: Part I Correlated model for linear growth: Optimal designs for slope parameter estimation and growth prediction

In the present paper we discuss the situation for a linear growth with correlated structure of the errors and indicate the nature of optimal designs for estimation and prediction problems. We study the intraclass structure of the error distribution. As regards estimation of the slope parameter, we look for robust optimal designs. Here robustness means that optimality should hold for a large variety of correlation parameters. The robust optimal designs for the prediction problem center around a performance measure of the predictors for all design points simultaneously. We have also studied the autocorrelated error structure and found similar results which are reported very briefly.     

Using the Canonical Design Space to Obtain c-optimal Designs for the Quadratic Logistic Model

c-optimal designs for estimating the model parameters of the quadratic logistic regression model are considered. The designs are constructed via the canonical design space. It is shown that the number of design points varies between 1 and 4 depending on the parameter being estimated. Furthermore, formulae for finding the design points along with the corresponding design weights are derived.     

Optimum Mixture Design for the Detection of Synergistic Effects

In mixture experiments, optimal designs for the estimation of parameters, both linear and non-linear, have been discussed by several authors. Optimal designs for the estimation of a subset of parameters have also been investigated. However, designs for testing the effects of certain factors and interactions have been studied only in the context of response surface models. In this article, we attempt to find the optimum design for testing the presence of synergistic effects in a mixture model. The classical F-test has been considered and the optimum design has been obtained so as to maximize the power of the test. It is observed that the barycenters are necessarily the support points of the trace-optimal design.     

The robustness of response surface designs with errors in factor levels

ABSRTACT

Since errors in factor levels affect the traditional statistical properties of response surface designs, an important question to consider is robustness of design to errors. However, when the actual design could be observed in the experimental settings, its optimality and prediction are of interest. Various numerical and graphical methods are useful tools for understanding the behavior of the designs. The D- and G-efficiencies and the fraction of design space plot are adapted to assess second-order response surface designs where the predictor variables are disturbed by a random error. Our study shows that the D-efficiencies of the competing designs are considerably low for big variance of the error, while the G-efficiencies are quite good. Fraction of design space plots display the distribution of the scaled prediction variance through the design space with and without errors in factor levels. The robustness of experimental designs against factor errors is explored through comparative study. The construction and use of the D- and G-efficiencies and the fraction of design space plots are demonstrated with several examples of different designs with errors.     

Model robust extrapolation designs

We seek designs which are optimal in some sense for extrapolation when the true regression function is in a certain class of regression functions. More precisely, the class is defined to be the collection of regression functions such that its (h + 1)-th derivative is bounded. The class can be viewed as representing possible departures from an ‘ideal’ model and thus describes a model robust setting. The estimates are restricted to be linear and the designs are restricted to be with minimal number of points. The design and estimate sought is minimax for mean square error. The optimal designs for cases X = [0, ∞] and X = [-1, 1], where X is the place where observations can be taken, are discussed.