On minimax designs when there are two candidate models

This work is motivated by the need to find experimental designs which are robust under different model assumptions. We measure robustness by calculating a measure of design efficiency with respect to a design optimality criterion and say that a design is robust if it is reasonably efficient under different model scenarios. We discuss two design criteria and an algorithm which can be used to obtain robust designs. The first criterion employs a Bayesian-type approach by putting a prior or weight on each candidate model and possibly priors on the corresponding model parameters. We define the first criterion as the expected value of the design efficiency over the priors. The second design criterion we study is the minimax design which minimizes the worst value of a design criterion over all candidate models. We establish conditions when these two criteria are equivalent when there are two candidate models. We apply our findings to the area of accelerated life testing and perform sensitivity analysis of designs with respect to priors and misspecification of planning values.     

Optimal U-type design for Bayesian nonparametric multiresponse prediction

This paper presents an extension of the work of Yue and Chatterjee (2010) about U-type designs for Bayesian nonparametric response prediction. We consider nonparametric Bayesian regression model with p responses. We use U-type designs with n runs, m factors and q levels for the nonparametric multiresponse prediction based on the asymptotic Bayesian criterion. A lower bound for the proposed criterion is established, and some optimal and nearly optimal designs for the illustrative models are given.     

An investigation of a quadratic design criterion for estimation of the hemodynamic response function

Optimal experimental design for estimation of the hemodynamic response function (HRF) is investigated using a nonlinear model with a quadratic mean squared error design criterion. This criterion is used, along with a genetic algorithm, to select locally optimal designs that are shown to be, in most cases, more efficient than designs selected with the more commonly used linear expansion criterion. These designs are also shown to result in lower overall asymptotic estimator variance and bias. The investigation focuses on a single stimulus type, but the criterion can also be used with multiple stimulus types.     

Adaptive paired comparison design

An important aspect of paired comparison experiments is the decision of how to form pairs in advance of collecting data. A weakness of typical paired comparison experimental designs is the difficulty in incorporating prior information, which can be particularly relevant for the design of tournament schedules for players of games and sports. Pairing methods that make use of prior information are often ad hoc algorithms with little or no formal basis. The problem of pairing objects can be formalized as a Bayesian optimal design. Assuming a linear paired comparison model for outcomes, we develop a pairing method that maximizes the expected gain in Kullback–Leibler information from the prior to the posterior distribution. The optimal pairing is determined using a combinatorial optimization method commonly used in graph-theoretic contexts. We discuss the properties of our optimal pairing criterion, and demonstrate our method as an adaptive procedure for pairing objects multiple times. We compare the performance of our method on simulated data against random pairings, and against a system that is currently in use in tournament chess.     

Non-linear design problem in a chemical kinetic model with non-constant error variance

For a locally optimum non-linear design problem for a chemical kinetic model, we investigate the influence of the dispersion structure of the random observation errors on the design and its efficiency. We find that there are two kinds of design determined by the model parameters and the error variance function: “interior” designs, and “boundary” designs that depend also on the design range. We give an exact criterion for determining which kind of design will arise and we illustrate the qualitative difference between the two kinds of design in terms of the design locus and the equivalence theorem. We tabulate quantitative details of the designs for a range of parameter values.     

Bayesian experimental design for a toxicokinetic-toxicodynamic model

The aim of this study is to apply the Bayesian method of identifying optimal experimental designs to a toxicokinetic-toxicodynamic model that describes the response of aquatic organisms to time dependent concentrations of toxicants. As for experimental designs, we restrict ourselves to pulses and constant concentrations. A design of an experiment is called optimal within this set of designs if it maximizes the expected gain of knowledge about the parameters. Focus is on parameters that are associated with the auxiliary damage variable of the model that can only be inferred indirectly from survival time series data. Gain of knowledge through an experiment is quantified both with the ratio of posterior to prior variances of individual parameters and with the entropy of the posterior distribution relative to the prior on the whole parameter space. The numerical methods developed to calculate expected gain of knowledge are expected to be useful beyond this case study, in particular for multinomially distributed data such as survival time series data.     

D-optimal minimax design criterion for two-level fractional factorial designs

A D-optimal minimax design criterion is proposed to construct two-level fractional factorial designs, which can be used to estimate a linear model with main effects and some specified interactions. D-optimal minimax designs are robust against model misspecification and have small biases if the linear model contains more interaction terms. When the D-optimal minimax criterion is compared with the D-optimal design criterion, we find that the D-optimal design criterion is quite robust against model misspecification. Lower and upper bounds derived for the loss functions of optimal designs can be used to estimate the efficiencies of any design and evaluate the effectiveness of a search algorithm. Four algorithms to search for optimal designs for any run size are discussed and compared through several examples. An annealing algorithm and a sequential algorithm are particularly effective to search for optimal designs.     

Optimal predictive design augmentation for spatial generalised linear mixed models

A typical model for geostatistical data when the observations are counts is the spatial generalised linear mixed model. We present a criterion for optimal sampling design under this framework which aims to minimise the error in the prediction of the underlying spatial random effects. The proposed criterion is derived by performing an asymptotic expansion to the conditional prediction variance. We argue that the mean of the spatial process needs to be taken into account in the construction of the predictive design, which we demonstrate through a simulation study where we compare the proposed criterion against the widely used space-filling design. Furthermore, our results are applied to the Norway precipitation data and the rhizoctonia disease data.     

Bayesian optimal blocking of factorial designs

The presence of block effects makes the optimal selection of fractional factorial designs a difficult task. The existing frequentist methods try to combine treatment and block wordlength patterns and apply minimum aberration criterion to find the optimal design. However, ambiguities exist in combining the two wordlength patterns and therefore, the optimality of such designs can be challenged. Here we propose a Bayesian approach to overcome this problem. The main technique is to postulate a model and a prior distribution to satisfy the common assumptions in blocking and then, to develop an optimal design criterion for the efficient estimation of treatment effects. We apply our method to develop regular, nonregular, and mixed-level blocked designs. Several examples are presented to illustrate the advantages of the proposed method.     

Design and analysis of variable fidelity experimentation applied to engine valve heat treatment process design

Summary.  When experimentation on a real system is expensive, data are often collected by using cheaper, lower fidelity surrogate systems. The paper concerns response surface methods in the context of variable fidelity experimentation. We propose the use of generalized least squares to generate the predictions. We also present perhaps the first optimal designs for variable fidelity experimentation, using an extension of the expected integrated mean-squared error criterion. Numerical tests are used to compare the performance of the method with alternatives and to investigate the robustness to incorporated assumptions. The method is applied to automotive engine valve heat treatment process design in which real world data were mixed with data from two types of computer simulation.     

The robustness of response surface designs with errors in factor levels

ABSRTACT

Since errors in factor levels affect the traditional statistical properties of response surface designs, an important question to consider is robustness of design to errors. However, when the actual design could be observed in the experimental settings, its optimality and prediction are of interest. Various numerical and graphical methods are useful tools for understanding the behavior of the designs. The D- and G-efficiencies and the fraction of design space plot are adapted to assess second-order response surface designs where the predictor variables are disturbed by a random error. Our study shows that the D-efficiencies of the competing designs are considerably low for big variance of the error, while the G-efficiencies are quite good. Fraction of design space plots display the distribution of the scaled prediction variance through the design space with and without errors in factor levels. The robustness of experimental designs against factor errors is explored through comparative study. The construction and use of the D- and G-efficiencies and the fraction of design space plots are demonstrated with several examples of different designs with errors.     

Maximum expected entropy transformed Latin hypercube designs

Existing projection designs (e.g. maximum projection designs) attempt to achieve good space-filling properties in all projections. However, when using a Gaussian process (GP), model-based design criteria such as the entropy criterion is more appropriate. We employ the entropy criterion averaged over a set of projections, called expected entropy criterion (EEC), to generate projection designs. We show that maximum EEC designs are invariant to monotonic transformations of the response, i.e. they are optimal for a wide class of stochastic process models. We also demonstrate that transformation of each column of a Latin hypercube design (LHD) based on a monotonic function can substantially improve the EEC. Two types of input transformations are considered: a quantile function of a symmetric Beta distribution chosen to optimize the EEC, and a nonparametric transformation corresponding to the quantile function of a symmetric density chosen to optimize the EEC. Numerical studies show that the proposed transformations of the LHD are efficient and effective for building robust maximum EEC designs. These designs give projections with markedly higher entropies and lower maximum prediction variances (MPV''s) at the cost of small increases in average prediction variances (APV''s) compared to state-of-the-art space-filling designs over wide ranges of covariance parameter values.     

A semi-infinite programming based algorithm for finding minimax optimal designs for nonlinear models

Minimax optimal experimental designs are notoriously difficult to study largely because the optimality criterion is not differentiable and there is no effective algorithm for generating them. We apply semi-infinite programming (SIP) to solve minimax design problems for nonlinear models in a systematic way using a discretization based strategy and solvers from the General Algebraic Modeling System (GAMS). Using popular models from the biological sciences, we show our approach produces minimax optimal designs that coincide with the few theoretical and numerical optimal designs in the literature. We also show our method can be readily modified to find standardized maximin optimal designs and minimax optimal designs for more complicated problems, such as when the ranges of plausible values for the model parameters are dependent and we want to find a design to minimize the maximal inefficiency of estimates for the model parameters.     

Optimal experimental design criterion for discriminating semiparametric models

This paper studies the optimal experimental design problem to discriminate two regression models. Recently, López-Fidalgo et al. [2007. An optimal experimental design criterion for discriminating between non-normal models. J. Roy. Statist. Soc. B 69, 231–242] extended the conventional T-optimality criterion by Atkinson and Fedorov [1975a. The designs of experiments for discriminating between two rival models. Biometrika 62, 57–70; 1975b. Optimal design: experiments for discriminating between several models. Biometrika 62, 289–303] to deal with non-normal parametric regression models, and proposed a new optimal experimental design criterion based on the Kullback–Leibler information divergence. In this paper, we extend their parametric optimality criterion to a semiparametric setup, where we only need to specify some moment conditions for the null or alternative regression model. Our criteria, called the semiparametric Kullback–Leibler optimality criteria, can be implemented by applying a convex duality result of partially finite convex programming. The proposed method is illustrated by a simple numerical example.     

A note on the multivariate miser for balancing experiments

The MISER criterion for pre-experiment balancing of experiments that will be analyzed by the analysis of covariance was proposed by Press (1987). The idea was to minimize, over all designs, the inflation produced by design imbalance (inequality of cell covariate vectors) , of the average standard errors of contrasts. In this note we consider the balancing problem in the case of multiple response experiments, and we show that under certain reasonable assumptions, the multivariate result is analogous to that in the univariate case.     

D-optimal minimax regression designs on discrete design space

One classical design criterion is to minimize the determinant of the covariance matrix of the regression estimates, and the designs are called D-optimal designs. To reflect the nature that the proposed models are only approximately true, we propose a robust design criterion to study response surface designs. Both the variance and bias are considered in the criterion. In particular, D-optimal minimax designs are investigated and constructed. Examples are given to compare D-optimal minimax designs with classical D-optimal designs.     

Rank-order choice-based conjoint experiments: Efficiency and design

In a rank-order choice-based conjoint experiment, the respondent is asked to rank a number of alternatives of a number of choice sets. In this paper, we study the efficiency of those experiments and propose a D-optimality criterion for rank-order experiments to find designs yielding the most precise parameter estimators. For that purpose, an expression of the Fisher information matrix for the rank-ordered conditional logit model is derived which clearly shows how much additional information is provided by each extra ranking step. A simulation study shows that, besides the Bayesian D-optimal ranking design, the Bayesian D-optimal choice design is also an appropriate design for this type of experiments. Finally, it is shown that considerable improvements in estimation and prediction accuracy are obtained by including extra ranking steps in an experiment.     

A relaxation procedure for calculating (G–)minimax optimal designs

Summary: In nonlinear statistical models, standard optimality functions for experimental designs depend on the unknown parameters of the model. An appealing and robust concept for choosing a design is the minimax criterion. However, so far, minimax optimal designs have been calculated efficiently under various restrictive conditions only. We extend an iterative relaxation scheme originally proposed by Shimizu and Aiyoshi (1980) and prove its convergence under very general assumptions which cover a variety of situations considered in experimental design. Application to different specific design criteria is discussed and issues of practical implementation are addressed. First numerical results suggest that the method may be very efficient with respect to the number of iterations required.*Supported by a grant from the Deutsche Forschungsgemeinschaft. We are grateful to a referee for his constructive suggestions.     

Response surface design evaluation and comparison

Designing an experiment to fit a response surface model typically involves selecting among several candidate designs. There are often many competing criteria that could be considered in selecting the design, and practitioners are typically forced to make trade-offs between these objectives when choosing the final design. Traditional alphabetic optimality criteria are often used in evaluating and comparing competing designs. These optimality criteria are single-number summaries for quality properties of the design such as the precision with which the model parameters are estimated or the uncertainty associated with prediction. Other important considerations include the robustness of the design to model misspecification and potential problems arising from spurious or missing data. Several qualitative and quantitative properties of good response surface designs are discussed, and some of their important trade-offs are considered. Graphical methods for evaluating design performance for several important response surface problems are discussed and we show how these techniques can be used to compare competing designs. These graphical methods are generally superior to the simplistic summaries of alphabetic optimality criteria. Several special cases are considered, including robust parameter designs, split-plot designs, mixture experiment designs, and designs for generalized linear models.     

Optimal design of experimental epidemics

We consider the optimal design of controlled experimental epidemics or transmission experiments, whose purpose is to inform the practitioner about disease transmission and recovery rates. Our methodology employs Gaussian diffusion approximations, applicable to epidemics that can be modeled as density-dependent Markov processes and involving relatively large numbers of organisms. We focus on finding (i) the optimal times at which to collect data about the state of the system for a small number of discrete observations, (ii) the optimal numbers of susceptible and infective individuals to begin an experiment with, and (iii) the optimal number of replicate epidemics to use. We adopt the popular D-optimality criterion as providing an appropriate objective function for designing our experiments, since this leads to estimates with maximum precision, subject to valid assumptions about parameter values. We demonstrate the broad applicability of our methodology using a diverse array of compartmental epidemic models: a time-homogeneous SIS epidemic, a time-inhomogeneous SI epidemic with exponentially decreasing transmission rates and a partially observed SIR epidemic where the infectious period for an individual has a gamma distribution.