Saturated main-effect designs for factorial experiments

The results of a computer search for saturated designs for 2ⁿ factorial experiments with n runs is reported, (where n = 2 mod 4). A complete search of the design space is avoided by focussing on designs constructed from cyclic generators. A method of searching quickly for the best generators is given. The resulting designs are as good as, and sometimes better than, designs obtained via search algorithms reported in the literature. The addition of a further factor having three levels is also considered. Here, too, a complete search is avoided by restricting attention to the most efficient part of the design space under p-efficiency.     

Simplex based space filling designs

Space filling designs are important for deterministic computer experiments. Even a single experiment can be very time consuming and can have many input parameters. Furthermore the underlying function generating the output is often nonlinear. Thus, the computer experiment has to be designed carefully. There exist many design criteria, which can be numerically optimized. Here, a method is developed, which does not need algorithmic optimization. A mesh of nearly regular simplices is constructed and the vertices of the simplices are used as potential design points. The extraction of a design from these meshes is very fast and easy to implement once the underlying mesh has been constructed. The extracted designs are highly competitive regarding the maximin design criterion and it is easy to extract designs for nonstandard design spaces.     

Rotation designs for experiments in high-bias situations

Many experiments in the physical and engineering sciences study complex processes in which bias due to model inadequacy dominates random error. A noteworthy example of this situation is the use of computer experiments, in which scientists simulate the phenomenon being studied by a computer code. Computer experiments are deterministic: replicate observations from running the code with the same inputs will be identical. Such high-bias settings demand different techniques for design and prediction. This paper will focus on the experimental design problem introducing a new class of designs called rotation designs. Rotation designs are found by taking an orthogonal starting design D and rotating it to obtain a new design matrix D_R=DR, where R is any orthonormal matrix. The new design is still orthogonal for a first-order model. In this paper, we study some of the properties of rotation designs and we present a method to generate rotation designs that have some appealing symmetry properties.     

An efficient algorithm for constructing optimal design of computer experiments

The long computational time required in constructing optimal designs for computer experiments has limited their uses in practice. In this paper, a new algorithm for constructing optimal experimental designs is developed. There are two major developments involved in this work. One is on developing an efficient global optimal search algorithm, named as enhanced stochastic evolutionary (ESE) algorithm. The other is on developing efficient methods for evaluating optimality criteria. The proposed algorithm is compared to existing techniques and found to be much more efficient in terms of the computation time, the number of exchanges needed for generating new designs, and the achieved optimality criteria. The algorithm is also very flexible to construct various classes of optimal designs to retain certain desired structural properties.     

Designing Experiments with Respect to 'Standardized' Optimality Criteria

We introduce a new class of `standardized' optimality criteria which depend on `standardized' covariances of the least squares estimators and provide an alternative to the commonly used criteria in design theory. Besides a nice statistical interpretation the new criteria satisfy an extremely useful invariance property which allows an easy calculation of optimal designs on many linearly transformed design spaces.     

A Simple Method for Obtaining Minimum Aberration Designs

Minimum aberration designs are preferred in practice, especially when it is desired to carry out a multi-factor experiment using less number of runs. Several authors considered constructions of minimum aberration designs. Some used computer algorithms and some listed good designs from the exhausted search. We propose a simple method to obtain minimum aberration designs for experiments of size less than or equal to thirty-two. Here, we use an ordered sequence of columns from an orthogonal array to design experiments and blocked experiments. When the method is implemented in MS Excel, minimum aberration designs can be easily achieved.     

Multiphase experiments with at least one later laboratory phase. II. Nonorthogonal designs

Principles and laws that apply to nonorthogonal multiphase experiments are developed and illustrated using examples that are nonorthogonal but structure‐balanced, not structure, but first‐order, balanced or unbalanced, thus exposing the differences between the different design types. The design of such experiments using standard designs, a catalogue of designs and computer searches is exemplified. Factor–allocation diagrams are employed to depict the allocations in the examples, and used in producing the anatomies of designs or, when possible, the related skeleton‐analysis‐of‐variance tables, to assess the properties of designs. The formulation of mixed models based on them is also described. Tools used for structure‐balanced experiments are also shown to be applicable to those experiments that are not.     

Covariance Analysis for Split-Plot and Split-Block Designs

Commentaries are informative essays dealing with viewpoints of statistical practice, statistical education, and other topics considered to be of general interest to the broad readership of The American Statistician. Commentaries are similar in spirit to Letters to the Editor, but they involve longer discussions of background, issues, and perspectives. All commentaries will be refereed for their merit and compatibility with these criteria.

Proper methodology for the analysis of covariance for experiments designed in a split-plot or split-block design is not found in the statistical literature. Analyses for these designs are often performed incompletely or even incorrectly. This is especially true when popular statistical computer software packages are used for the analysis of these designs. This article provides several appropriate models, ANOVA tables, and standard errors for comparisons from experiments arranged in a standard split-plot, split–split-plot, or split-block design where a covariate has been measured on the smallest size experimental unit.     

Optimal regular graph designs

A typical problem in optimal design theory is finding an experimental design that is optimal with respect to some criteria in a class of designs. The most popular criteria include the A- and D-criteria. Regular graph designs occur in many optimality results, and if the number of blocks is large enough, an A-optimal (or D-optimal) design is among them (if any exist). To explore the landscape of designs with a large number of blocks, we introduce extensions of regular graph designs. These are constructed by adding the blocks of a balanced incomplete block design repeatedly to the original design. We present the results of an exact computer search for the best regular graph designs and the best extended regular graph designs with up to 20 treatments v, block size \(k \le 10\) and replication r \(\le 10\) and \(r(k-1)-(v-1)\lfloor r(k-1)/(v-1)\rfloor \le 9\).     

Designs for two-colour microarray experiments

Summary.  Designs for two-colour microarray experiments can be viewed as block designs with two treatments per block. Explicit formulae for the A- and D-criteria are given for the case that the number of blocks is equal to the number of treatments. These show that the A- and D-optimality criteria conflict badly if there are 10 or more treatments. A similar analysis shows that designs with one or two extra blocks perform very much better, but again there is a conflict between the two optimality criteria for moderately large numbers of treatments. It is shown that this problem can be avoided by slightly increasing the number of blocks. The two colours that are used in each block effectively turn the block design into a row–column design. There is no need to use a design in which every treatment has each colour equally often: rather, an efficient row–column design should be used. For odd replication, it is recommended that the row–column design should be based on a bipartite graph, and it is proved that the optimal such design corresponds to an optimal block design for half the number of treatments. Efficient row–column designs are given for replications 3–6. It is shown how to adapt them for experiments in which some treatments have replication only 2.     

Computer experiment designs for accurate prediction

Computer experiments using deterministic simulators are sometimes used to replace or supplement physical system experiments. This paper compares designs for an initial computer simulator experiment based on empirical prediction accuracy; it recommends designs for producing accurate predictions. The basis for the majority of the designs compared is the integrated mean squared prediction error (IMSPE) that is computed assuming a Gaussian process model with a Gaussian correlation function. Designs that minimize the IMSPE with respect to a fixed set of correlation parameters as well as designs that minimize a weighted IMSPE over the correlation parameters are studied. These IMSPE-based designs are compared with three widely-used space-filling designs. The designs are used to predict test surfaces representing a range of stationary and non-stationary functions. For the test conditions examined in this paper, the designs constructed under IMSPE-based criteria are shown to outperform space-filling Latin hypercube designs and maximum projection designs when predicting smooth functions of stationary appearance, while space-filling and maximum projection designs are superior for test functions that exhibit strong non-stationarity.     

Algorithmic construction of nearly column-orthogonal designs

Computer models can describe complicated phenomena encountered in science and engineering fields. To use these models for scientific investigation, however, their generally long running time and mostly deterministic nature require a specially designed experiment. The column-orthogonal design (COD) is a popular choice for computer experiments. Because of the restriction on the orthogonality, however, only little CODs can be constructed. In this article, we propose two algorithms for constructing nearly CODs by rotating orthogonal arrays under two different criteria. Further, some obtained nearly CODs are nearly Latin hypercube designs. Some examples are provided to show the advantages of our algorithms. Some rotation matrices obtained via the algorithms are listed.     

PROBABILITY-BASED OPTIMAL DESIGN

Optimal design of experiments has generally concentrated on parameter estimation and, to a much lesser degree, on model discrimination. Often an experimenter is interested in a particular outcome and wishes to maximize in some way the probability of this outcome. We propose a new class of compound criteria and designs that address this issue for generalized linear models. The criteria offer a method of achieving designs that possess the properties of efficient parameter estimation and a high probability of a desired outcome.     

Optimal experimental design criterion for discriminating semiparametric models

This paper studies the optimal experimental design problem to discriminate two regression models. Recently, López-Fidalgo et al. [2007. An optimal experimental design criterion for discriminating between non-normal models. J. Roy. Statist. Soc. B 69, 231–242] extended the conventional T-optimality criterion by Atkinson and Fedorov [1975a. The designs of experiments for discriminating between two rival models. Biometrika 62, 57–70; 1975b. Optimal design: experiments for discriminating between several models. Biometrika 62, 289–303] to deal with non-normal parametric regression models, and proposed a new optimal experimental design criterion based on the Kullback–Leibler information divergence. In this paper, we extend their parametric optimality criterion to a semiparametric setup, where we only need to specify some moment conditions for the null or alternative regression model. Our criteria, called the semiparametric Kullback–Leibler optimality criteria, can be implemented by applying a convex duality result of partially finite convex programming. The proposed method is illustrated by a simple numerical example.     

Maximin design on non hypercube domains and kernel interpolation

In the paradigm of computer experiments, the choice of an experimental design is an important issue. When no information is available about the black-box function to be approximated, an exploratory design has to be used. In this context, two dispersion criteria are usually considered: the minimax and the maximin ones. In the case of a hypercube domain, a standard strategy consists of taking the maximin design within the class of Latin hypercube designs. However, in a non hypercube context, it does not make sense to use the Latin hypercube strategy. Moreover, whatever the design is, the black-box function is typically approximated thanks to kernel interpolation. Here, we first provide a theoretical justification to the maximin criterion with respect to kernel interpolations. Then, we propose simulated annealing algorithms to determine maximin designs in any bounded connected domain. We prove the convergence of the different schemes. Finally, the methodology is applied on a challenging real example where the black-blox function describes the behaviour of an aircraft engine.     

Design of experiments for locally weighted regression

We consider the design of experiments when estimation is to be performed using locally weighted regression methods. We adopt criteria that consider both estimation error (variance) and error resulting from model misspecification (bias). Working with continuous designs, we use the ideas developed in convex design theory to analyze properties of the corresponding optimal designs. Numerical procedures for constructing optimal designs are developed and applied to a variety of design scenarios in one and two dimensions. Among the interesting properties of the constructed designs are the following: (1) Design points tend to be more spread throughout the design space than in the classical case. (2) The optimal designs appear to be less model and criterion dependent than their classical counterparts.(3) While the optimal designs are relatively insensitive to the specification of the design space boundaries, the allocation of supporting points is strongly governed by the points of interest and the selected weight function, if the latter is concentrated in areas significantly smaller than the design region. Some singular and unstable situations occur in the case of saturated designs. The corresponding phenomenon is discussed using a univariate linear regression example.     

Constructing optimal projection designs

ABSTRACT

The early stages of many real-life experiments involve a large number of factors among which only a few factors are active. Unfortunately, the optimal full-dimensional designs of those early stages may have bad low-dimensional projections and the experimenters do not know which factors turn out to be important before conducting the experiment. Therefore, designs with good projections are desirable for factor screening. In this regard, significant questions are arising such as whether the optimal full-dimensional designs have good projections onto low dimensions? How experimenters can measure the goodness of a full-dimensional design by focusing on all of its projections?, and are there linkages between the optimality of a full-dimensional design and the optimality of its projections? Through theoretical justifications, this paper tries to provide answers to these interesting questions by investigating the construction of optimal (average) projection designs for screening either nominal or quantitative factors. The main results show that: based on the aberration and orthogonality criteria the full-dimensional design is optimal if and only if it is optimal projection design; the full-dimensional design is optimal via the aberration and orthogonality if and only if it is uniform projection design; there is no guarantee that a uniform full-dimensional design is optimal projection design via any criterion; the projection design is optimal via the aberration, orthogonality and uniformity criteria if it is optimal via any criterion of them; and the saturated orthogonal designs have the same average projection performance.     

Computer experiments: a review

In this paper we provide a broad introduction to the topic of computer experiments. We begin by briefly presenting a number of applications with different types of output or different goals. We then review modelling strategies, including the popular Gaussian process approach, as well as variations and modifications. Other strategies that are reviewed are based on polynomial regression, non-parametric regression and smoothing spline ANOVA. The issue of multi-level models, which combine simulators of different resolution in the same experiment, is also addressed. Special attention is given to modelling techniques that are suitable for functional data. To conclude the modelling section, we discuss calibration, validation and verification. We then review design strategies including Latin hypercube designs and space-filling designs and their adaptation to computer experiments. We comment on a number of special issues, such as designs for multi-level simulators, nested factors and determination of experiment size.     

Orthogonal Latin hypercube designs from generalized orthogonal designs

Latin hypercube designs is a class of experimental designs that is important when computer simulations are needed to study a physical process. In this paper, we proposed some general criteria for evaluating Latin hypercube designs through their alias matrices. Moreover, a general method is proposed for constructing orthogonal Latin hypercube designs. In particular, links between orthogonal designs (ODs), generalized orthogonal designs (GODs) and orthogonal Latin hypercube designs are established. The generated Latin hypercube designs have some favorable properties such as uniformity, orthogonality of the first and some second order terms, and optimality under the defined criteria.     

Comparisons of search designs using search probabilities

Search designs are considered for searching and estimating one nonzero interaction from the two and three factor interactions under the search linear model. We compare three 12-run search designs D1, D2, and D3, and three 11-run search designs D4, D5, and D6, for a 2⁴ factorial experiment. Designs D2 and D3 are orthogonal arrays of strength 2, D1 and D4 are balanced arrays of full strength, D5 is a balanced array of strength 2, and D6 is obtained from D3 by deleting the duplicate run. Designs D4 and D5 are also obtained by deleting a run from D1 and D2, respectively. Balanced arrays and orthogonal arrays are commonly used factorial designs in scientific experiments. “Search probabilities” are calculated for the comparison of search designs. Three criteria based on search probabilities are presented to determine the design which is most likely to identify the nonzero interaction. The calculation of these search probabilities depends on an unknown parameter ρ which has a signal-to-noise ratio form. For a given value of ρ, Criteria I and II are newly proposed in this paper and Criteria III is given in Shirakura et al. (Ann. Statist. 24 (6) (1996) 2560). We generalize Criteria I–III for all values of ρ so that the comparison of search designs can be made without requiring a specific value of ρ. We have developed simplified methods for comparing designs under these three criteria for all values of ρ. We demonstrate, under all three criteria, that the balanced array D1 is more likely to identify the nonzero interaction than the orthogonal arrays D2 and D3, and the design D4 is more likely to identify the nonzero interaction than the designs D5 and D6.The methods of comparing designs developed in this paper are applicable to other factorial experiments for searching one nonzero interaction of any order.