Construction of nested space-filling designs using difference matrices

Experiments that study complex real world systems in business, engineering and sciences can be conducted at different levels of accuracy or sophistication. Nested space-filling designs are suitable for such multi-fidelity experiments. In this paper, we propose a systematic method to construct nested space-filling designs for experiments with two levels of accuracy. The method that makes use of nested difference matrices can be easily performed, many nested space-filling designs for experiments with two levels of accuracy can thus be constructed, and the resulting designs achieve stratification in low dimensions. In addition, the proposed method can also be used to obtain sliced space-filling designs for conducting computer experiments with both qualitative and quantitative factors.     

Computer experiment designs for accurate prediction

Computer experiments using deterministic simulators are sometimes used to replace or supplement physical system experiments. This paper compares designs for an initial computer simulator experiment based on empirical prediction accuracy; it recommends designs for producing accurate predictions. The basis for the majority of the designs compared is the integrated mean squared prediction error (IMSPE) that is computed assuming a Gaussian process model with a Gaussian correlation function. Designs that minimize the IMSPE with respect to a fixed set of correlation parameters as well as designs that minimize a weighted IMSPE over the correlation parameters are studied. These IMSPE-based designs are compared with three widely-used space-filling designs. The designs are used to predict test surfaces representing a range of stationary and non-stationary functions. For the test conditions examined in this paper, the designs constructed under IMSPE-based criteria are shown to outperform space-filling Latin hypercube designs and maximum projection designs when predicting smooth functions of stationary appearance, while space-filling and maximum projection designs are superior for test functions that exhibit strong non-stationarity.     

Computer experiments: a review

In this paper we provide a broad introduction to the topic of computer experiments. We begin by briefly presenting a number of applications with different types of output or different goals. We then review modelling strategies, including the popular Gaussian process approach, as well as variations and modifications. Other strategies that are reviewed are based on polynomial regression, non-parametric regression and smoothing spline ANOVA. The issue of multi-level models, which combine simulators of different resolution in the same experiment, is also addressed. Special attention is given to modelling techniques that are suitable for functional data. To conclude the modelling section, we discuss calibration, validation and verification. We then review design strategies including Latin hypercube designs and space-filling designs and their adaptation to computer experiments. We comment on a number of special issues, such as designs for multi-level simulators, nested factors and determination of experiment size.     

Nested Latin Hypercube Designs with Sliced Structures

In computer experiments, space-filling designs with a sliced structure or nested structure have received much recent interest and been studied separately. However, it is likely that designs with both structures are needed in some situations, but there are no suitable designs so far. In this paper, we construct a special class of nested Latin hypercube designs with sliced structures, in such a design, a small sliced Latin hypercube design is nested within a large one. The construction method is easy to implement and the number of factors is flexible. Numerical simulations show the usefulness of the newly proposed designs.     

Comparing and generating Latin Hypercube designs in Kriging models

In Computer Experiments (CE), a careful selection of the design points is essential for predicting the system response at untried points, based on the values observed at tried points. In physical experiments, the protocol is based on Design of Experiments, a methodology whose basic principles are questioned in CE. When the responses of a CE are modeled as jointly Gaussian random variables with their covariance depending on the distance between points, the use of the so called space-filling designs (random designs, stratified designs and Latin Hypercube designs) is a common choice, because it is expected that the nearer the untried point is to the design points, the better is the prediction. In this paper we focus on the class of Latin Hypercube (LH) designs. The behavior of various LH designs is examined according to the Gaussian assumption with exponential correlation, in order to minimize the total prediction error at the points of a regular lattice. In such a special case, the problem is reduced to an algebraic statistical model, which is solved using both symbolic algebraic software and statistical software. We provide closed-form computation of the variance of the Gaussian linear predictor as a function of the design, in order to make a comparison between LH designs. In principle, the method applies to any number of factors and any number of levels, and also to classes of designs other than LHs. In our current implementation, the applicability is limited by the high computational complexity of the algorithms involved.     

Optimal Latin hypercube designs for the Kullback–Leibler criterion

Space-filling designs are commonly used for selecting the input values of time-consuming computer codes. Computer experiment context implies two constraints on the design. First, the design points should be evenly spread throughout the experimental region. A space-filling criterion (for instance, the maximin distance) is used to build optimal designs. Second, the design should avoid replication when projecting the points onto a subset of input variables (non-collapsing). The Latin hypercube structure is often enforced to ensure good projective properties. In this paper, a space-filling criterion based on the Kullback–Leibler information is used to build a new class of Latin hypercube designs. The new designs are compared with several traditional optimal Latin hypercube designs and appear to perform well.     

A new variable selection method for uniform designs

As an important class of space-filling designs, uniform designs (UDs) choose a set of points over a certain domain such that these points are uniformly scattered, under a specific discrepancy measure. They have been applied successfully in many industrial and scientific experiments since they appeared in 1980. A noteworthy and practical advantage is their ability to investigate a large number of high-level factors simultaneously with a fairly economical set of experimental runs. As a result, UDs can be properly used as experimental plans that are intended to derive the significant factors from a list of many potential ones. To this end, a new screening procedure is introduced via penalized least squares. A simulation study is conducted to support the proposed method, which reveals that it can be considered quite promising and expedient, as judged in terms of Type I and Type II error rates.     

On algorithmic construction of maximin distance designs

Maximin distance designs are useful for conducting expensive computer experiments. In this article, we compare some global optimization algorithms for constructing such designs. We also introduce several related space-filling designs, including nested maximin distance designs, sliced maximin distance designs, and general maximin distance designs with better projection properties. These designs possess more flexible structures than their analogs in the literature. Examples of these designs constructed by the algorithms are presented.     

Space-filling orthogonal arrays of strength two

This paper considers the use of orthogonal arrays of strength two as experimental designs for fitting a surrogate model. Contrary to standard space-filling designs or Latin hypercube designs, the points of an orthogonal array of strength two are well distributed when they are projected on the two-dimensional faces of the unit cube. The aim is to determine if this property allows one to fit an accurate surrogate model when the computer response is governed by second-order interactions of some input variables. The first part of the paper is devoted to the construction of orthogonal arrays with space-filling properties. In the second part, orthogonal arrays are compared with standard designs for fitting a Gaussian process model.     

Maximum expected entropy transformed Latin hypercube designs

Existing projection designs (e.g. maximum projection designs) attempt to achieve good space-filling properties in all projections. However, when using a Gaussian process (GP), model-based design criteria such as the entropy criterion is more appropriate. We employ the entropy criterion averaged over a set of projections, called expected entropy criterion (EEC), to generate projection designs. We show that maximum EEC designs are invariant to monotonic transformations of the response, i.e. they are optimal for a wide class of stochastic process models. We also demonstrate that transformation of each column of a Latin hypercube design (LHD) based on a monotonic function can substantially improve the EEC. Two types of input transformations are considered: a quantile function of a symmetric Beta distribution chosen to optimize the EEC, and a nonparametric transformation corresponding to the quantile function of a symmetric density chosen to optimize the EEC. Numerical studies show that the proposed transformations of the LHD are efficient and effective for building robust maximum EEC designs. These designs give projections with markedly higher entropies and lower maximum prediction variances (MPV''s) at the cost of small increases in average prediction variances (APV''s) compared to state-of-the-art space-filling designs over wide ranges of covariance parameter values.     

A Bayesian criterion for selecting super saturated screening designs

Super-saturated designs in which the number of factors under investigation exceeds the number of experimental runs have been suggested for screening experiments initiated to identify important factors for future study. Most of the designs suggested in the literature are based on natural but ad hoc criteria. The “average s²” criteria introduced by Booth and Cox (Technometrics 4 (1962) 489) is a popular choice. Here, a decision theoretic approach is pursued leading to an optimality criterion based on misclassification probabilities in a Bayesian model. In certain cases, designs optimal under the average s² criterion are also optimal for the new criterion. Necessary conditions for this to occur are presented. In addition, the new criterion often provides a strict preference between designs tied under the average s² criterion, which is advantageous in numerical search as it reduces the number of local minima.     

Design of computer experiments: space filling and beyond

When setting up a computer experiment, it has become a standard practice to select the inputs spread out uniformly across the available space. These so-called space-filling designs are now ubiquitous in corresponding publications and conferences. The statistical folklore is that such designs have superior properties when it comes to prediction and estimation of emulator functions. In this paper we want to review the circumstances under which this superiority holds, provide some new arguments and clarify the motives to go beyond space-filling. An overview over the state of the art of space-filling is introducing and complementing these results.     

An Adaptive Exploration-Exploitation Algorithm for Constructing Metamodels in Random Simulation Using a Novel Sequential Experimental Design

In this article, we propose a novel algorithm for sequential design of metamodels in random simulation, which combines the exploration capability of most one-shot space-filling designs with the exploitation feature of common sequential designs. The algorithm continuously maintains a balance between the exploration and the exploitation search throughout the search process in a sequential and adaptive manner. The numerical results indicate that the proposed approach is superior to one of the existing well-known sequential designs in terms of both the computational efficiency and speed in generating efficient experimental designs.     

Split-plot designs for multistage experimentation

Most of today’s complex systems and processes involve several stages through which input or the raw material has to go before the final product is obtained. Also in many cases factors at different stages interact. Therefore, a holistic approach for experimentation that considers all stages at the same time will be more efficient. However, there have been only a few attempts in the literature to provide an adequate and easy-to-use approach for this problem. In this paper, we present a novel methodology for constructing two-level split-plot and multistage experiments. The methodology is based on the Kronecker product representation of orthogonal designs and can be used for any number of stages, for various numbers of subplots and for different number of subplots for each stage. The procedure is demonstrated on both regular and nonregular designs and provides the maximum number of factors that can be accommodated in each stage. Furthermore, split-plot designs for multistage experiments with good projective properties are also provided.     

Minimum projection uniformity for computer experiment with quantitative factors

Computer experiments involving quantitative factors at high levels are becoming more and more important in the study of complex experiments arising in the area of science and engineering. Uniform designs are found to be widely applicable in computer experiments in the form of space-filling designs. In this paper, the projection uniformity for quantitative designs is studied under wrap-around $L_2$-discrepancy. A lower bound of uniformity pattern for general asymmetric designs is provided, which can be used to serve as a benchmark for both comparing different designs and also to determine the optimal design. As a byproduct, a lower bound of wrap-around $L_2$-discrepancy measure for the asymmetric design is also obtained. Some illustrative examples and numerical comparisons are also provided for supporting our theoretical results.     

Robust surface estimation in multi-response multistage statistical optimization problems

As the ordinary least squares (OLS) method is very sensitive to outliers as well as to correlated responses, a robust coefficient estimation method is proposed in this paper for multi-response surfaces in multistage processes based on M-estimators. In this approach, experimental designs are used in which the intermediate response variables may act as covariates in the next stages. The performances of both the ordinary multivariate OLS and the proposed robust multi-response surface approach are analyzed and compared through extensive simulation experiments. Sum of the squared errors in estimating the regression coefficients reveals the efficiency of the proposed robust approach.     

Experimentation order in factorial designs: new findings

Under some very reasonable hypotheses, it becomes evident that randomizing the run order of a factorial experiment does not always neutralize the effect of undesirable factors. Yet, these factors do have an influence on the response, depending on the order in which the experiments are conducted. On the other hand, changing the factor levels is many times costly; therefore it is not reasonable to leave to chance the number of changes necessary. For this reason, run orders that offer the minimum number of factor level changes and at the same time minimize the possible influence of undesirable factors on the experimentation have been sought. Sequences which are known to produce the desired properties in designs with 8 and 16 experiments can be found in the literature. In this paper, we provide the best possible sequences for designs with 32 experiments, as well as sequences that offer excellent properties for designs with 64 and 128 experiments. The method used to find them is based on a mixture of algorithmic searches and an augmentation of smaller designs.     

Two-dimensional projection uniformity for space-filling designs

We investigate a space-filling criterion based on $L_2$-type discrepancies, namely the uniform projection criterion, aiming at improving designs' two-dimensional projection uniformity. Under a general reproducing kernel, we establish a formula for the uniform projection criterion function, which builds a connection between rows and columns of the design. For the commonly used discrepancies, we further use this formula to represent the two-dimensional projection uniformity in terms of the $L_p$-distances of U-type designs. These results generalize existing works and reveal new links between the two seemingly unrelated criteria of projection uniformity and the maximin $L_p$-distance for U-type designs. We also apply the obtained results to study several families of space-filling designs with appealing projection uniformity. Because of good projected space-filling properties, these designs are well adapted for computer experiments, especially for the case where not all the input factors are active.     

Efficient three-level screening designs using weighing matrices

Screening is the first stage of many industrial experiments and is used to determine efficiently and effectively a small number of potential factors among a large number of factors which may affect a particular response. In a recent paper, Jones and Nachtsheim [A class of three-level designs for definitive screening in the presence of second-order effects. J. Qual. Technol. 2011;43:1–15] have given a class of three-level designs for screening in the presence of second-order effects using a variant of the coordinate exchange algorithm as it was given by Meyer and Nachtsheim [The coordinate-exchange algorithm for constructing exact optimal experimental designs. Technometrics 1995;37:60–69]. Xiao et al. [Constructing definitive screening designs using conference matrices. J. Qual. Technol. 2012;44:2–8] have used conference matrices to construct definitive screening designs with good properties. In this paper, we propose a method for the construction of efficient three-level screening designs based on weighing matrices and their complete foldover. This method can be considered as a generalization of the method proposed by Xiao et al. [Constructing definitive screening designs using conference matrices. J. Qual. Technol. 2012;44:2–8]. Many new orthogonal three-level screening designs are constructed and their properties are explored. These designs are highly D-efficient and provide uncorrelated estimates of main effects that are unbiased by any second-order effect. Our approach is relatively straightforward and no computer search is needed since our designs are constructed using known weighing matrices.