Construction of nested space-filling designs using difference matrices

Experiments that study complex real world systems in business, engineering and sciences can be conducted at different levels of accuracy or sophistication. Nested space-filling designs are suitable for such multi-fidelity experiments. In this paper, we propose a systematic method to construct nested space-filling designs for experiments with two levels of accuracy. The method that makes use of nested difference matrices can be easily performed, many nested space-filling designs for experiments with two levels of accuracy can thus be constructed, and the resulting designs achieve stratification in low dimensions. In addition, the proposed method can also be used to obtain sliced space-filling designs for conducting computer experiments with both qualitative and quantitative factors.     

Block-circulant matrices for constructing optimal Latin hypercube designs

Computer simulations are usually needed to study a complex physical process. In this paper, we propose new procedures for constructing orthogonal or low-correlation block-circulant Latin hypercube designs. The basic concept of these methods is to use vectors with a constant periodic autocorrelation function to obtain suitable block-circulant Latin hypercube designs. A general procedure for constructing orthogonal Latin hypercube designs with favorable properties and allowing run sizes being different from a power of 2 (or a power of 2 plus 1), is presented here for the first time. In addition, an expansion of the method is given for constructing Latin hypercube designs with low correlation. This expansion is useful when orthogonal Latin hypercube designs do not exist. The properties of the generated designs are further investigated. Some examples of the new designs, as generated by the proposed procedures, are tabulated. In addition, a brief comparison with the designs that appear in the literature is given.     

Projection array based designs for computer experiments

In this paper we define a new class of designs for computer experiments. A projection array based design defines sets of simulation runs with properties that extend the conceptual properties of orthogonal array based Latin hypercube sampling, particularly to underlying design structures other than orthogonal arrays. Additionally, we illustrate how these designs can be sequentially augmented to improve the overall projection properties of the initial design or focus on interesting regions of the design space that need further exploration to improve the overall fit of the underlying response surface. We also illustrate how an initial Latin hypercube sample can be expressed as a projection array based design and show how one can augment these designs to improve higher dimensional space filling properties.     

Nested Latin Hypercube Designs with Sliced Structures

In computer experiments, space-filling designs with a sliced structure or nested structure have received much recent interest and been studied separately. However, it is likely that designs with both structures are needed in some situations, but there are no suitable designs so far. In this paper, we construct a special class of nested Latin hypercube designs with sliced structures, in such a design, a small sliced Latin hypercube design is nested within a large one. The construction method is easy to implement and the number of factors is flexible. Numerical simulations show the usefulness of the newly proposed designs.     

Design selection for strong orthogonal arrays

Strong orthogonal arrays (SOAs) were recently introduced and studied as a class of space‐filling designs for computer experiments. An important problem that has not been addressed in the literature is that of design selection for such arrays. In this article, we conduct a systematic investigation into this problem, and we focus on the most useful SOA(n,m,4,2 + )s and SOA(n,m,4,2)s. This article first addresses the problem of design selection for SOAs of strength 2+ by examining their three‐dimensional projections. Both theoretical and computational results are presented. When SOAs of strength 2+ do not exist, we formulate a general framework for the selection of SOAs of strength 2 by looking at their two‐dimensional projections. The approach is fruitful, as it is applicable when SOAs of strength 2+ do not exist and it gives rise to them when they do. The Canadian Journal of Statistics 47: 302–314; 2019 © 2019 Statistical Society of Canada     

An Approach to Constructing Nested Space-Filling Designs for Multi-Fidelity Computer Experiments

Multi-fidelity computer experiments are widely used in many engineering and scientific fields. Nested space-filling designs (NSFDs) are suitable for conducting such experiments. Two classes of NSFDs are currently available. One class is based on special orthogonal arrays of strength two and the other consists of nested Latin hypercube designs. Both of them assume all factors are continuous. We propose an approach to constructing new NSFDs based on powerful (t, s)-sequences. The method is simple, easy to implement, and quite general. For continuous factors, this approach produces NSFDs with better space-filling properties than existing ones. Unlike the previous methods, this method can also construct NSFDs for categorical and mixed factors. Some illustrative examples are given. Other applications of the constructed designs are briefly discussed.     

Orthogonal Latin hypercube designs for Fourier-polynomial models

Latin hypercube designs (LHDs) are widely used in computer experiments because of their one-dimensional uniformity and other properties. Recently, a number of methods have been proposed to construct LHDs with properties that all linear effects are mutually orthogonal and orthogonal to all second-order effects, i.e., quadratic effects and bilinear interactions. This paper focuses on the construction of LHDs with the above desirable properties under the Fourier-polynomial model. A convenient and flexible algorithm for constructing such orthogonal LHDs is provided. Most of the resulting designs have different run sizes from that of Butler (2001), and thus are new and very suitable for factor screening and building Fourier-polynomial models in computer experiments as discussed in Butler (2001).     

Pairwise orthogonal F-rectangle designs

The concept of pairwise orthogonal Latin square design is applied to r row by c column experiment designs which are called pairwise orthogonal F-rectangle designs. These designs are useful in designing successive and/or simulataneous experiments on the same set of rc experimental units, in constructing codes, and in constructing orthogonal arrays. A pair of orthogonal F-rectangle designs exists for any set of v treatment (symbols), whereas no pair of orthogonal Latin square designs of order two and six exists; one of the two construction methods presented does not rely on any previous knowledge about the existence of a pair of orthogonal Latin square designs, whereas the second one does. It is shown how to extend the methods to r=pv row by c=qv column designs and how to obtain t pairwise orthogonal F-rectangle design. When the maximum possible number of pairwise orthogonal F-rectangle designs is attained the set is said to be complete. Complete sets are obtained for all v for which v is a prime power. The construction method makes use of the existence of a complete set of pairwise orthogonal Latin square designs and of an orthogonal array with vⁿ columns, (vⁿ−1)/(v−1) rows, v symbols, and of strength two.     

Two-dimensional projection uniformity for space-filling designs

We investigate a space-filling criterion based on $L_2$-type discrepancies, namely the uniform projection criterion, aiming at improving designs' two-dimensional projection uniformity. Under a general reproducing kernel, we establish a formula for the uniform projection criterion function, which builds a connection between rows and columns of the design. For the commonly used discrepancies, we further use this formula to represent the two-dimensional projection uniformity in terms of the $L_p$-distances of U-type designs. These results generalize existing works and reveal new links between the two seemingly unrelated criteria of projection uniformity and the maximin $L_p$-distance for U-type designs. We also apply the obtained results to study several families of space-filling designs with appealing projection uniformity. Because of good projected space-filling properties, these designs are well adapted for computer experiments, especially for the case where not all the input factors are active.     

Defining equations for two-level factorial designs

Defining equations are introduced in the context of two-level factorial designs and they are shown to provide a concise specification of both regular and nonregular designs. The equations are used to find orthogonal arrays of high strength and some optimal designs. The latter optimal designs are formed in a new way by augmenting notional orthogonal arrays which are allowed to have some runs with a negative number of replicates before augmentation. Defining equations are also shown to be useful when the factorial design is blocked.     

A survey and evaluation of methods for determination of combinatorial equivalence of factorial designs

Equivalent factorial designs have identical statistical properties for estimation of factorial contrasts and for model fitting. Non-equivalent designs, however, may have the same statistical properties under one particular model but different properties under a different model. In this paper, we describe known methods for the determination of equivalence or non-equivalence of two-level factorial designs, whether they be regular factorial designs, non-regular orthogonal arrays, or have no particular structure. In addition, we evaluate a number of potential fast screening methods for detecting non-equivalence of designs. Although the paper concentrates mainly on symmetric designs with factors at two levels, we also evaluate methods of determining combinatorial equivalence and non-equivalence of three-level designs and indicate extensions to larger numbers of levels and to asymmetric designs.     

Comparisons of search designs using search probabilities

Search designs are considered for searching and estimating one nonzero interaction from the two and three factor interactions under the search linear model. We compare three 12-run search designs D1, D2, and D3, and three 11-run search designs D4, D5, and D6, for a 2⁴ factorial experiment. Designs D2 and D3 are orthogonal arrays of strength 2, D1 and D4 are balanced arrays of full strength, D5 is a balanced array of strength 2, and D6 is obtained from D3 by deleting the duplicate run. Designs D4 and D5 are also obtained by deleting a run from D1 and D2, respectively. Balanced arrays and orthogonal arrays are commonly used factorial designs in scientific experiments. “Search probabilities” are calculated for the comparison of search designs. Three criteria based on search probabilities are presented to determine the design which is most likely to identify the nonzero interaction. The calculation of these search probabilities depends on an unknown parameter ρ which has a signal-to-noise ratio form. For a given value of ρ, Criteria I and II are newly proposed in this paper and Criteria III is given in Shirakura et al. (Ann. Statist. 24 (6) (1996) 2560). We generalize Criteria I–III for all values of ρ so that the comparison of search designs can be made without requiring a specific value of ρ. We have developed simplified methods for comparing designs under these three criteria for all values of ρ. We demonstrate, under all three criteria, that the balanced array D1 is more likely to identify the nonzero interaction than the orthogonal arrays D2 and D3, and the design D4 is more likely to identify the nonzero interaction than the designs D5 and D6.The methods of comparing designs developed in this paper are applicable to other factorial experiments for searching one nonzero interaction of any order.     

Some new classes of orthogonal Latin hypercube designs

Orthogonal Latin hypercube (OLH) is a good design choice in a polynomial function model for computer experiments, because it ensures uncorrelated estimation of linear effects when a first-order model is fitted. However, when a second-order model is adopted, an OLH also needs to satisfy the additional property that each column is orthogonal to the elementwise square of all columns and orthogonal to the elementwise product of every pair of columns. Such class of OLHs is called OLHs of order two while the former class just possessing two-dimensional orthogonality is called OLHs of order one. In this paper we present a general method for constructing OLHs of orders one and two for n=s^m$n=s^m$ runs, where s and m may be any positive integers greater than one, by rotating a grouped orthogonal array with a column-orthogonal rotation matrix. The Kronecker product and the stacking methods are revisited and combined to construct some new classes of OLHs of orders one and two with other flexible numbers of runs. Some useful OLHs of order one or two with larger factor-to-run ratio and moderate runs are tabulated and discussed.     

Computer experiment designs for accurate prediction

Computer experiments using deterministic simulators are sometimes used to replace or supplement physical system experiments. This paper compares designs for an initial computer simulator experiment based on empirical prediction accuracy; it recommends designs for producing accurate predictions. The basis for the majority of the designs compared is the integrated mean squared prediction error (IMSPE) that is computed assuming a Gaussian process model with a Gaussian correlation function. Designs that minimize the IMSPE with respect to a fixed set of correlation parameters as well as designs that minimize a weighted IMSPE over the correlation parameters are studied. These IMSPE-based designs are compared with three widely-used space-filling designs. The designs are used to predict test surfaces representing a range of stationary and non-stationary functions. For the test conditions examined in this paper, the designs constructed under IMSPE-based criteria are shown to outperform space-filling Latin hypercube designs and maximum projection designs when predicting smooth functions of stationary appearance, while space-filling and maximum projection designs are superior for test functions that exhibit strong non-stationarity.     

Design of computer experiments: space filling and beyond

When setting up a computer experiment, it has become a standard practice to select the inputs spread out uniformly across the available space. These so-called space-filling designs are now ubiquitous in corresponding publications and conferences. The statistical folklore is that such designs have superior properties when it comes to prediction and estimation of emulator functions. In this paper we want to review the circumstances under which this superiority holds, provide some new arguments and clarify the motives to go beyond space-filling. An overview over the state of the art of space-filling is introducing and complementing these results.     

On the construction of balanced and orthogonal arrays

Two series of three symbol balanced arrays of strength two are constructed. Using special classes of BIB designs, two classes of two symbol orthogonal arrays of strength three are constructed.     

A Comparison of Three-level Orthogonal Arrays in the Presence of a Possible Correlation in Observations

When we want to compare two designs we usually assume the standard linear model with uncorrelated observations. In this paper we use the comparison method proposed by Ghosh & Shen (2006) to compare three level orthogonal arrays with 18, 27 and 36 runs under a possible presence of correlation in observations.     

Some new constructions of strength 3 mixed orthogonal arrays

Orthogonal arrays of strength 3 permit estimation of all the main effects of the experimental factors free from confounding or contamination with 2-factor interactions. We introduce methods of using arithmetic formulations and Latin squares to construct mixed orthogonal arrays of strength 3. Although the methods could be well extended to computing larger arrays, we confine computing to at most 100 run orthogonal arrays for practical uses. We find new arrays with run sizes 80 and 96, each has many distinct factor levels.     

Optimal Latin hypercube designs for the Kullback–Leibler criterion

Space-filling designs are commonly used for selecting the input values of time-consuming computer codes. Computer experiment context implies two constraints on the design. First, the design points should be evenly spread throughout the experimental region. A space-filling criterion (for instance, the maximin distance) is used to build optimal designs. Second, the design should avoid replication when projecting the points onto a subset of input variables (non-collapsing). The Latin hypercube structure is often enforced to ensure good projective properties. In this paper, a space-filling criterion based on the Kullback–Leibler information is used to build a new class of Latin hypercube designs. The new designs are compared with several traditional optimal Latin hypercube designs and appear to perform well.     

On the inadequacy of the customary orthogonal arrays in quality control and general scientific experimentation,and the need of probing designs of higher revealing power ∗

This paper breaks new ground concerning the general problem of factorial experimentation, namely, the identification and estimation of nonnegligible factorial effects (with minimal number of runs), without making the usual unrealistic and artificial assumptions concerning the negligibility of higher order interactions. (In other words, we consider the general factor screening problem when interactions may be present.) Through an example, it is shown that the customary orthogonal arrays fall short of the need. New principles for sieving the set of factorial effects to determine the large ones, are introduced. The concept of ‘revealing power’of designs, i.e. of their ability to help identify nonnegligible parameters is developed, and the usefulness in this direction of balanced arrays of full strength is studied.