Adaptive grid semidefinite programming for finding optimal designs

We find optimal designs for linear models using a novel algorithm that iteratively combines a semidefinite programming (SDP) approach with adaptive grid techniques. The proposed algorithm is also adapted to find locally optimal designs for nonlinear models. The search space is first discretized, and SDP is applied to find the optimal design based on the initial grid. The points in the next grid set are points that maximize the dispersion function of the SDP-generated optimal design using nonlinear programming. The procedure is repeated until a user-specified stopping rule is reached. The proposed algorithm is broadly applicable, and we demonstrate its flexibility using (i) models with one or more variables and (ii) differentiable design criteria, such as A-, D-optimality, and non-differentiable criterion like E-optimality, including the mathematically more challenging case when the minimum eigenvalue of the information matrix of the optimal design has geometric multiplicity larger than 1. Our algorithm is computationally efficient because it is based on mathematical programming tools and so optimality is assured at each stage; it also exploits the convexity of the problems whenever possible. Using several linear and nonlinear models with one or more factors, we show the proposed algorithm can efficiently find optimal designs.     

Computing A-optimal and E-optimal designs for regression models via semidefinite programming

In semidefinite programming (SDP), we minimize a linear objective function subject to a linear matrix being positive semidefinite. A powerful program, SeDuMi, has been developed in MATLAB to solve SDP problems. In this article, we show in detail how to formulate A-optimal and E-optimal design problems as SDP problems and solve them by SeDuMi. This technique can be used to construct approximate A-optimal and E-optimal designs for all linear and nonlinear regression models with discrete design spaces. In addition, the results on discrete design spaces provide useful guidance for finding optimal designs on any continuous design space, and a convergence result is derived. Moreover, restrictions in the designs can be easily incorporated in the SDP problems and solved by SeDuMi. Several representative examples and one MATLAB program are given.     

SDP vs. LP Relaxations for the Moment Approach in Some Performance Evaluation Problems

Abstract

Given a Markov process, we are interested in the numerical computation of the moments of the exit time from a bounded domain. We use a moment approach which, together with appropriate semidefinite positivity moment conditions, yields a sequence of semidefinite programs (or SDP relaxations), depending on the number of moments considered, that provide a sequence of nonincreasing (resp. nondecreasing) upper (resp. lower) bounds. The results are compared to the linear Hausdorff moment conditions approach considered for the LP relaxations in Helmes et al. [Helmes, K., Röhl, S., Stockbridge, R.H. Computing moments of the exit time distribution for Markov processes by linear programming. Oper. Res. 2001, 49, 516–530]. The SDP relaxations are shown to be more general and more precise than the LP relaxations.     

Bayesian optimal one point designs for one parameter nonlinear models

For nonlinear one parameter models and concave optimality criteria there always exists a locally optimal one point design. This can be proved by an application of Caratheodory's theorem (Läuter, Math. Operationsforsch. Statist. Ser. Statist. 5 (1974a) 625–636). If prior distributions with densities are used, this theorem gives no useful bound on the number of support points of a Bayesian optimal design. Chaloner (J. Statist. Plann. Inference, 37 (1993) 229–236) gave a sufficient condition on the support of the prior distribution for the existence of a Bayesian optimal one point design. In this article, a condition on the shape of the prior density is given, which is also sufficient for the existence of a Bayesian optimal one point design in nonlinear models with one parameter.     

Optimal designs for experiments with mixtures: a survey

This is a survey article on known results about analytic solutions and numerical solutions of optimal designs for various regression models for experiments with mixtures. The regression models include polynomial models, models containing homogeneous functions, models containing inverse terms and ratios, log contrast models, models with quantitative variables, and mod els containing the amount of mixture, Optimality criteria considered include D-, A-, E-,φ_p- and I_λ-Optimalities. Uniform design and uniform optimal design for mixture components, and efficiencies of the {q,2} simplex-controid design are briefly discussed.     

On computer generation of balanced arrays

This paper provides an algebraic (and hence computing) procedure for generation of balanced arrays having two symbols, m rows, specified minimum and maximum column weights, arbitrary strength t≦m, and index set parameters μ^t₁, μ^t₂,…, μ^t_t. μ^t₀ is unspecified, and calculated as part of the algorithm, although the procedure for specifying it is straightforward and can be used if desired. Array generation is herein reduced to finding integral solutions to a linear programming problem. It is shown that the integral solutions of the system of equations comprise all balanced arrays with the given set of parameters.A computing algorithm is provided which constructs the system of equations to be solved; it has been interfaced with a standard linear programming package to provide some preliminary results.Additional algorithms whose development should result in substantial decreases in computing costs are discussed.     

On linear regression designs which maximize information

Necessary and sufficient conditions are established when a continuous design contains maximal information for a prescribed s-dimensional parameter in a classical linear model. The development is based on a thorough study of a particular dual problem and its interplay with the optimal design problem, extending partial results and earlier approaches based on differential calculus, game theory, and other programming methods. The results apply in particular to a class of information functionals which covers c-, D-, A-, L-optimality, they include a complete account of the non-differentiable criterion of E-optimality, and provide a constructive treatment of those situations in which the information matrix is singular. Corollaries pertain to the case of s out of k parameters, simultaneous optimality with respect to several criteria, multiplicity of optimal designs, bounds on their weights, and optimality which is induced by admissibility.     

Some sufficient conditions for the type 1 optimality of block designs

In this paper, we investigate the problem of determining block designs which are optimal under type 1 optimality criteria within various classes of designs having υ treatments arranged in b blocks of size k. The solutions to two optimization problems are given which are related to a general result obtained by Cheng (1978) and which are useful in this investigation. As one application of the solutions obtained, the definition of a regular graph design given in Mitchell and John (1977) is extended to that of a semi-regular graph design and some sufficient conditions are derived for the existence of a semi-regular graph design which is optimal under a given type 1 criterion. A result is also given which shows how the sufficient conditions derived can be used to establish the optimality under a specific type 1 criterion of some particular types of semi- regular graph designs having both equal and unequal numbers of replicates. Finally,some sufficient conditions are obtained for the dual of an A- or D-optimal design to be A- or D-optimal within an appropriate class of dual designs.     

Rotation designs for experiments in high-bias situations

Many experiments in the physical and engineering sciences study complex processes in which bias due to model inadequacy dominates random error. A noteworthy example of this situation is the use of computer experiments, in which scientists simulate the phenomenon being studied by a computer code. Computer experiments are deterministic: replicate observations from running the code with the same inputs will be identical. Such high-bias settings demand different techniques for design and prediction. This paper will focus on the experimental design problem introducing a new class of designs called rotation designs. Rotation designs are found by taking an orthogonal starting design D and rotating it to obtain a new design matrix D_R=DR, where R is any orthonormal matrix. The new design is still orthogonal for a first-order model. In this paper, we study some of the properties of rotation designs and we present a method to generate rotation designs that have some appealing symmetry properties.     

Construction of orthogonal Latin hypercube designs with flexible run sizes

Latin hypercube designs (LHDs) have recently found wide applications in computer experiments. A number of methods have been proposed to construct LHDs with orthogonality among main-effects. When second-order effects are present, it is desirable that an orthogonal LHD satisfies the property that the sum of elementwise products of any three columns (whether distinct or not) is 0. The orthogonal LHDs constructed by Ye (1998), Cioppa and Lucas (2007), Sun et al. (2009) and Georgiou (2009) all have this property. However, the run size n of any design in the former three references must be a power of two (n=2^c) or a power of two plus one (n=2^c+1), which is a rather severe restriction. In this paper, we construct orthogonal LHDs with more flexible run sizes which also have the property that the sum of elementwise product of any three columns is 0. Further, we compare the proposed designs with some existing orthogonal LHDs, and prove that any orthogonal LHD with this property, including the proposed orthogonal LHD, is optimal in the sense of having the minimum values of ave(|t|), t_max, ave(|q|) and q_max.     

Optimal and efficient designs for 2-parameter nonlinear models

By Carathéodory's theorem, for a k-parameter nonlinear model, the minimum number of support points for any D-optimal design is between k and k(k+1)/2. Characterizing classes of models for which a D-optimal design sits on exactly k support points is of great theoretical interest. By utilizing the equivalence theorem, we identify classes of 2-parameter nonlinear models for which a D-optimal design is precisely supported on 2 points. We also introduce the theory of maximum principle from differential equations into the design area and obtain some results on characterizing the minimally supported nonlinear designs. Examples are given to demonstrate our results. Designs with minimum number of support points may not always be suitable in practice. To alleviate this problem, we utilize some geometric and analytical methods to obtain some efficient designs which provide more opportunity for the model checking and prevent biases due to mis-specified initial parameters.     

Optimal experimental design criterion for discriminating semiparametric models

This paper studies the optimal experimental design problem to discriminate two regression models. Recently, López-Fidalgo et al. [2007. An optimal experimental design criterion for discriminating between non-normal models. J. Roy. Statist. Soc. B 69, 231–242] extended the conventional T-optimality criterion by Atkinson and Fedorov [1975a. The designs of experiments for discriminating between two rival models. Biometrika 62, 57–70; 1975b. Optimal design: experiments for discriminating between several models. Biometrika 62, 289–303] to deal with non-normal parametric regression models, and proposed a new optimal experimental design criterion based on the Kullback–Leibler information divergence. In this paper, we extend their parametric optimality criterion to a semiparametric setup, where we only need to specify some moment conditions for the null or alternative regression model. Our criteria, called the semiparametric Kullback–Leibler optimality criteria, can be implemented by applying a convex duality result of partially finite convex programming. The proposed method is illustrated by a simple numerical example.     

Optimal designs for second-degree Kronecker model mixture experiments

The paper investigates optimal designs in the second-degree Kronecker model for mixture experiments. Three groups of novel results are presented: (i) characterization of feasible weighted centroid designs for a maximum parameter system, (ii) derivations of D-, A-, and E-optimal weighted centroid designs, and (iii) numerically φ_p-optimal weighted centroid designs. Results on a quadratic subspace of invariant symmetric matrices containing the information matrices involved in the design problem serve as a main tool throughout the analysis.     

Discrepancy for uniform design of experiments with mixtures

The uniform design is a kind of space filling design that is robust against the model specification. The uniform design has been widely applied to experiments with mixtures. In this paper, we propose a new discrepancy DM₂-discrepancy as a new criterion to measure the uniformity of designs with mixtures. A computational formula of the new discrepancy, by the functional method, is also given. This property overcome the main disadvantage of the discrepancies proposed before.     

Minimax regression designs for approximately linear models with autocorrelated errors

We study the construction of regression designs, when the random errors are autocorrelated. Our model of dependence assumes that the spectral density g(ω) of the error process is of the form g(ω) = (1 − α)g₀(ω) + αg₁(ω), where g₀(ω) is uniform (corresponding to uncorrelated errors), α ϵ [0, 1) is fixed, and g₁(ω) is arbitrary. We consider regression responses which are exactly, or only approximately, linear in the parameters. Our main results are that a design which is asymptotically (minimax) optimal for uncorrelated errors retains its optimality under autocorrelation if the design points are a random sample, or a random permutation, of points from this distribution. Our results are then a partial extension of those of Wu (Ann. Statist. 9 (1981), 1168–1177), on the robustness of randomized experimental designs, to the field of regression design.     

A theorem for studying the Multiplier Conjecture and its applications

In this paper a theorem for studying the Multiplier Conjecture is proved. Using this theorem and the method presented in Qiu, J. Combin. Des. (1997), we give simplified proofs of our multiplier theorems in cases n=2n₁ and 3n₁.     

Maximin efficient design of experiment for exponential regression models

In this paper robust and efficient designs are derived for several exponential decay models. These models are widely used in chemistry, pharmacokinetics or microbiology. We propose a maximin approach, which determines the optimal design such that a minimum of the D-efficiencies (taken over a certain range) becomes maximal. Analytic solutions are derived if optimization is performed in the class of minimal supported designs. In general the optimal designs with respect to the maximin criterion have to be determined numerically and some properties of these designs are also studied. We also illustrate the benefits of our approach by reanalysing a data example from the literature.     

Minimax subset selection for the multinomial distribution

Let (X₁,…,X_k) be a multinomial vector with unknown cell probabilities (p₁,?,p_k). A subset of the cells is to be selected in a way so that the cell associated with the smallest cell probability is included in the selected subset with a preassigned probability, P¹. Suppose the loss is measured by the size of the selected subset, S. Using linear programming techniques, selection rules can be constructed which are minimax with respect to S in the class of rules which satisfy the P¹-condition. In some situations, the rule constructed by this method is the rule proposed by Nagel (1970). Similar techniques also work for selection in terms of the largest cell probability.