Optimal Designs for 2 k Choice Experiments

Abstract

In this article we establish the choice sets in the D-optimal design for a choice experiment for testing main effects and for testing main effects and two-factor interactions, when there are k attributes, each with two levels, for choice set size m. We also give a method to construct optimal and near-optimal designs with small numbers of choice sets.     

Optimal stated preference choice experiments when all choice sets contain a specific option

Stated preference choice experiments are routinely used in many areas from marketing to medicine. While results on the optimal choice sets to present for the forced choice setting have been determined in a variety of situations, no results have appeared to date on the optimal choice sets to use when either all choice sets are to contain a common base alternative or when all choice sets contain a “none of these” option. These problems are considered in this paper.     

Choice experiments for estimating main effects and interactions

Choice-based conjoint experiments are used when choice alternatives can be described in terms of attributes. The objective is to infer the value that respondents attach to attribute levels. This method involves the design of profiles on the basis of attributes specified at certain levels. Respondents are presented sets of profiles and asked to select the one they consider best. However if choice sets have too many profiles, they may be difficult to implement. In this paper we provide strategies for reducing the number of profiles in choice sets. We consider situations where only a subset of interactions is of interest, and we obtain connected main effect plans with smaller choice sets that are capable of estimating subsets of two-factor and three-factor interactions in 2ⁿ and 3ⁿ plans. We also provide connected main effect plans for mixed level designs.     

Design strategies for sequential choice experiments involving economic alternatives

Valuation is a fundamental task of government, for-profit, and not-for-profit business. A major subset of valuation issues concerns situations where decision alternatives may be described by benefits and costs and the objective is to infer the values respondents attach to benefit/cost levels. For studies of this sort, computer administration enables the course of data collection to depend on prior responses, which allows the study to adapt to responses made by subjects. This capability is very useful when the objective is to identify which coefficients to include in a model, e.g., whether to include interaction terms. A disadvantage of computer administration, however, is that presenting many alternatives in a single presentation may not be possible because limited screen real estate may sharply limit the number of attributes and alternatives that may appear in choice sets. This paper shows how attribute and attribute level sub-setting may be used to create choice sets for use in choice-based evaluation studies sequentially. Initially data are collected and the model fit tested for main effects. If main effects model gives a good fit, conclusions are drawn on main effects. Otherwise, more choice sets will be included, data are collected and the model fit tested for main effects and two-way interactions. If that model fits, conclusions are drawn on main effects and two-way interactions. Otherwise, more data collected with added choice sets and conclusions are drawn on main effects, two-way and three-way interactions.     

A catalogue of designs for partial profiles in paired comparison experiments with three groups of factors

A common strategy for avoiding information overload in multi-factor paired comparison experiments is to employ pairs of options which have different levels for only some of the factors in a study. For the practically important case where the factors fall into three groups such that all factors within a group have the same number of levels and where one is only interested in estimating the main effects, a comprehensive catalogue of D-optimal approximate designs is presented. These optimal designs use at most three different types of pairs and have a block diagonal information matrix.     

Optimal designs for stated choice experiments that incorporate ties

In 1970 Davidson generalised the Bradley–Terry model to allow respondents to say that the two options presented in a choice task were equally attractive. In this paper we extend this idea to the MNL model with m options in each choice set and we show that the optimal designs for the MNL model are also optimal in this setting.     

Optimal design for discrete choice experiments

In this paper we derive locally optimal designs for discrete choice experiments. As in Kanninen (2002) we consider a multinomial logistic model, which contains various qualitative attributes as well as a quantitative one, which may range over a sufficiently large interval. The derived optimal designs improve upon those given in the literature, but have the feature that every choice set contains alternatives, which coincide in all but the quantitative attributes. The multinomial logistic model will then lead to a response behavior, which is apparently unrealistic.     

A nonparametric plug-in rule for selecting optimal block lengths for block bootstrap methods

In this paper, we consider the problem of empirical choice of optimal block sizes for block bootstrap estimation of population parameters. We suggest a nonparametric plug-in principle that can be used for estimating ‘mean squared error’-optimal smoothing parameters in general curve estimation problems, and establish its validity for estimating optimal block sizes in various block bootstrap estimation problems. A key feature of the proposed plug-in rule is that it can be applied without explicit analytical expressions for the constants that appear in the leading terms of the optimal block lengths. Furthermore, we also discuss the computational efficacy of the method and explore its finite sample properties through a simulation study.     

Effect of choice complexity on design efficiency in conjoint choice experiments

Conjoint choice experiments have become a powerful tool to explore individual preferences. The consistency of respondents' choices depends on the choice complexity. For example, it is easier to make a choice between two alternatives with few attributes than between five alternatives with several attributes. In the latter case it will be much harder to choose the preferred alternative which is reflected in a higher response error. Several authors have dealt with this choice complexity in the estimation stage but very little attention has been paid to set up designs that take this complexity into account. The core issue of this paper is to find out whether it is worthwhile to take this complexity into account in the design stage. We construct efficient semi-Bayesian D-optimal designs for the heteroscedastic conditional logit model which is used to model the across respondent variability that occurs due to the choice complexity. The degree of complexity is measured by the entropy, as suggested by Swait and Adamowicz (2001). The proposed designs are compared with a semi-Bayesian D-optimal design constructed without taking the complexity into account. The simulation study shows that it is much better to take the choice complexity into account when constructing conjoint choice experiments.     

Optimal designs for estimating a choice hierarchy by a general nested multinomial logit model

Abstract

In choice experiments the process of decision-making can be more complex than the proposed by the Multinomial Logit Model (MNL). In these scenarios, models such as the Nested Multinomial Logit Model (NMNL) are often employed to model a more complex decision-making. Understanding the decision-making process is important in some fields such as marketing. Achieving a precise estimation of the models is crucial to the understanding of this process. To do this, optimal experimental designs are required. To construct an optimal design, information matrix is key. A previous research by others has developed the expression for the information matrix of the two-level NMNL model with two nests: Alternatives nest (J alternatives) and No-Choice nest (1 alternative). In this paper, we developed the likelihood function for a two-stage NMNL model for M nests and we present the expression for the information matrix for 2 nests with any amount of alternatives in them. We also show alternative D-optimal designs for No-Choice scenarios with similar relative efficiency but with less complex alternatives which can help to obtain more reliable answers and one application of these designs.     

OPTIMAL DESIGNS FOR 2 k PAIRED COMPARISON EXPERIMENTS

In this paper we establish the form of the optimal paired comparison design when there are k attributes, each with two levels, for testing for main effects, for main effects and two factor interactions and for main effects and two and three factor interactions. In all cases we assume that all pairs with the same number of attributes different appear equally often. In this setting the D and A optimal designs for main effects are the foldover pairs and those for main effects and two factor interactions have pairs in which about half the attributes are different.     

Locally D-Optimal Design for a Logit Model in Discrete Choice Experiment

Conjoint analysis is concerned with understanding how people make choice between products or services (alternatives) or a combination of product and service (choice set), so that businesses can design new products or services that better meet customers needs. In this situation, logit model (Multinomial Logit Model) has been used to calculate the probability related to choosing an alternative in a choice set with the highest utility. Then I considered several choice sets instead of one. In this article, I have used the locally D-optimal design for the combination of the level of attributes (two attributes each with two levels) to create alternatives. The optimal combination of alternatives in choice sets which help us to have a suitable choice.     

Building adaptive estimating equations when inverse of covariance estimation is difficult

Summary. To construct an optimal estimating function by weighting a set of score functions, we must either know or estimate consistently the covariance matrix for the individual scores. In problems with high dimensional correlated data the estimated covariance matrix could be unreliable. The smallest eigenvalues of the covariance matrix will be the most important for weighting the estimating equations, but in high dimensions these will be poorly determined. Generalized estimating equations introduced the idea of a working correlation to minimize such problems. However, it can be difficult to specify the working correlation model correctly. We develop an adaptive estimating equation method which requires no working correlation assumptions. This methodology relies on finding a reliable approximation to the inverse of the variance matrix in the quasi-likelihood equations. We apply a multivariate generalization of the conjugate gradient method to find estimating equations that preserve the information well at fixed low dimensions. This approach is particularly useful when the estimator of the covariance matrix is singular or close to singular, or impossible to invert owing to its large size.     

Optimal Designs for Stated Choice Experiments that Incorporate Position Effects

Davidson and Beaver (1977 Davidson , R. R. , Beaver , R. J. ( 1977 ). On extending the Bradley–Terry model to incorporate within-pair order effects . Biometrics 33 : 693 – 702 .[Crossref], [Web of Science ®] , [Google Scholar]) extended the Bradley–Terry model to incorporate the possible effect of position within a choice set on the choices made in paired comparisons experiments. We further extend the Davidson–Beaver result to choice sets of any size and show, under a mild restriction, that designs optimal for the multinomial logit model are still optimal. Designs balanced for carry-over effects of all orders can be used to construct designs with a diagonal information matrix for attribute effects. The theoretical results are derived assuming equal merits and we discuss the possible consequences of assuming unequal merits in an example.     

Stochastic variational inference for large-scale discrete choice models using adaptive batch sizes

Discrete choice models describe the choices made by decision makers among alternatives and play an important role in transportation planning, marketing research and other applications. The mixed multinomial logit (MMNL) model is a popular discrete choice model that captures heterogeneity in the preferences of decision makers through random coefficients. While Markov chain Monte Carlo methods provide the Bayesian analogue to classical procedures for estimating MMNL models, computations can be prohibitively expensive for large datasets. Approximate inference can be obtained using variational methods at a lower computational cost with competitive accuracy. In this paper, we develop variational methods for estimating MMNL models that allow random coefficients to be correlated in the posterior and can be extended easily to large-scale datasets. We explore three alternatives: (1) Laplace variational inference, (2) nonconjugate variational message passing and (3) stochastic linear regression. Their performances are compared using real and simulated data. To accelerate convergence for large datasets, we develop stochastic variational inference for MMNL models using each of the above alternatives. Stochastic variational inference allows data to be processed in minibatches by optimizing global variational parameters using stochastic gradient approximation. A novel strategy for increasing minibatch sizes adaptively within stochastic variational inference is proposed.     

THE ANALYSIS OF DISCRETE CHOICE EXPERIMENTS WITH CORRELATED ERROR STRUCTURE

In a stated preference discrete choice experiment each subject is typically presented with several choice sets, and each choice set contains a number of alternatives. The alternatives are defined in terms of their name (brand) and their attributes at specified levels. The task for the subject is to choose from each choice set the alternative with highest utility for them. The multinomial is an appropriate distribution for the responses to each choice set since each subject chooses one alternative, and the multinomial logit is a common model. If the responses to the several choice sets are independent, the likelihood function is simply the product of multinomials. The most common and generally preferred method of estimating the parameters of the model is maximum likelihood (that is, selecting as estimates those values that maximize the likelihood function). If the assumption of within-subject independence to successive choice tasks is violated (it is almost surely violated), the likelihood function is incorrect and maximum likelihood estimation is inappropriate. The most serious errors involve the estimation of the variance-covariance matrix of the model parameter estimates, and the corresponding variances of market shares and changes in market shares.

In this paper we present an alternative method of estimation of the model parameter coefficients that incorporates a first-order within-subject covariance structure. The method involves the familiar log-odds transformation and application of the multivariate delta method. Estimation of the model coefficients after the transformation is a straightforward generalized least squares regression, and the corresponding improved estimate of the variance-covariance matrix is in closed form. Estimates of market share (and change in market share) follow from a second application of the multivariate delta method. The method and comparison with maximum likelihood estimation are illustrated with several simulated and actual data examples.

Advantages of the proposed method are: 1) it incorporates the within-subject covariance structure; 2) it is completely data driven; 3) it requires no additional model assumptions; 4) assuming asymptotic normality, it provides a simple procedure for computing confidence regions on market shares and changes in market shares; and 5) it produces results that are asymptotically equivalent to those produced by maximum likelihood when the data are independent.     

The limitations of using school league tables to inform school choice

Summary. In England, so-called 'league tables' based on examination results and test scores are published annually, ostensibly to inform parental choice of secondary schools. A crucial limitation of these tables is that the most recent published information is based on the current performance of a cohort of pupils who entered secondary schools several years earlier, whereas for choosing a school it is the future performance of the current cohort that is of interest. We show that there is substantial uncertainty in predicting such future performance and that incorporating this uncertainty leads to a situation where only a handful of schools' future performances can be separated from both the overall mean and from one another with an acceptable degree of precision. This suggests that school league tables, including value-added tables, have very little to offer as guides to school choice.     

Optimal designs for main effects in linear paired comparison models

In psychology, marketing research and sensory analysis paired comparisons which demand judges to evaluate the trade-off between two alternatives constitute a popular method of data collection. For this situation we present optimal designs in a discrete setting when the alternatives are specified by an analysis of variance model with main effects only. We employ combinatorial tools to achieve optimal designs which have sufficiently small sample sizes. Moreover, optimal designs are constructed when the number of factors presented is restricted for each pair of alternatives.     

Efficiency of generalized estimating equations for binary responses

Summary.  Using standard correlation bounds, we show that in generalized estimation equations (GEEs) the so-called 'working correlation matrix' R ( α ) for analysing binary data cannot in general be the true correlation matrix of the data. Methods for estimating the correlation param-eter in current GEE software for binary responses disregard these bounds. To show that the GEE applied on binary data has high efficiency, we use a multivariate binary model so that the covariance matrix from estimating equation theory can be compared with the inverse Fisher information matrix. But R ( α ) should be viewed as the weight matrix, and it should not be confused with the correlation matrix of the binary responses. We also do a comparison with more general weighted estimating equations by using a matrix Cauchy–Schwarz inequality. Our analysis leads to simple rules for the choice of α in an exchangeable or autoregressive AR(1) weight matrix R ( α ), based on the strength of dependence between the binary variables. An example is given to illustrate the assessment of dependence and choice of α .