Pretesting Experimental Instructions

Laboratory experiments, well established in sociology and social psychology, are alternate realities constructed for assessing derivations from theories. Experiments instantiate a theory's scope and initial conditions, and that information is usually delivered through instructions to participants. Because experiments often use video and computer technology and often test very precise predictions of new theories, we suggest developing objective means to assess information delivery. We illustrate these points by reference to a widely used standard experiment to assess theories of status processes. We first describe elements of good experimental design with their justifications. Next, we describe new techniques we have developed and illustrate their usefulness, showing results of a first use of the new techniques. While the assessment still relies somewhat on judgments, we find the technique useful and suggest further developments that might improve it for experimental and other research uses.     

Some Limitations of Qualitative Risk Rating Systems

Qualitative systems for rating animal antimicrobial risks using ordered categorical labels such as “high,”“medium,” and “low” can potentially simplify risk assessment input requirements used to inform risk management decisions. But do they improve decisions? This article compares the results of qualitative and quantitative risk assessment systems and establishes some theoretical limitations on the extent to which they are compatible. In general, qualitative risk rating systems satisfying conditions found in real‐world rating systems and guidance documents and proposed as reasonable make two types of errors: (1) Reversed rankings, i.e., assigning higher qualitative risk ratings to situations that have lower quantitative risks; and (2) Uninformative ratings, e.g., frequently assigning the most severe qualitative risk label (such as “high”) to situations with arbitrarily small quantitative risks and assigning the same ratings to risks that differ by many orders of magnitude. Therefore, despite their appealing consensus‐building properties, flexibility, and appearance of thoughtful process in input requirements, qualitative rating systems as currently proposed often do not provide sufficient information to discriminate accurately between quantitatively small and quantitatively large risks. The value of information (VOI) that they provide for improving risk management decisions can be zero if most risks are small but a few are large, since qualitative ratings may then be unable to confidently distinguish the large risks from the small. These limitations suggest that it is important to continue to develop and apply practical quantitative risk assessment methods, since qualitative ones are often unreliable.     

Why Reduced-Form Regression Models of Health Effects Versus Exposures Should Not Replace QRA: Livestock Production and Infant Mortality as an Example

Do pollution emissions from livestock operations increase infant mortality rate (IMR)? A recent regression analysis of changes in IMR against changes in aggregate “animal units” (a weighted sum of cattle, pig, and poultry numbers) over time, for counties throughout the United States, suggested the provocative conclusion that they do: “[A] doubling of production leads to a 7.4% increase in infant mortality.” Yet, we find that regressing IMR changes against changes in specific components of “animal units” (cattle, pigs, and broilers) at the state level reveals statistically significant negative associations between changes in livestock production (especially, cattle production) and changes in IMR. We conclude that statistical associations between livestock variables and IMR variables are very sensitive to modeling choices (e.g., selection of explanatory variables, and use of specific animal types vs. aggregate “animal units). Such associations, whether positive or negative, do not warrant causal interpretation. We suggest that standard methods of quantitative risk assessment (QRA), including emissions release (source) models, fate and transport modeling, exposure assessment, and dose-response modeling, really are important—and indeed, perhaps, essential—for drawing valid causal inferences about health effects of exposures to guide sound, well-informed public health risk management policy. Reduced-form regression models, which skip most or all of these steps, can only quantify statistical associations (which may be due to model specification, variable selection, residual confounding, or other noncausal factors). Sound risk management requires the extra work needed to identify and model valid causal relations.     

Game Theory and Risk Analysis

Risk analysts often analyze adversarial risks from terrorists or other intelligent attackers without mentioning game theory. Why? One reason is that many adversarial situations—those that can be represented as attacker‐defender games, in which the defender first chooses an allocation of defensive resources to protect potential targets, and the attacker, knowing what the defender has done, then decides which targets to attack—can be modeled and analyzed successfully without using most of the concepts and terminology of game theory. However, risk analysis and game theory are also deeply complementary. Game‐theoretic analyses of conflicts require modeling the probable consequences of each choice of strategies by the players and assessing the expected utilities of these probable consequences. Decision and risk analysis methods are well suited to accomplish these tasks. Conversely, game‐theoretic formulations of attack‐defense conflicts (and other adversarial risks) can greatly improve upon some current risk analyses that attempt to model attacker decisions as random variables or uncertain attributes of targets (“threats”) and that seek to elicit their values from the defender's own experts. Game theory models that clarify the nature of the interacting decisions made by attackers and defenders and that distinguish clearly between strategic choices (decision nodes in a game tree) and random variables (chance nodes, not controlled by either attacker or defender) can produce more sensible and effective risk management recommendations for allocating defensive resources than current risk scoring models. Thus, risk analysis and game theory are (or should be) mutually reinforcing.     

Letters to the Editor

Letters to the editor will be confined to discussions of papers that have appeared in The American Statistician and to important issues facing the statistical community. Letters discussing papers in The American Statistician must be received within two months of publication of the paper; the author of the paper will then be given an opportunity to reply, and the letters and reply will be published together. All letters to the editor will be refereed. Corrections of errors that have been noted in papers published in The American Statistician will be listed as corrections at the end of this section.     

A Computer Program for Evaluating and Comparing Experimental Designs and Some Applications

For the past decade low cost computer graphics terminals have consisted almost exclusively of direct view storage tube (DVST) devices. Recently, however, a new variety of terminal based on “raster TV technology” has emerged which is competitive with storage tubes in price and readily supports variable intensity and color.

We summarize the basic organization of raster terminals so as to indicate their capabilities, discuss their advantages and disadvantages, and illustrate some of their uses in statistics and elsewhere. Because color is almost universally available on these terminals, we also discuss some elementary principles of color perception and use. An understanding of this material will help the data analyst present data effectively.     

A note on optimum linear feature extraction for gaussian populations with equal covariances and equal a priori probabilities

We consider the linear feature selection problem of obtaining a nonzero 1 × n matrix B which minimizes the probability of misclassification based on the Bayes decision rule applied to the random variable Y = BX, where X is a random n-vector arising from one of m Gaussian populations with equal covariances and equal apriori probabilities. It is shown that the optimal B satisfies a fixed point equation B = F(B) which can be solved by successive substitution.     

Maximum and Minimum Prediction Variance for Spherical and Cuboidal Split Plot Designs

The impact of restricted randomization on the information matrix has created challenges for the computation of design optimality criteria. This article focuses on the computation of the maximum and minimum prediction variance for Central Composite (CCD) and Box–Behnken (BBD) split plot designs (SPD). The approach is to analytically determine the exact maximum and minimum prediction variance for both spherical and cuboidal second-order SPD. A particular feature of these analytical functions is that they are functions of the design parameters. Finally, the application of these analytical functions is demonstrated for a CCD SPD.