Uncovering bias in order assignment

To mitigate sequencing effects in decision-making, many situations require a set of items to be considered in a random order. When such orderings are repeated, one can test whether randomization indeed obtains, or whether some orderings have been manipulated in order to achieve a favorable result. This paper articulates the key features of this problem and presents three general tests for randomness. These methods are used to analyze the order in which lottery numbers are drawn in Powerball, contestants perform on American Idol, and candidates are placed on election ballots. This last application features frequent manipulation, with potentially serious consequences.