Entropy Principles in Decision Making Under Risk

This paper considers decision problems where: (1) The exact probability distribution over the states of nature is not precisely known, but certain prior information is available about the possibilities of these outcomes; (2) A prior distribution over the states of nature is known, but new constraint information about the probabilities becomes available. The maximum entropy principle asserts that the probability distribution with maximum entropy, satisfying the prior knowledge, should be used in the decision problem. The minimum cross-entropy principle says that the posterior distribution is the one which minimizes cross-entropy, subject to the new constraint information. The entropy principles have not gone uncriticized, and this literature, together with that justifying the principles, is surveyed. Both principles are illustrated in a number of situations where the distribution is either discrete or continuous. The discrete distribution case with prior interval estimates based on expert opinions is considered in detail.