Limited role of entropy in information economics

Information transmitted is defined as the amount by which added evidence (or message received) diminishes uncertainty. The latter is characterized by some properties intuitively suggested by this word and possessed by conditional entropy, a parameter of the posterior probability distribution. However, conditional entropy shares these properties with some other concave symmetric functions on the probability space.Moreover, a given transmission channel (or, in the context of statistical inference, a given experiment) yields a higher maximum expected benefit than anotherto any user if and only ifall concave functions of the posterior probability vector have higher values for the former channel (or experiment). Hence one information system (channel, experiment) may be preferable to another for a given user although its transmission rate, in entropy terms, is lower.But only entropy has the economically relevant property of measuring, in the limit, the expected length of efficiently coded messages sent in long sequences. Thus, while irrelevant to the value (maximum expected benefit) of an information system and to the costs of observing, estimating, and deciding, entropy formulas are indeed relevant to the cost of communicating, i.e., of storing, coding and transmitting messages.Revised version of a paper presented at the 5th Colloquium on Optimization Techniques (International Federation for Information Processing, Rome, May 1973), Lecture Notes series of the Springer Verlag, Heidelberg and New York. The author wishes to thank Springer Verlag for reprint permission. Acknowledgements of support are due to the Alexander von Humboldt Foundation (Special Program for the Cooperation of Research Institutes of F.R.G. and U.S.A.), the U.S. Office of Naval Research and the Institute for Economic Theory, University of Bonn.