Conditional probability and improper priors

ABSTRACT

According to Jeffreys improper priors are needed to get the Bayesian machine up and running. This may be disputed, but usage of improper priors flourish. Arguments based on symmetry or information theoretic reference analysis can be most convincing in concrete cases. The foundations of statistics as usually formulated rely on the axioms of a probability space, or alternative information theoretic axioms that imply the axioms of a probability space. These axioms do not include improper laws, but this is typically ignored in papers that consider improper priors.

The purpose of this paper is to present a mathematical theory that can be used as a foundation for statistics that include improper priors. This theory includes improper laws in the initial axioms and has in particular Bayes theorem as a consequence. Another consequence is that some of the usual calculation rules are modified. This is important in relation to common statistical practice which usually include improper priors, but tends to use unaltered calculation rules. In some cases, the results are valid, but in other cases inconsistencies may appear. The famous marginalization paradoxes exemplify this latter case.

An alternative mathematical theory for the foundations of statistics can be formulated in terms of conditional probability spaces. In this case, the appearance of improper laws is a consequence of the theory. It is proved here that the resulting mathematical structures for the two theories are equivalent. The conclusion is that the choice of the first or the second formulation for the initial axioms can be considered a matter of personal preference. Readers that initially have concerns regarding improper priors can possibly be more open toward a formulation of the initial axioms in terms of conditional probabilities. The interpretation of an improper law is given by the corresponding conditional probabilities.