Hypotheses Testing as a Fuzzy Set Estimation Problem

For many scientific experiments computing a p-value is the standard method for reporting the outcome. It is a simple way of summarizing the information in the data. One theoretical justification for p-values is the Neyman-Pearson theory of hypotheses testing. However, the decision making focus of this theory does not correspond well with the desire, in most scientific experiments, for a simple and easily interpretable summary of the data. Fuzzy set theory with its notion of a membership function gives a non-probabilistic way to talk about uncertainty. Here, we argue that for some situations, where a p-value is computed, it may make more sense to formulate the question as one of estimating a membership function of the subset of special parameter points which are of particular interest for the experiment. Choosing the appropriate membership function can be more difficult than specifying the null and alternative hypotheses but the resulting payoff is greater. This is because a membership function can better represent the shades of desirability among the parameter points than the sharp division of the parameter space into the null and alternative hypotheses. This approach yields an estimate which is easy to interpret and more flexible and informative than the cruder p-value.