A comparison of three weight elicitation methods: good, better, and best

This paper compares the properties and performance of three weight elicitation methods. It is in effect a “second round contest” in which the Bottomley et al. (2000) champion, direct rating (DR), locks horns with two new challengers. People using DR rate each attribute in turn on a scale of 0–100, whilst people using Max100 first assign to the most important attribute(s) a rating of 100, and then rate the other attributes relative to it/them. People using Min10 first assign the least important attribute(s) a rating of 10, and then rate the other attributes relative to it/them.The weights produced by Max100 were somewhat more test–retest reliable than DR. Both methods were considerably more reliable than Min10. Using people's test–retest data as attribute weights on simulated alternative values in a multi-attribute choice scenario, the same alternative would be chosen on 91% of occasions using Max100, 87% of occasions using DR, but only 75% of occasions using Min10. Moreover, the three methods are shown to have very distinct “signatures”, that is profiles relating weights to rank position. Finally, people actually preferred using Max100 and DR rather than Min10, an important pragmatic consideration.