Scoring Rules and the Evaluation of Probability Assessors

Abstract

The personalistic theory of probability prescribes that personal probability assessments to be used in decision-making situations should correspond with the assessor's judgments. A payoff function which depends on the assessor's stated probabilities and on the event which actually occurs may be used (1) to keep the assessor honest or (2) to evaluate the assessor. It is shown that with the exception of a logarithmic payoff function, these two uses of payoff functions for assessors are not compatible. This conflict is explained in terms of the differences in the situations facing the assessor and the evaluator (the user of the probabilistic predictions).