Deriving the expected utility of a predictive model when the utilities are uncertain

Cooper GF, Visweswaran S. Deriving the expected utility of a predictive model when the utilities are uncertain. In: Proceedings of the Annual Symposium of the American Medical Informatics Association (2005) 161-165.  PMID:  16779022

Predictive models are often constructed from clinical databases with the goal of eventually helping make better clinical decisions. Evaluating models using decision theory is therefore natural. When constructing a model using statistical and machine learning methods, however, we are often uncertain about precisely how a model will be used. Thus, decisionindependent measures of classification performance, such as the area under an ROC curve, are popular. As a complementary method of evaluation, we investigate techniques for deriving the expected utility of a model under uncertainty about the model's utilities. We demonstrate an example of the application of this approach to the evaluation of two models that diagnose coronary artery disease.

Publication Year: 
2005
Faculty Author: 
Publication Credits: 
Cooper GF, Visweswaran S.
AttachmentSize
PDF icon Cooper Visweswaran.pdf85.56 KB
^