Binary classifier calibration using a Bayesian non-parametric approach

Naeini MP, Cooper GF, Hauskrecht M. Binary classifier calibration using a Bayesian non-parametric approach. In: Proceedings of the SIAM International Conference on Data Mining (2015).

Learning probabilistic predictive models that are well calibrated is critical for many prediction and decision-making tasks in Data mining. This paper presents two new non-parametric methods for calibrating outputs of binary classification models: a method based on the Bayes optimal selection and a method based on the Bayesian model averaging. The advantage of these methods is that they are independent of the algorithm used to learn a predictive model, and they can be applied in a post-processing step, after the model is learned. This makes them applicable to a wide variety of machine learning models and methods. These calibration methods, as well as other methods, are tested on a variety of datasets in terms of both discrimination and calibration performance. The results show the methods either outperform or are comparable in performance to the state-of-the-art calibration methods.

Read More: http://epubs.siam.org/doi/10.1137/1.9781611974010.24
Publication Year: 
2015
Faculty Author: 
Publication Credits: 
Naeini MP, Cooper GF, Hauskrecht M.
AttachmentSize
PDF icon Naeini_Binary classifier.pdf267.54 KB
^