SIPTA Seminar- E. Hullërmeier:The Challenge of Quantifying Epistemic Uncertainty in Machine Learning

ABSTRACT: Due to the growing relevance of machine learning for real-world applications, many of which are coming with safety requirements, the notion of uncertainty has received increasing attention in machine learning research in the recent past. This talk will address questions regarding the adequate representation and quantification of (predictive) uncertainty in (supervised) machine learning and elaborate on the distinction between two important types of uncertainty, often referred to as aleatoric and epistemic. Roughly speaking, while aleatoric uncertainty is due to the randomness inherent in the data-generating process, epistemic uncertainty is caused by the learner’s lack of knowledge about this process. Bayesian methods are commonly used to quantify both types of uncertainty, but alternative approaches have become popular in recent years, notably so-called evidential deep learning methods that are based on the idea of second-order loss minimisation. By exploring some conceptual and theoretical issues of such approaches, the challenging nature of quantifying epistemic uncertainty will be highlighted. The talk concludes with a brief discussion of the role of credal sets in uncertainty quantification and, more generally, the potential of a “credal” approach to machine learning.

This talk is part of a series of seminars on imprecise probabilities that are organized by SIPTA, the "Society for Imprecise Probabilities: Theories and Applications". We also organize conferences and schools, provide documentation and maintain a mailing list and blog. More information is available at http://sipta.org. Info on the SIPTA seminars in particular is available at http://sipta.org/events/sipta-seminars

Contents
00:00 - Start
07:21 - Introduction
22:47 - Second order loss minimisation
40:26 - Uncertainty quantification
57:36 - Conclusion