Analysis and Calibration of Neural Networks for Skin Cancers Diagnose

This report is for the term project of HKUST COMP 5212 Machine Learning Spring 2021.

The preliminary diagnose of skin cancers relies on the visual appearance of the pigmented lesion of skin. Many researchers have studied using deep learning to classify different types of skin diseases from the images.

However, the reliability of the deep learning models is a big concern for their application in skin disease diagnose.

In this project, we conduct experiments on a skin cancers classification dataset (HAM10000 dataset) and MNIST, and find that the confidence values produced by the Softmax layer of deep learning models are not reliable (mis-calibrated).

We study and experiment on five calibration methods and find that these calibration methods can help reduce the miscalibration issue of the model.