Скачать или смотреть [NeurIPS'24] Beyond Accuracy: Ensuring Correct Predictions with Correct Rationales

In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS), 2024.
Paper: https://arxiv.org/pdf/2411.00132

Authors:
Tang Li https://tangli0305.github.io/
Mengmeng Ma https://www.mengmenm.top/
Xi Peng https://sites.google.com/site/xipengc...

Deep-REAL Lab: https://deep-real.github.io/

Keyworks:
NeurIPS 2024, Deep Learning, Machine Learning, Explainable AI, Explainable Machine Learning, Computer Vision, Vision-Language Models, Foundation Models.

Abstract:
Large pretrained foundation models exhibit outstanding performance or even surpass human experts in some high-stakes applications. However, most of these models are currently evaluated primarily on prediction accuracy, overlooking the validity of the rationales behind their accurate predictions. There exists a pressing need to ensure the dual-correctness of predictions, i.e., correct prediction for correct rationales, for the safe deployment of foundation models. To this end, we propose a two-phase scheme for developing dual-correct predictions. We first curate a new dataset by representing prediction rationale in a machine-readable format, then leverage the relations between rationale to guide the learning of models without manual annotations. The extensive experiments and ablation studies show that our model outperforms the state-of-the-art and fine-tuned models in a wide range of tasks including classification and retrieval. Furthermore, our method significantly improves the model's rationale correctness in terms of rationale localization and disentanglement.