MedAI

Title: EHRXQA: A Multi-Modal Question Answering Dataset for Electronic Health Records with Chest X-ray Images

Speaker: Seongsu Bae

Abstract:
Electronic Health Records (EHRs) are rich in clinical information, presented in a multi-modal format. Developing a question answering (QA) system to interact with EHRs can significantly aid clinicians in making informed decisions, assist researchers in conducting precise studies, and enhance hospital operational efficiency. Current research mainly focuses on uni-modal QA, such as table-based or image-based QA. This leaves a gap in multi-modal QA, a largely underexplored area. In this talk, I will introduce EHRXQA, the first multi-modal clinical QA dataset requiring reasoning over both tables and images. I will detail the creation of two datasets: a new X-ray-based QA dataset utilizing MIMIC-CXR and Chest ImaGenome, and a table-based QA dataset from MIMIC-IV using EHRSQL, where the two are combined to create EHRXQA. Additionally, I will discuss the potential of NeuralSQL, which integrates SQL with external function calls, in addressing multi-modal questions.

Speaker Bio:
Seongsu Bae is a second-year Ph.D. student at KAIST's Kim Jaechul Graduate School of AI, advised by Prof. Edward Choi. His research focuses on multi-modal learning, healthcare AI, and the evaluation and benchmarking of AI systems for deployment. Seongsu is dedicated to developing and assessing AI models that understand multi-modal Electronic Health Records (EHRs), including both structured EHRs and medical images/reports. Recently, he completed an internship at Microsoft Research Asia (MSRA) in 2023, focused on multi-modal question answering and generative models in the healthcare domain.

------

The MedAI Group Exchange Sessions are a platform where we can critically examine key topics in AI and medicine, generate fresh ideas and discussion around their intersection and most importantly, learn from each other.

We will be having weekly sessions where invited speakers will give a talk presenting their work followed by an interactive discussion and Q&A.

Our sessions are held every Monday from 1pm-2pm PST.

To get notifications about upcoming sessions, please join our mailing list: https://mailman.stanford.edu/mailman/...

For more details about MedAI, check out our website: https://medai.stanford.edu. You can follow us on Twitter @MedaiStanford

Organized by members of the Rubin Lab (http://rubinlab.stanford.edu) and Machine Intelligence in Medicine and Imaging (MI-2) Lab:
- Nandita Bhaskhar (https://www.stanford.edu/~nanbhas)
- Amara Tariq (  / amara-tariq-475815158  )