Скачать или смотреть Monte Carlo Seminar|Max Welling| A Variational Bayesian Method for AR and Recurrent Models

Organizer: Online Monte Carlo Seminar ‪@MonteCarloSeminar‬
[sites.google.com/view/monte-carlo-seminar]
Speaker: Max Welling (University of Amsterdam and CuspAI)
Title: A Variational Bayesian Method for Autoregressive and Recurrent Models
Abstract: Autoregressive models are the key technology behind most of today's AI revolution: from language models such as ChatGPT, molecular generative models such as LSTM, to PDE emulator models based on autoregressive graph neural networks. While these models work incredibly well, they lack the ability to assess the confidence in these predictions, especially when they are asked to predict outside of the training regime. As such we need to urgently develop models that have accurate uncertainty quantification built in. In this work we propose BARNN, a variational Bayesian method for Autoregressive and Recurrent Neural Networks. We apply the idea of local reparameterization (or variable dropout) to sequence models and choose a prior that generalizes the "VampPrior" to sequence models. Together this results in a scalable approximate Bayesian method that achieves SOTA on both prediction accuracy and uncertainty prediction. We test the method on a number of tasks in AI4Science such as PDE surrogate models and molecule generation.

#machinelearning #aiscience #physics #uncertainty