QuantFormer: Learning to quantize for forecasting neural responses in two-photon calcium imaging

Salvatore Calcagno (Università di Catania, FAIR Spoke 10 - Bio-socio-cognitive AI) presents "QuantFormer: Learning to quantize for forecasting neural responses in two-photon calcium imaging".

This presentation is part of the Virtual Young Poster Session of the FAIR 2024 General Conference.
For more information: https://fondazione-fair.it/general-co...

Understanding complex animal behaviors hinges on deciphering the intricate neural activities within specific brain circuits. Two-photon imaging emerges as a powerful tool, offering significant insights into the dynamics
of neuronal ensembles. In this context, forecasting neural activities is crucial for neuroscientists to create mathematical models of brain dynamics. Existing transformer-based methods, while effective in many
domains, struggle to capture the distinctiveness of neural signals characterized by spatiotemporal sparsity and intricate dependencies. We present QuantFormer, a novel transformer-based model designed for forecasting neural activity in two-photon calcium imaging data. Unlike traditional regression-based approaches, ours reframes the forecasting task as a classification problem through dynamic signal quantization, enabling better learning of sparse activity patterns. Additionally, our method addresses the challenge of analyzing multivariate signals with an arbitrary number of neurons by using specialized neuron prompts. Leveraging unsupervised quantization training on the Allen dataset, the largest publicly available dataset of two-photon calcium imaging, we establish a new benchmark in mouse neural forecasting. It provides robustness and generalization across individuals and stimuli variations, thus defining the route towards a robust foundation model of the mouse visual cortex.