Скачать или смотреть Frame Averaging for Invariant and Equivariant Network Design | Omri Puny

Join the Learning on Graphs and Geometry Reading Group: https://hannes-stark.com/logag-readin...

Paper “Frame Averaging for Invariant and Equivariant Network Design": https://arxiv.org/abs/2110.03336

Abstract: Many machine learning tasks involve learning functions that are known to be invariant or equivariant to certain symmetries of the input data. However, it is often challenging to design neural network architectures that respect these symmetries while being expressive and computationally efficient. For example, Euclidean motion invariant/equivariant graph or point cloud neural networks. We introduce Frame Averaging (FA), a general purpose and systematic framework for adapting known (backbone) architectures to become invariant or equivariant to new symmetry types. Our framework builds on the well known group averaging operator that guarantees invariance or equivariance but is intractable. In contrast, we observe that for many important classes of symmetries, this operator can be replaced with an averaging operator over a small subset of the group elements, called a frame. We show that averaging over a frame guarantees exact invariance or equivariance while often being much simpler to compute than averaging over the entire group. Furthermore, we prove that FA-based models have maximal expressive power in a broad setting and in general preserve the expressive power of their backbone architectures. Using frame averaging, we propose a new class of universal Graph Neural Networks (GNNs), universal Euclidean motion invariant point cloud networks, and Euclidean motion invariant Message Passing (MP) GNNs. We demonstrate the practical effectiveness of FA on several applications including point cloud normal estimation, beyond 2-WL graph separation, and n-body dynamics prediction, achieving state-of-the-art results in all of these benchmarks.

Authors: Omri Puny, Matan Atzmon, Heli Ben-Hamu, Ishan Misra, Aditya Grover, Edward J. Smith, Yaron Lipman

Twitter Hannes:   / hannesstaerk  
Twitter Dominique:   / dom_beaini  
Twitter Valence Discovery:   / valence_ai  

Reading Group Slack: https://join.slack.com/t/logag/shared...

~

Chapters

00:00 Intro
02:52 - Statistical Learnings and Symmetries
09:35 - Equivariance
17:00 - Parameter Sharing Models
32:09 - What is a Frame?
39:26 - Frame Examples & Discussion
1:02:22 - n-body
1:14:19 - Conclusion
1:15:50 - Q&A