3D Deep Learning In Function Space

Michael Niemeyer's NVIDIA GTC 2020 presentation on implicit models for 3D reconstruction and differentiable volumetric rendering (DVR) for learning implicit representations from RGB images alone.

Abstract:

Recent advances in GPU technology and scalable algorithms have led to breakthroughs in deep learning. In particular, convolutional neural networks (CNNs) achieve state-of-the-art results in longstanding vision problems, such as image classification or object detection. However, autonomous agents that navigate and interact in our world need to reason in 3D. Unlike images in the 2D case, it is not clear how to represent 3D geometry and how to make it amenable for deep-learning techniques. We'll introduce our approach to learning 3D representations in function space. First, we'll show how this approach can represent arbitrary topologies without discretization at fixed memory cost. Then we'll extend this framework to learning to predict not only the shape of an object, but also its texture and motion. Finally, we'll show how we can learn implicit representations from 2D RGB images without 3D supervision.

Blog:
https://autonomousvision.github.io/di...

Code:
https://github.com/autonomousvision/d...

Paper:
http://www.cvlibs.net/publications/Ni...
http://www.cvlibs.net/publications/Me...
http://www.cvlibs.net/publications/Oe...
http://www.cvlibs.net/publications/Ni...