Скачать или смотреть UofT Robotics: Mac Schwager - Reimagining Robot Autonomy w/ Neural Environment Representations

Title: Reimagining Robot Autonomy w/ Neural Environment Representations

Abstract: New developments in computer vision and deep learning powered by next-generation GPUs have led to the rise of neural environment representations: 3D maps that are stored as deep networks that
spatially register occupancy, color, texture, and other physical properties. These environment models can generate photo-realistic synthetic images from unseen view points, and can store 3D informationin exquisite detail. In this talk, I investigate the question: How can robots use neural environment representations for perception, motion planning, manipulation, and simulation? I will show recent
work from my lab in which we build a robot navigation pipeline using a Neural Radiance Field (NeRF) map of an environment. We develop a trajectory optimization algorithm that interfaces with the NeRF model
to find dynamically feasible, collision-free trajectories for a robot moving through a NeRF world. We also develop an optimization-based state estimator that uses the NeRF model to give full dynamic state
estimates for a robot from only on board images. I will discuss our algorithms for dexterous manipulation using NeRF object models, and will describe our development a differentiable physics simulator that
operates directly on NeRF object models. I will conclude with future opportunities and challenges in integrating neural environment representations into the robot autonomy stack.

Bio: Mac Schwager is an Associate Professor of Aeronautics and Astronautics at Stanford University. He directs the Multi-robot Systems Lab (MSL) where he studies distributed algorithms for control, perception, and learning in groups of robots and autonomous systems. He is interested in a range of applications including cooperative surveillance with teams of UAVs, autonomous driving in traffic, cooperative robotic
manipulation, distributed SLAM, distributed trajectory planning, and autonomous drone and car racing. He obtained his BS degree from Stanford, and his MS and PhD degrees from MIT. He was a postdoctoral
researcher at the University of Pennsylvania and at MIT. Prior to joining Stanford, he was an assistant professor at Boston University from 2012 to 2015. He received the NSF CAREER award in 2014, the
DARPA YFA in 2018, and has received numerous best paper awards in conferences and journals including the IEEE Transactions on Robotics best paper award in 2016, the Best Paper Award in Robot Manipulation
in ICRA 2018, and the Best Paper Award in Multi-Robot Systems in ICRA 2020.