Stanford Seminar - Connecting Robotics and Foundation Models, Brian Ichter of Google DeepMind

May 5, 2023
Brian Ichter of Google Brain / DeepMind

Foundation models can encode a wealth of semantic knowledge about the world, but can be limited by their lack of interactive, real-world experience. This poses a challenge for leveraging them in robotics, which requires interactive decision making and reasoning for a given embodiment. This talk will discuss several research directions towards addressing these challenges, from grounding them in their environment (SayCan, InnerMonologue, Grounded Decoding, NL-Maps), to directly outputting grounded code (Code as Policies), and finally training them with embodied robotics data (PaLM-E, RT-1).

About the speaker:
Brian is a Research Scientist with Google Brain (or Deepmind now) where he works on the Robotics team. Generally, his research interests lie in enabling mobile robotic systems to perform complex, long-horizon tasks in real-world environments through machine learning -- and in the last couple years, foundation models. Prior to joining Google Brain, he received a PhD and MS in Aeronautics and Astronautics from Stanford University under the supervision of Prof. Marco Pavone.

#robotics #deepmind