Learning Robot Control: From RL to Differential Simulation - (PhD Defense of Yunlong Song)

This thesis focuses on Learning Robot Control by integrating deep reinforcement learning (RL) and model-based control methods. It aims to develop advanced control methods that bridge the gap between data-driven learning and model-based control. The proposed methods enhance robot agility and robustness in real-world applications.
Key contributions are:
- Show that RL outperforms Optimal Control in autonomous racing because it directly optimizes a non-differentiable task-level objective.
- Propose a policy-search-for-model-predictive-control (MPC) framework, combining RL's ability to optimize high-level task objectives with MPC's precise actuation and constraint handling.
- Introduce a differentiable simulation framework to leverage robot dynamics for more stable and - efficient policy training.
- Develop a high-performance drone racing system outperforming optimal control methods and professional pilots.
- Develop Flightmare, a flexible modular quadrotor simulator for reinforcement learning and vision-based flight.

OUTLINE:
00:00 - Introduction
02:37 - Robot Control: An Optimal Control Perspective
03:14 - Robot Control: A Reinforcement Learning Perspective
05:06 - Project 1: Autonomous Drone Racing: Optimal Control vs. Reinforcement Learning
12:05 - Project 2: Flying Through Dynamic Gates: Reinforcement Learning for Optimal Control
16:04 - Project 3: Quadrupedal Locomotion: Differentiable Simulation
20:18 - Conclusions
23:05 - One More Thing