Bridging the Sim2real Gap in Robotics with Marius Memmel - 695

Today, we're joined by Marius Memmel, a PhD student at the University of Washington, to discuss his research on sim-to-real transfer approaches for developing autonomous robotic agents in unstructured environments. Our conversation focuses on his recent ASID - https://arxiv.org/abs/2404.12308 and URDFormer - https://arxiv.org/abs/2405.11656 papers. We explore the complexities presented by real-world settings like a cluttered kitchen, data acquisition challenges for training robust models, the importance of simulation, and the challenge of bridging the sim2real gap in robotics. Marius introduces ASID, a framework designed to enable robots to autonomously generate and refine simulation models to improve sim-to-real transfer. We discuss the role of Fisher information as a metric for trajectory sensitivity to physical parameters and the importance of exploration and exploitation phases in robot learning. Additionally, we cover URDFormer, a transformer-based model that generates URDF documents for scene and object reconstruction to create realistic simulation environments.

🎧 / 🎥 Listen or watch the full episode on our page: https://twimlai.com/go/695.

🔔 Subscribe to our channel for more great content just like this: https://youtube.com/twimlai?sub_confi...


🗣️ CONNECT WITH US!
===============================
Subscribe to the TWIML AI Podcast: https://twimlai.com/podcast/twimlai/
Follow us on Twitter:   / twimlai  
Follow us on LinkedIn:   / twimlai  
Join our Slack Community: https://twimlai.com/community/
Subscribe to our newsletter: https://twimlai.com/newsletter/
Want to get in touch? Send us a message: https://twimlai.com/contact/


📖 CHAPTERS
===============================
00:00 - Introduction
2:54 - Complexity of robotics in unstructured environments
7:05 - Sim-to-real
9:48 - ASID
20:45 - Fisher information
31:44 - Fisher information as a general purpose reward function
37:47 - URDFormer paper
41:55 - Text to image generation in URDFormer paper
48:35 - URDFormer model
49:20 - URDFormer model challenges
54:24 - Future directions


🔗 LINKS & RESOURCES
===============================
ASID: Active Exploration for System Identification in Robotic Manipulation - https://arxiv.org/abs/2404.12308
URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Images - https://arxiv.org/abs/2405.11656
Applying RL to Real-World Robotics with Abhishek Gupta - 466 - https://twimlai.com/podcast/twimlai/a...


📸 Camera: https://amzn.to/3TQ3zsg
🎙️Microphone: https://amzn.to/3t5zXeV
🚦Lights: https://amzn.to/3TQlX49
🎛️ Audio Interface: https://amzn.to/3TVFAIq
🎚️ Stream Deck: https://amzn.to/3zzm7F5