Скачать или смотреть TransFuser Demo Video

This video demonstrates an autonomous driving system using the TransFuser (Transformer-based Sensor Fusion) model within the CARLA Simulator 0.9.11 environment. Beyond standard autonomous navigation, this project focuses on Explainable AI (XAI). I implemented a custom interface that not only visualizes real-time sensor data from LiDAR and RGB cameras but also explicitly displays the reasoning behind the AI's decisions—such as why it chooses to stop or slow down—to provide a transparent view of the driving logic.

The core of this implementation is based on the TransFuser algorithm, which fuses RGB camera images with LiDAR point clouds to predict optimal waypoints and control vehicle steering and velocity. To enhance system transparency, an Explainable AI (XAI) module was developed to analyze and display the specific causes for vehicle actions in real-time, such as detecting red traffic lights, pedestrians, or stop signs. Furthermore, a comprehensive multi-view monitoring system was integrated into the UI. This includes a status HUD for vehicle telemetry, a third-person AR view with 3D bounding boxes, a raw front-facing camera feed used for model input, and a top-down LiDAR point cloud visualization, allowing viewers to observe the sensor fusion process at a glance.