Visual-Inertial Sensor Calibration -- A Complete Tutorial and Discussion

This video walks through the process of performing visual-inertial sensor calibration. This calibration is crucial for downstream applications which try to fuse the two sources of information. The video was recorded in a single session from start to finish, so please use the chapters to skip to the sections which are of interest. The sensor used is the Intel Realsense D455 color camera and internal IMU. The key software used is Kalibr and allan_variance_ros.



00:00 - Introduction
01:17 - What is calibration?
06:25 - Dataset
10:00 - Multi-camera calibration
20:08 - IMU noise recovery
27:33 - Camera-IMU calibration
42:26 - Summary



Key software we are going to use:
kalibr -- https://github.com/ethz-asl/kalibr
allan_variance_ros -- https://github.com/ori-drs/allan_vari...

Calibration slides (slide 51):
https://pgeneva.com/downloads/notes/2...

Self promotion of visual-inertial state estimation
https://github.com/rpng/open_vins/
https://docs.openvins.com/gs-calibrat...

Ensure you have installed and built both packages
https://github.com/ethz-asl/kalibr/wi...

We are going to be using the calibration from here (d455 realsense)
https://github.com/rpng/ar_table_data...

Multi-camera calibration guide:
https://github.com/ethz-asl/kalibr/wi...
https://github.com/ethz-asl/kalibr/wi...

How to calibrate the IMU intrinsics? What is the IMU noise model?
https://github.com/ethz-asl/kalibr/wi...
https://github.com/ori-drs/allan_vari...

Lets use in kalibr to calibrate the IMU and camera jointly:
https://github.com/ethz-asl/kalibr/wi...

When collecting data need to avoid degenerate directions (e.g. planar).
Excite two axis motion with non-constant accelerations is crucial!
https://pgeneva.com/downloads/preprin...
https://pgeneva.com/downloads/preprin...

The different IMU intrinsic models are covered in here:
https://timohinzmann.com/publications...

How to interpret results?
https://docs.openvins.com/gs-calibrat...
- Inspect the IMU time dt plot carefully in the PDF report!
- Your accelerometer and gyroscope errors are within their 3-sigma bounds (if not then your IMU noise or the dataset are incorrect)
- Ensure that your estimated biases do not leave your 3-sigma bounds. If they do leave then your trajectory was too dynamic, or your noise values are not good.
- Sanity check your final rotation and translation with hand-measured values.

Example of bad IMU timestamps:
https://github.com/ethz-asl/kalibr/pu...