Event Penguins: Low-power, Continuous Remote Behavioral Localization with Event Cameras (CVPR 2024)

Project page: https://tub-rip.github.io/eventpenguins/
https://www.scienceofintelligence.de/...
Researchers in natural science need reliable methods for quantifying animal behavior. Recently, numerous computer vision methods emerged to automate the process. However, observing wild species at remote locations remains a challenging task due to difficult lighting conditions and constraints on power supply and data storage. Event cameras offer unique advantages for battery-dependent remote monitoring due to their low power consumption and high dynamic range capabilities. We use this novel sensor to quantify a behavior in Chinstrap penguins called ecstatic display. We formulate the problem as a temporal action detection task, determining the start and end times of the behavior. For this purpose, we recorded a colony of breeding penguins in Antarctica during several weeks and labeled event data on 16 nests. The developed method consists of a generator of candidate time intervals (proposals) and a classifier of the actions within them. The experiments show that the event cameras' natural response to motion is effective for continuous behavior monitoring and detection, reaching a mean average precision (mAP) of 58% (which increases to 63% in good weather conditions). The results also demonstrate the robustness against various lighting conditions contained in the challenging dataset. The low-power capabilities of the event camera allows to record three times longer than with a conventional camera.

Reference:
F. Hamann, S. Ghosh, I. Juarez Martinez, T. Hart, A. Kacelnik, G. Gallego,
Low-power, Continuous Remote Behavioral Localization with Event Cameras,
IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2024.

Affiliations:
Technical University of Berlin, Oxford University, Oxford Brookes University.
Science of Intelligence Excellence Cluster, Berlin, Germany.
https://www.scienceofintelligence.de/

Event-based Vision:
Research: https://sites.google.com/view/guiller...
Survey paper: https://arxiv.org/abs/1904.08405
Course at TU Berlin: https://sites.google.com/view/guiller...