Event based Background-Oriented Schlieren (TPAMI 2023) (5 min)

Project page: https://github.com/tub-rip/event_base...
PDF: https://doi.org/10.1109/TPAMI.2023.33...
Schlieren imaging is an optical technique to observe the flow of transparent media, such as air or water, without any particle seeding. However, conventional frame-based techniques require both high spatial and temporal resolution cameras, which impose bright illumination and expensive computation limitations. Event cameras offer potential advantages (high dynamic range, high temporal resolution, and data efficiency) to overcome such limitations due to their bio-inspired sensing principle. This paper presents a novel technique for perceiving air convection using events and frames by providing the first theoretical analysis that connects event data and schlieren. We formulate the problem as a variational optimization one combining the linearized event generation model with a physically-motivated parameterization that estimates the temporal derivative of the air density. The experiments with accurately aligned frame- and event camera data reveal that the proposed method enables event cameras to obtain on par results with existing frame-based optical flow techniques. Moreover, the proposed method works under dark conditions where frame-based schlieren fails, and also enables slow-motion analysis by leveraging the event camera’s advantages. Our work pioneers and opens a new stack of event camera applications, as we publish the source code as well as the first schlieren dataset with high-quality frame and event data.

Reference:
Shintaro Shiba, Friedhelm Hamann, Yoshimitsu Aoki, Guillermo Gallego
Event based Background-Oriented Schlieren,
IEEE Trans. Pattern Anal. Machine Intell. (TPAMI), 2023

Event-based Vision:
Research: https://sites.google.com/view/guiller...
Survey paper: https://arxiv.org/abs/1904.08405
Course at TU Berlin: https://sites.google.com/view/guiller...

Affiliation:
Shintaro Shiba and Yoshimitsu Aoki are with the Faculty of Science and Technology, Keio University, Kanagawa, Japan.
Friedhelm Hamann and Guillermo Gallego are with TU Berlin and the SCIoI Excellence Cluster, Berlin, Germany. G. Gallego is also with the Einstein Center Digital Future, Berlin, Germany.