SeaThru-NeRF: Neural Radiance Fields in Scattering Media (CVPR 2023) - Curacao Scene

This is a result from our CVPR 2023 paper:
"SeaThru-NeRF: Neural Radiance Fields in Scattering Media"' , Deborah Levy, Amit Peleg , Naama Pearl, Dan Roesnbaum, Derya Akkaynak, Simon Korman, Tali Treibitz.
https://sea-thru-nerf.github.io/
https://arxiv.org/abs/2304.07743

Research on neural radiance fields (NeRFs) for novel view generation is
exploding with new models and extensions. However, a question that remains unanswered is what happens in underwater or foggy scenes where the medium strongly influences the appearance of objects? Thus far, NeRF and its variants have ignored these cases. However, since the NeRF framework is based on volumetric rendering, it has inherent capability to account for the medium's effects, once modeled appropriately. We develop a new rendering model for NeRFs in scattering media, which is based on the SeaThru image formation model, and suggest a suitable architecture for learning both scene information and medium parameters. We demonstrate the strength of our method using simulated and real-world scenes with diverse optical properties on several tasks, correctly rendering novel photorealistic views underwater. Even more excitingly, we can render clear views of these scenes, removing the interfering medium between the camera and the scene and reconstructing the appearance and depth of further objects, which are severely occluded by the medium. We make our code and unique dataset available and discuss the promise and limitations of NeRFs in scattering media.

More related research:
https://www.viseaon.haifa.ac.il/
https://www.deryaakkaynak.com/