Simulated Transient Images for Non-Line-of-Sight Imaging and Recognition
Wenzheng Chen (graduate student, University of Toronto), Fangyin Wei (graduate student), Kyros Kutulakos (faculty, University of Toronto), Szymon Rusinkiewicz (faculty), Felix Heide (faculty)
Objects hidden by occluders (other objects), are considered lost in the images acquired by conventional camera systems, making it impossible to see and understand such hidden objects. Non-line-of-sight (NLOS) methods aim to recover information about hidden scenes, which could help make medical imaging less invasive, improve the safety of autonomous vehicles, and potentially enable capturing unprecedented high-definition RGB-D (red, green, blue-depth) data sets that include geometry beyond the directly visible parts.
In the image and videos, we show transient images rendered by our rendering pipeline using hardware-accelerated rasterization to create a 3D image. In the image, our approach renders a car model. These transient images are part of our results published in ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2020) called “Learned Feature Embeddings for Non-Line-of-Sight Imaging and Recognition.” In this paper, we proposed a method that leverages physical models to learn hidden scene feature representations tailored to both reconstruction and recognition tasks such as classification or object detection. Our method, trained on our simulated transient images, can generalize to unseen classes and unseen real-world scenes.