DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image

June 2019

tl;dr: Depth completion from a single RGB image and sparse depth (from lidar and low-cost lidars or one-line lidars).

Overall impression

The idea should benefit similar projects for radar+camera sensor fusion. The main idea is to take a sparse but accurate depth from a low-cost lidar and make it dense with the help of an aligned color image. However the paper only explored uniform sparsity, not structured sparsity (missing lines of depth completely) which is perhaps more relevant for cheap lidars.

The paper heavily builds on the indoor depth completion paper. It is in a sense color guided depth impainting.

Key ideas

Technical details