PointPainting: Sequential Fusion for 3D Object Detection

December 2020

tl;dr: Augment point cloud with semantic segmentation results

Overall impression

The paper proposes a general method to fuse image results with lidar point cloud. This leads to minimum change to existing lidar detection networks and gives decent boosts to performance.

Lidar and image are complementary: the point cloud provides accurate range view but with low resolution and texture info. The image has inherent depth ambiguity but offers fine-grained texture and color information. The addition of image info with lidar should boost lidar performance.

The paper has a nice summary of previous camera-lidar early fusion methods and why they do not work well.

Key ideas

Technical details