PseudoLidarV3: End-to-End Pseudo-LiDAR for Image-Based 3D Object Detection

May 2020

tl;dr: End to end finetuning of depth prediction network with 3d object detector.

Overall impression

The paper proposed end to end method to train depth net and 3d object detector in pseudo-lidar. It is different from CMU’s end to end pseudo lidar approach, and proposed two methods to truly make the depth lifting differentiable.

The paper praised pseudo-lidar for its plug-and-play modularity but it also pointed out that PL suffers from the sub-optimal solution from proxy losses. Only 10% of point clouds belong to foreground (cf ForeSee, and PLv3 targets to finetune these points.

The sparsification is based on r, azimuth and elevation angle. A better approach may be Refined MPL.

The paper explored two representative lidar object detector, one one-stage method based on voxelization (PIXOR) and the other two-staged method based on point cloud directly (PointRCNN)

Although it will probably always be beneficial to include active sensors like LIDARs in addition to passive cameras, it seems possible that the benefit may soon be too small to justify large expense.

Key ideas

Technical details