Self-Mono-SF: Self-Supervised Monocular Scene Flow Estimation

June 2020

tl;dr: Self supervised estimation of depth and 3D scene flow (and as a side product, 2D optical flow).

Overall impression

Scene flow estimation is the 3D version of optical flow. It requires depth estimation as well. It is a natural idea to extend the monocular prediction of optical flow and depth to scene flow.

Optical flow is the 2D projection of a 3D point and its 3D scene flow. The scene flow is defined as 3D motion wrt the camera.

The architecture is heavily based on PWC Net for optical flow estimation.

Key ideas

Technical details