PointTrack: Segment as Points for Efficient Online Multi-Object Tracking and Segmentation

July 2020

tl;dr: Use PointNet to extract embedding vectors from instance segmentation masks for tracking.

Overall impression

This work tackles the newly created track of MOTS (multiple object tracking and segmentation) CVPR 2019. It uses tracking-by-segmentation. It uses existing work of instance segmentation, and the main contribution lies in the association/tracking part.

PointTrack uses a single stage instance segmentation method with a seeding location. This makes it compatible with many instance segmentation method, such as CenterMask or BlendMask.

deepSORT extracts features from image patches, while PointTrack extracts features from 2D point cloud.

Key ideas

Technical details