AP vs MR vs F1 in object detection

October 2020

tl;dr: Summarizes the different metrics used in general object detection vs pedestrian detection.

Overall impression

Both AP and MR are used heavily in academia in evaluating object detection algorithms, just like AUC of ROC for classifier. But in deployment, neither MR or AP is a good metric, as we have to select one working point. The good old F1 score may be better.

In other words, mAP is used to evaluate detection algorithms, and acc (or F1 score) is used to evaluate detectors in specific scenarios. The former is used in academia and papers, and the latter is used in industry and production.

Key ideas


Papers on pedestrian detection (in a crowd)