SimMIM: A Simple Framework for Masked Image Modeling

November 2021

tl;dr: Large scale pretraining based on Masked Image Modeling. Similar to MAE.

Overall impression

This paper is published a week after MAE, obviously rushed by the publication of the latter. The ideas are very similar, but execution (hyperparameter tuning, paper writing) is considerably inferior to MAE.

Difference between MAE and SimMIM:

Similarities between MAE and SimMIM:

Key ideas

Technical details