ICCV 2023 – Diffusion Models as Masked Autoencoders

In this episode we discuss Diffusion Models as Masked Autoencoders
by Chen Wei, Karttikeya Mangalam, Po-Yao Huang, Yanghao Li, Haoqi Fan, Hu Xu, Huiyu Wang, Cihang Xie, Alan Yuille, Christoph Feichtenhofer. The authors present a method called Diffusion Models as Masked Autoencoders (DiffMAE) that combines generative pre-training with diffusion models for visual data. They show that DiffMAE can be a strong initialization for recognition tasks, perform high-quality image inpainting, and achieve state-of-the-art classification accuracy for video. The paper emphasizes the need to consider the specific challenges and requirements of downstream tasks when using generative pre-training.