arxiv preprint – Guiding Instruction-based Image Editing via Multimodal Large Language Models

In this episode, we discuss Guiding Instruction-based Image Editing via Multimodal Large Language Models by Tsu-Jui Fu, Wenze Hu, Xianzhi Du, William Yang Wang, Yinfei Yang, Zhe Gan. The paper introduces MLLM-Guided Image Editing (MGIE), a system that uses multimodal large language models (MLLMs) to enhance the quality of instruction-based image editing. MGIE generates more expressive instructions from brief human commands, enabling more accurate and controllable image manipulation. The system was extensively tested and showed significant improvements in various image editing tasks according to both automatic metrics and human evaluations, while also preserving inference efficiency.