Category: Uncategorized
-
Arxiv Paper – When Does Perceptual Alignment Benefit Vision Representations?
In this episode, we discuss When Does Perceptual Alignment Benefit Vision Representations? by Shobhita Sundaram, Stephanie Fu, Lukas Muttenthaler, Netanel Y. Tamir, Lucy Chai, Simon Kornblith, Trevor Darrell, Phillip Isola. The paper examines how aligning vision model representations with human perception affects various computer vision tasks by finetuning models on human similarity judgments and testing…
-
Arxiv paper – SceneCraft: Layout-Guided 3D Scene Generation
In this episode, we discuss SceneCraft: Layout-Guided 3D Scene Generation by Xiuyu Yang, Yunze Man, Jun-Kun Chen, Yu-Xiong Wang. SceneCraft is a method for generating detailed indoor 3D scenes based on user-provided textual descriptions and spatial preferences, using a rendering-based technique and a semantic and depth-conditioned diffusion model to enhance scene representation. It extends beyond…
-
arxiv preprint – A Tale of Tails: Model Collapse as a Change of Scaling Laws
In this episode, we discuss A Tale of Tails: Model Collapse as a Change of Scaling Laws by Elvis Dohmatob, Yunzhen Feng, Pu Yang, Francois Charton, Julia Kempe. The paper investigates the impact of incorporating synthetic data into training datasets on neural scaling laws and future model performance, questioning whether this integration will lead to…
-
arxiv preprint – Thinking LLMs: General Instruction Following with Thought Generation
In this episode, we discuss Thinking LLMs: General Instruction Following with Thought Generation by Tianhao Wu, Janice Lan, Weizhe Yuan, Jiantao Jiao, Jason Weston, Sainbayar Sukhbaatar. The paper introduces a novel approach to enhance Large Language Models by incorporating an iterative thought process before response generation, which helps in overcoming limitations of current models that…
-
arxiv preprint – Thinking LLMs: General Instruction Following with Thought Generation
In this episode, we discuss Thinking LLMs: General Instruction Following with Thought Generation by Tianhao Wu, Janice Lan, Weizhe Yuan, Jiantao Jiao, Jason Weston, Sainbayar Sukhbaatar. The paper introduces a novel approach to enhance Large Language Models by incorporating an iterative thought process before response generation, which helps in overcoming limitations of current models that…
-
arxiv preprint – Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think
In this episode, we discuss Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think by Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, Saining Xie. The paper presents a novel approach called REPresentation Alignment (REPA) to enhance the training efficiency and quality of generative diffusion models by integrating…
-
arxiv preprint – F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching
In this episode, we discuss F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching by Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, Jian Zhao, Kai Yu, Xie Chen. F5-TTS is a fully non-autoregressive text-to-speech system that utilizes flow matching with Diffusion Transformer (DiT) and addresses limitations of previous systems…
-
arxiv preprint – One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation
In this episode, we discuss One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation by Fabian Paischer, Lukas Hauzenberger, Thomas Schmied, Benedikt Alkin, Marc Peter Deisenroth, Sepp Hochreiter. The paper introduces Explained Variance Adaptation (EVA), a method that enhances the fine-tuning of foundation models by using singular value decomposition for a more effective…
-
arxiv preprint – Eliminating Oversaturation and Artifacts of High Guidance Scales in Diffusion Models
In this episode, we discuss Eliminating Oversaturation and Artifacts of High Guidance Scales in Diffusion Models by Seyedmorteza Sadat, Otmar Hilliges, Romann M. Weber. The paper addresses issues with high guidance scales in classifier-free guidance (CFG) for diffusion models, which can cause oversaturation and artifacts. The authors propose a modified update rule by reducing the…
-
arxiv preprint – NEPTUNE: THE LONG ORBIT TO BENCHMARKING LONG VIDEO UNDERSTANDING
In this episode, we discuss NEPTUNE: THE LONG ORBIT TO BENCHMARKING LONG VIDEO UNDERSTANDING by The authors of the paper “NEPTUNE: THE LONG ORBIT TO BENCHMARKING LONG VIDEO UNDERSTANDING” are: – Arsha Nagrani – Mingda Zhang – Ramin Mehran – Rachel Hornung – Nitesh Bharadwaj Gundavarapu – Nilpa Jha – Austin Myers – Xingyi Zhou…
-
arxiv preprint – SHIC: Shape-Image Correspondences with no Keypoint Supervision
In this episode, we discuss SHIC: Shape-Image Correspondences with no Keypoint Supervision by Aleksandar Shtedritski, Christian Rupprecht, Andrea Vedaldi. The paper introduces SHIC, a novel method for learning canonical surface mappings without manual supervision by using foundation models such as DINO and Stable Diffusion. SHIC simplifies the task to image-to-image correspondence prediction, outperforming some supervised…
-
arxiv preprint – E.T. Bench: Towards Open-Ended Event-Level Video-Language Understanding
In this episode, we discuss E.T. Bench: Towards Open-Ended Event-Level Video-Language Understanding by Ye Liu, Zongyang Ma, Zhongang Qi, Yang Wu, Ying Shan, Chang Wen Chen. The paper introduces E.T. Bench, a comprehensive benchmark for fine-grained event-level video understanding, evaluating Video-LLMs across 12 tasks and 7K videos. It highlights the challenges these models face in…
-
arxiv preprint – LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness
In this episode, we discuss LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness by Chenming Zhu, Tai Wang, Wenwei Zhang, Jiangmiao Pang, Xihui Liu. Recent advancements in Large Multimodal Models (LMMs) have significantly improved 2D visual understanding but 3D scene understanding has lagged due to dataset and encoder limitations. The paper introduces…
-
arxiv preprint – DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos
In this episode, we discuss DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos by Wenbo Hu, Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xiaodong Cun, Yong Zhang, Long Quan, Ying Shan. DepthCrafter is a novel method for estimating temporally consistent depth in open-world videos without needing additional data like camera poses or optical flow. It…
-
arxiv preprint – Programming Every Example: Lifting Pre-training Data Quality like Experts at Scale
In this episode, we discuss Programming Every Example: Lifting Pre-training Data Quality like Experts at Scale by Fan Zhou, Zengzhi Wang, Qian Liu, Junlong Li, Pengfei Liu. The paper introduces Programming Every Example (PROX), a framework that enables small language models to refine pre-training corpora by executing fine-grained operations on individual examples, outperforming traditional human-crafted…
-
arxiv preprint – Phantom of Latent for Large Language and Vision Models
In this episode, we discuss Phantom of Latent for Large Language and Vision Models by Byung-Kwan Lee, Sangyun Chung, Chae Won Kim, Beomchan Park, Yong Man Ro. The paper introduces Phantom, an efficient LLVM family designed to perform comparably to larger models but with significantly smaller sizes, ranging from 0.5B to 7B parameters. By temporarily…
-
arxiv preprint – Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think
In this episode, we discuss Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think by Gonzalo Martin Garcia, Karim Abou Zeid, Christian Schmidt, Daan de Geus, Alexander Hermans, Bastian Leibe. The study identifies and corrects a flaw in the inference pipeline of large diffusion models used for monocular depth estimation, achieving over 200× speed improvement…
-
arxiv preprint – On the Diagram of Thought
In this episode, we discuss On the Diagram of Thought by Yifan Zhang, Yang Yuan, Andrew Chi-Chih Yao. Diagram of Thought (DoT) is a framework for modeling iterative reasoning in large language models (LLMs) using a directed acyclic graph (DAG) to organize propositions, critiques, refinements, and verifications. This method allows the model to navigate complex…
-
arxiv preprint – Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources
In this episode, we discuss Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources by Alisia Lupidi, Carlos Gemmell, Nicola Cancedda, Jane Dwivedi-Yu, Jason Weston, Jakob Foerster, Roberta Raileanu, Maria Lomeli. The paper presents Source2Synth, a method designed to enhance Large Language Models (LLMs) by generating synthetic data with intermediate reasoning steps, grounded…
-
arxiv preprint – SongCreator: Lyrics-based Universal Song Generation
In this episode, we discuss SongCreator: Lyrics-based Universal Song Generation by Shun Lei, Yixuan Zhou, Boshi Tang, Max W. Y. Lam, Feng Liu, Hangyu Liu, Jingcheng Wu, Shiyin Kang, Zhiyong Wu, Helen Meng. The paper introduces SongCreator, a novel song-generation system designed to create songs with both vocals and accompaniment from given lyrics. This is…
-
arxiv preprint – Achieving Human Level Competitive Robot Table Tennis
In this episode, we discuss Achieving Human Level Competitive Robot Table Tennis by David B. D’Ambrosio, Saminda Abeyruwan, Laura Graesser, Atil Iscen, Heni Ben Amor, Alex Bewley, Barney J. Reed, Krista Reymann, Leila Takayama, Yuval Tassa, Krzysztof Choromanski, Erwin Coumans, Deepali Jain, Navdeep Jaitly, Natasha Jaques, Satoshi Kataoka, Yuheng Kuang, Nevena Lazic, Reza Mahjourian, Sherry…
-
arxiv preprint – Sapiens: Foundation for Human Vision Models
In this episode, we discuss Sapiens: Foundation for Human Vision Models by Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stuart Anderson, Shunsuke Saito. The Sapiens model family addresses four key human-centric vision tasks and supports 1K high-resolution inference, with easy adaptability through fine-tuning on a large dataset of human images.…
-
arxiv preprint – Re-Reading Improves Reasoning in Large Language Models
In this episode, we discuss Re-Reading Improves Reasoning in Large Language Models by Xiaohan Xu, Chongyang Tao, Tao Shen, Can Xu, Hongbo Xu, Guodong Long, Jian-guang Lou. The paper presents a novel prompting method called RE2 (Re-Reading) that improves the reasoning capabilities of Large Language Models by processing questions twice for better understanding. Unlike conventional…
-
arxiv preprint – SPIRE: Semantic Prompt-Driven Image Restoration
In this episode, we discuss SPIRE: Semantic Prompt-Driven Image Restoration by Chenyang Qi, Zhengzhong Tu, Keren Ye, Mauricio Delbracio, Peyman Milanfar, Qifeng Chen, Hossein Talebi. The paper introduces SPIRE, a novel framework that utilizes semantic and restoration prompts to guide image restoration tasks such as denoising, super-resolution, deblurring, and compression artifact removal. Current text-driven diffusion…
-
arxiv preprint – Automated Design of Agentic Systems
In this episode, we discuss Automated Design of Agentic Systems by Shengran Hu, Cong Lu, Jeff Clune. The paper introduces Automated Design of Agentic Systems (ADAS), which aims to replace hand-designed AI solutions with automatically created ones using a new approach where agents are defined and improved by a meta agent through programming. They propose…