Podcast
The podcast where we breakdown the recent AI papers and explain them in simple terms for you to understand.
Arxiv paper – How much do language models memorize? – AI Breakdown
In this episode, we discuss How much do language models memorize? by John X. Morris, Chawin Sitawarin, Chuan Guo, Narine Kokhlikyan, G. Edward Suh, Alexander M. Rush, Kamalika Chaudhuri, Saeed Mahloujifar. The paper introduces a method to quantify how much a language model memorizes versus generalizes from data, defining model capacity as total memorization excluding generalization. Through extensive experiments on GPT-family models of varying sizes, the authors find that models memorize data until their capacity is full, after which generalization (or "grokking") increases and unintended memorization decreases. They establish scaling laws linking model capacity, data size, and membership inference, estimating GPT models have about 3.6 bits-per-parameter capacity.
- Arxiv paper – How much do language models memorize?
- Arxiv paper – MMaDA: Multimodal Large Diffusion Language Models
- Arxiv paper – Superhuman performance of a large language model on the reasoning tasks of a physician
- Arxiv paper – The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models
- Arxiv paper – DanceGRPO: Unleashing GRPO on Visual Generation
News
- Arxiv paper – How much do language models memorize?In this episode, we discuss How much do language models memorize? by John X. Morris, Chawin Sitawarin, Chuan Guo, Narine… Read more: Arxiv paper – How much do language models memorize?
- Arxiv paper – MMaDA: Multimodal Large Diffusion Language ModelsIn this episode, we discuss MMaDA: Multimodal Large Diffusion Language Models by Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang,… Read more: Arxiv paper – MMaDA: Multimodal Large Diffusion Language Models
- Arxiv paper – Superhuman performance of a large language model on the reasoning tasks of a physicianIn this episode, we discuss Superhuman performance of a large language model on the reasoning tasks of a physician by… Read more: Arxiv paper – Superhuman performance of a large language model on the reasoning tasks of a physician
- Arxiv paper – The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language ModelsIn this episode, we discuss The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models… Read more: Arxiv paper – The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models
- Arxiv paper – DanceGRPO: Unleashing GRPO on Visual GenerationIn this episode, we discuss DanceGRPO: Unleashing GRPO on Visual Generation by Zeyue Xue, Jie Wu, Yu Gao, Fangyuan Kong,… Read more: Arxiv paper – DanceGRPO: Unleashing GRPO on Visual Generation