arxiv preprint – Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking

In this episode, we discuss Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking by Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, Noah D. Goodman. The paper presents Quiet-STaR, an advancement of Self-Taught Reasoner (STaR), which teaches Language Models to generate internal rationales to enhance text predictions. By introducing a tokenwise parallel sampling algorithm, learnable tokens for marking thoughts, and extending teacher-forcing, the approach addresses practical challenges in model implementation. Results demonstrate that the approach enables the model to better predict challenging tokens, answer complex questions, and improve performance on benchmarks without task-specific fine-tuning, signifying progress towards more generative and scalable reasoning in LMs.