Arxiv paper – s1: Simple test-time scaling

In this episode, we discuss s1: Simple test-time scaling by Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, Tatsunori Hashimoto. The paper introduces a straightforward method for test-time scaling in language models to enhance reasoning performance by utilizing additional computational resources during inference. The authors develop a curated dataset of 1,000 high-quality, diverse, and challenging questions with reasoning traces and implement a “budget forcing” technique that controls the model’s computation by either terminating its reasoning process or extending it to encourage double-checking answers. Using this approach, their fine-tuned Qwen2.5-32B-Instruct model outperforms OpenAI’s o1 model on competitive math benchmarks by up to 27% and the resources are made available as open-source.