arxiv preprint – Mission: Impossible Language Models

In this episode, we discuss Mission: Impossible Language Models by Julie Kallini, Isabel Papadimitriou, Richard Futrell, Kyle Mahowald, Christopher Potts. The paper investigates Chomsky’s claim that large language models (LLMs) can learn both possible and impossible languages by designing synthetic impossible languages with unnatural word orders and grammar rules. Experiments conducted using GPT-2 small models reveal that these models struggle to learn such impossible languages compared to English, challenging the initial claim. The study aims to inspire further research into testing various LLM architectures on impossible languages to better understand their cognitive and typological implications.