arxiv preprint – Lost in the Middle: How Language Models Use Long Contexts

In this episode we discuss Lost in the Middle: How Language Models Use Long Contexts
by Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang. This paper examines the impact of context length on the performance of language models in tasks such as multi-document question answering and key-value retrieval. The authors find that models perform best when relevant information is at the beginning or end of the context, but struggle to access information in the middle of long contexts. Additionally, performance decreases as the input context becomes longer, even for models specifically designed for long-context processing.