arxiv preprint – Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting

In this episode we discuss Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting
by Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, Xuanhui Wang, Michael Bendersky. The paper introduces Pairwise Ranking Prompting (PRP) as a technique to improve document ranking using Large Language Models (LLMs). Despite their impressive performance on natural language tasks, LLMs have struggled to outperform fine-tuned rankers on benchmark datasets. However, PRP achieves state-of-the-art ranking results with moderate-sized LLMs and offers benefits such as supporting both generation and scoring LLM APIs and being insensitive to input ordering.