arxiv preprint – Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimization

In this episode, we discuss Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimization by Costas Mavromatis, Petros Karypis, George Karypis. The paper presents PackLLM, a method for fusing knowledge from multiple Large Language Models (LLMs) during test-time by optimizing the importance of each LLM based on the input prompt to minimize perplexity. It introduces two variants: PackLLMsim, which validates perplexity as an expertise indicator, and PackLLMopt, which uses a greedy algorithm for perplexity minimization. Experiments with over 100 LLMs show that PackLLM outperforms existing test-time fusion approaches and learning-based fusers, demonstrating significant accuracy improvements.