Lost in the Middle: How Language Models Use Long Contexts

This academic paper explores how language models utilize long input contexts, focusing on their ability to identify and retrieve relevant information. The authors conducted experiments using multi-document question answering and key-value retrieval tasks, varying the position of crucial data within the input. Their findings reveal a "U-shaped" performance curve, indicating that models are most effective when relevant information is at the beginning or end of the context, with performance significantly declining when it's in the middle. The study further investigates the impact of model architecture, query contextualization, and instruction fine-tuning on this observed positional bias, ultimately suggesting that providing overly long contexts might not always be beneficial due to these limitations.

There are no comments yet.

Please log in to write the first comment.