“Lost in the middle” is the observed failure of language models to reliably use information placed in the middle of a long context. Recall is strongest for content at the very start and very end of the window and weakest in between.
Liu et al. measured the shape directly in Lost in the Middle: How Language Models Use Long Contexts (TACL 2024): holding the input identical and moving only the position of the relevant document, accuracy traces a U - highest at the beginning or end, lowest in the middle. It held on both multi-document QA and synthetic key-value retrieval, so it is a property of how the window is read, not of one benchmark’s subject matter.
Consequence
A larger window does not guarantee a fact buried mid-context is used. Quality can drop while the token count is still well under the hard limit, so “we have not hit the ceiling yet” is not evidence that the window is healthy - which is why context engineering matters more than raw window size. The full argument is in Context engineering beats a bigger context window.
What to do
- Put the most important instructions and data near the top or bottom of the prompt.
- Trim irrelevant middle content rather than trusting the model to ignore it.
- Prefer ranked, deduplicated retrieval over dumping whole files.
The same shape shows up without an API in sight: paste a long document into a chat window and the middle of it is what gets missed. The non-technical version of the fix is in how to use ChatGPT effectively.