Context window expansion is the set of techniques used to increase how much input a model can process in one request. It extends the context window so the model can receive longer documents, larger codebases, longer conversations, or more tool results.
Expansion can happen through several approaches:
- training or fine-tuning on longer sequences;
- changing positional encodings;
- modifying attention mechanisms;
- using recurrent, memory, or state-space components;
- compressing older context into learned or explicit summaries; and
- retrieval systems that simulate larger context by selecting relevant chunks.
Longer context is useful, but it is not a substitute for context engineering. A larger window can still contain irrelevant, contradictory, or poorly ordered information. It may also increase cost, latency, and exposure to prompt injection attacks.
The practical question is the effective context window: how much information the model can use reliably for a task, not merely the advertised token limit.
The IJCAI survey Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models reviews methods for extending LLM sequence length.
The LLM Knowledge Base is a collection of bite-sized explanations for commonly used terms and abbreviations related to Large Language Models and Generative AI.
It's an educational resource that helps you stay up-to-date with the latest developments in AI research and its applications.