What is a context window?

The short definition

The context window is the total number of tokens an LLM can process in a single request. Tokens include everything sent to the model: the system instruction, conversation history, retrieved documents, tool definitions, and the user message. Anything beyond the window limit is invisible to the model. It does not exist from the model perspective.

Why it is measured in tokens, not words

LLMs do not process characters or words. They process tokens, which are fragments of text produced by a tokenizer. English text averages roughly 0.75 tokens per word, so a 100,000-token context window holds approximately 75,000 words. Other languages are less efficient: Portuguese and Spanish text uses more tokens per word than English. Code is more efficient. This matters when estimating whether your use case fits within a given window.

A practical rule of thumb: a single-page PDF runs about 500 to 800 tokens. A dense legal contract of 20 pages is in the 10,000 to 16,000 token range. Knowing these ratios early prevents unpleasant surprises when you move from a prototype to production traffic at scale.

How window sizes have changed

Early production models in 2020 and 2021 had windows of 2,048 or 4,096 tokens, enough for a short document and a prompt. By 2023, the frontier moved to 32,000 and 100,000 tokens. In 2025 and 2026, windows of 1,000,000 tokens or more are available.

The expansion unlocked use cases that were impossible before: analyzing entire legal contracts, processing a full codebase in a single request, summarizing weeks of customer conversation history. Teams that were building complex chunking pipelines to work around small windows suddenly found those constraints gone. But new constraints replaced the old ones, and they are subtler.

Larger windows do not mean better results

A persistent misconception is that filling the context window is free or beneficial. It is neither. Longer inputs cost more per request, produce slower responses, and often produce worse output.

Research on long-context models consistently finds a phenomenon called lost in the middle: material at the very beginning and very end of a long context receives more attention than material in the middle. If the relevant document is number twelve in a list of twenty, the model is more likely to miss it than if it is number one or twenty. Deliberate context management outperforms naive stuffing. We covered this in depth in context engineering.

The practical consequence is that a system that sends 800,000 tokens per request is not necessarily more accurate than one that sends 8,000. It is almost certainly slower and more expensive, and it may be actively less accurate if the relevant content ends up buried.

What happens when you exceed the window

The API returns an error, or the request is silently truncated depending on the implementation. Neither is acceptable in production. Systems need to handle the case where input exceeds the window.

The standard approaches are: chunking the input and processing it in passes, using retrieval to select only the most relevant portions, summarizing prior conversation history rather than sending it verbatim, or routing to a model with a larger window. Which approach fits depends on the task. A summarization workflow tolerates chunked passes well. A reasoning task that requires seeing two pieces of evidence together in a single request does not.

The important thing is that this failure mode is designed for explicitly, not discovered during a support escalation.

Window usage in production

In a typical production request, system instructions consume 500 to 2,000 tokens. Retrieved document chunks consume 2,000 to 10,000 tokens. Conversation history consumes 500 to 5,000 tokens depending on length. Tool definitions consume 500 to 3,000 tokens. The user message itself often consumes fewer than 200 tokens.

Logging actual token consumption per component across production traffic is how you find where the window is being wasted. Teams are regularly surprised to find that tool definitions or conversation history are consuming the majority of available tokens. The retrieved passages, which carry the actual answer, get a fraction of what was available. This is a context allocation problem, not a model problem.

The practical implication for system design

Design for the case where the context is full, not for the average case. Enforce hard limits on conversation history length. Retrieve selectively using RAG rather than including entire documents. Define only the tools the model actually needs for the current request. Measure and monitor window utilization in production.

The teams that do this deliberately end up with systems that are faster, cheaper, and more accurate than those that treat the context window as unlimited. The constraint is real. Working with it intentionally is what separates a production system from a demo that breaks under load.