# Context window

> VI: https://5ac.vn/wiki/context-window/ · EN: https://5ac.vn/en/wiki/context-window/ · lang: en · spec_as_of 2026-10-19 (spec no formal spec) · cluster C8

The most text a model processes in one call, counted in tokens: feed too little and the agent lacks facts, feed too much and it gets lost — and the bill tracks what goes in.

---
## Definition

The context window is the maximum amount of text a large language model can process in a single call, counted in tokens: everything fed in — the instruction, retrieved documents, conversation history — plus the text the model writes back, all of it inside one window. Outside the window there is no memory: an agent only knows what you load into its prompt. Andrew Ng's AI engineering skills map places understanding the context window in the required set, next to tokenization, cache hits, knowledge cutoff and tool calling. An engineer needs the mechanism at the level where it supports decisions, not just API calls.

## Mechanism

Windows keep growing across model generations, but a longer window does not mean the model uses it evenly. *Lost in the Middle: How Language Models Use Long Contexts* (arXiv:2307.03172, Nelson Liu et al., 2023) measured question-answering across documents placed at different positions: when the key fact sits in the middle of a long context, accuracy drops sharply compared with the beginning or the end — a U-shaped curve. 5ac's four-skills piece states the practical consequence as a pair of lines: too little context and the agent lacks information, too much and it gets lost; when knowledge cannot fit the window at all, RAG retrieves the relevant passages and feeds them into the prompt. Deciding what enters the window, and in what position, is a skill of its own.

## Application context

The context window runs straight into the bill and into the architecture. An agent task that loads tens of thousands of tokens makes the input price line heavier — the model charges for what it reads; the cost entity belongs to the token cost control entry. Serving long windows also means holding the KV cache in engine memory — the inference engine entry owns that. And window length is one of the three criteria 5ac uses to route models automatically per task (execution steps, reasoning depth, context window needed): work that needs a million-token context goes to a model that carries that load, short work should not pay for a long table.

## Sources fetched

- [Andrew Ng's AI Engineering Skills Map: The 6 Skills That Actually Matter](/en/blog/andrew-ng-ai-engineering-skills-map/) — the model-mechanics cluster: context window tradeoffs, cache hits, knowledge cutoff — accessed 2026-10-19
- [Andrew Ng's 4 AI Engineering Skills Map — What Every Developer Needs](/en/blog/ai-engineering-skills-map/) — context management: too little starves the agent, too much drowns it; RAG when knowledge exceeds the window — accessed 2026-10-19
- [AI Agent Product Strategy in the Price War — The Model-Agnostic Advantage](/en/blog/chien-luoc-san-pham-ai-gia-re-doanh-nghiep/) — context window needed as one automatic-routing criterion — accessed 2026-10-19
- <https://arxiv.org/abs/2307.03172> — Lost in the Middle: How Language Models Use Long Contexts (v3) — verified via arXiv API 2026-10-19

## Related

- [Token cost control](/en/wiki/ai-token-cost/)
- [Inference engine](/en/wiki/inference-engine/)
- [RAG](/en/wiki/rag/)
- [Context engineering](/en/wiki/context-engineering/)
- [Model routing](/en/wiki/model-routing/)