Polp
← Back to glossary
Language models

Context window

The limited amount of information an AI model can reference while producing one response, measured in tokens and shared by instructions, conversation, retrieved evidence, and output.

Business example

An employee asks about a leave exception hidden inside hundreds of policy pages. Instead of sending the entire document library to the model, the knowledge system retrieves the permitted policy passages, places them in the context window, and links the answer back to those sources.

How it works

  1. 1.The system converts instructions, messages, documents, and tool results into tokens.
  2. 2.Those tokens occupy the model's available context together with the response it is about to generate.
  3. 3.When the useful material is larger than the window, the application must select, retrieve, summarize, cache, or remove information.
  4. 4.A larger window can hold more material, but it does not guarantee that every detail will be found or used correctly.
  5. 5.Enterprise assistants therefore combine context limits with retrieval, permissions, source ranking, and citations.

Common misconceptions

A large context window means the model remembers the company's knowledge permanently.
The window is temporary working context for a request. Durable company knowledge still needs maintained sources, retrieval, and access control.
More context always produces a better answer.
Extra material can add cost, latency, and distracting evidence. Selection and evaluation remain necessary even with long-context models.

Related concepts

Retrieval-Augmented Generation (RAG)ChunkingGroundingEmbeddingsVector database

Related reading

RAG is not uploading PDFs: permissions, freshness, traceability, and reliable sources
How to evaluate internal AI search tools: a 15-question checklist

Sources

Reviewed:

Reviewed by: Javier Chulvi Bernad · LLM Engineer · Madrid