Context window
The context window is the maximum number of tokens a language model can consider at once — the combined size of the prompt and the response it generates.
Everything the model 'sees' for a request — system instructions, prior conversation, retrieved documents, and the current question — must fit within the context window, along with room for the output. Exceed it and earlier content must be dropped or summarized.
A larger window allows more context but does not make it free: because cost scales with tokens processed, filling a large window on every call is expensive, which is why context management and prompt caching matter as much as window size.
Newsletter
Learn this stuff by email
New definitions, tools, and explainers when I publish them. No spam, unsubscribe anytime.