What's Being Said Right Now

Opening Scene

An improv performer, however experienced, can only genuinely respond to what’s actually been said in the current scene — not to a different scene from three years ago, not to a conversation happening in another room. There’s a real, bounded amount of “currently active” information a performer is working with at any given moment, no matter how much total experience they’ve accumulated over a career. A language model has an almost identical, formally defined limit.

In Plain English

The context window is the maximum amount of text — measured in tokens, covered in Article 5 — that a language model can consider at once when generating a response. Everything within the context window (the conversation so far, any provided documents, the current prompt) directly informs the next prediction; anything outside it is simply invisible to the model in that moment, regardless of how much the model “knows” in a general sense from pretraining.

The Old Way

Before modern LLMs with large context windows, earlier language models faced much tighter constraints:

Early n-gram models, covered in Article 3, effectively had a context window of only a few preceding words, far too short to capture any meaningful long-range structure.
Early neural language models had genuinely limited effective context, even before the transformer architecture, covered in Article 8, dramatically improved this.
Working with long documents historically required manual chunking and summarization workarounds, since models simply couldn’t process an entire long document in one pass.

Context window size has grown dramatically as transformer architectures and their variants have matured, directly expanding what’s practically possible in a single interaction.

What’s Changing (and Why AI Is the Reason)

Context windows have expanded dramatically across successive model generations — from a few thousand tokens in earlier models to context windows now spanning hundreds of thousands of tokens in some modern models, fundamentally changing what workflows are practical.
Larger context windows have reduced, though not eliminated, the need for external retrieval techniques like RAG, covered in this content library’s next dedicated series — though RAG remains valuable even with large context windows for reasons covered there.
Practical performance within a context window isn’t always uniform — research has shown models can attend less reliably to information in the middle of a very long context, a real, practical consideration worth knowing about when designing prompts and workflows.

The Metaphor, Fully Extended

The Improv Scene	Context Window Concept
Everything actually said so far in the current scene	Everything within the model’s current context window
A performer’s general experience, distinct from what’s currently active in this scene	A model’s pretrained knowledge, distinct from what’s currently in its context window
A scene too long to keep every early detail equally sharp in mind	A context window where information in the middle can be attended to less reliably
A scene partner’s earlier line, now genuinely forgotten once too much time has passed	Information that falls outside the context window, genuinely invisible to the model

For Beginners: What to Actually Do

Learn your specific model’s context window size in tokens, and practice roughly estimating how much text that actually represents.
Practice distinguishing “the model doesn’t know this” (missing from pretraining) from “the model can’t see this right now” (missing from the context window) — these are genuinely different problems with different solutions.
Experiment with placing critical information at the beginning or end of a long prompt, given the “middle of context” attention concern noted above.

For Practitioners and Leaders: The Deeper Layer

Factor context window size directly into workflow design, particularly for tasks involving long documents or extended conversations.
Recognize that a larger context window doesn’t automatically mean uniformly reliable attention across all of it — plan accordingly for genuinely critical information.
Weigh context window expansion against the RAG-based approaches covered in this content library’s next series, since both address related but distinct problems.

Quick Recap

The context window is the maximum amount of text a model can consider at once, measured in tokens.
Anything outside the context window is invisible to the model in that moment, regardless of its general pretrained knowledge.
Context windows have expanded dramatically across model generations, changing what workflows are practical.
Attention within a very long context window isn’t always uniform, with some evidence that middle-of-context information gets attended to less reliably.

Where This Fits in the Series

Article 7 covered the bounded space a model actually works within. Article 8 covers the mechanism that determines which parts of that space matter most for any given prediction.