Every time you send a message to a model, it rereads everything it is allowed to see: the instructions, the conversation so far, the documents you attached. That reading budget is the context window. It is measured in tokens, and it has a hard limit.
Pixel François shows an instruction given once disappearing when the conversation is compacted, then the same instruction surviving in an instructions file.
What fits, and what falls out
Long conversations do not grow forever. When the window is full, something has to go, and it is usually the oldest content: the brief you gave at the start, the constraint you mentioned once. The model then answers confidently without it.
Key concept
A model only knows what is inside its context window right now. It has no memory between calls. Whatever does not fit, or was pushed out, does not exist for it.
8 messages in the context window. 4 older messages have fallen out.
Assistants with a memory feature are no exception: the application keeps notes and puts them back in the window on every request.
More context is not always better either. As the input grows longer and fills with irrelevant material, the model gets worse at finding the information that matters: Anthropic calls this context rot. The effect varies by model, so measure it on your own long inputs. Every extra token also costs money and time on every request.
6 blocks of irrelevant material around the information that matters: it gets lost in the noise, and each one is paid for on every request. Illustrative fade: the effect varies by model.
Not every window is the same size
Window size depends first on the model: how it was designed and trained. As of October 2026, the flagship models of three major vendors take about a million tokens: 1,000,000 for Anthropic's current Claude models, 1,050,000 for the GPT models in OpenAI's API, 1,048,576 for Gemini 3.1 Pro, in preview at Google. Earlier generations took less: 200,000 tokens for several older Claude models.
It also depends on the product. Serving long inputs is expensive for the vendor, which does not always give you the model's full window. As of October 2026, ChatGPT gives its fast model, GPT Instant, 27,000 tokens on the free plan and 54,000 on the Go and Plus plans, far from the million in the API. As of the same date, Claude Haiku 5.5 has a million tokens in chats on Claude's paid plans, but 500,000 in Claude Cowork. So the same model does not get the same window in every product: check the window of the tool you use, not just the model's.
What a big window costs. A full window is paid for on every request: at the same price per token, a 500,000-token request costs a hundred times more in input than a 5,000-token one. Some vendors also raise the price past a threshold. As of October 2026, the input price doubles above 200,000 tokens for Gemini 3.1 Pro at Google, and above 272,000 tokens at OpenAI. Anthropic charges 0.50 dollars per million input tokens for Claude Haiku 5.5 above 100,000 tokens, instead of 0.10, but keeps the same price for its other current models. A long input can also delay the answer: Anthropic says latency depends on prompt length, but OpenAI says the length of the answer matters far more.
What it does not fix. A bigger window does not read better. The context rot described above gets worse as the window fills. A million tokens of poorly sorted documents can give a worse answer than a few thousand well-chosen ones.
When to search instead of growing the window. If your documents exceed the window, change often, or if each question only concerns a small part of them, RAG (retrieval-augmented generation) is the better fit: it searches for the useful passages and sends only those to the model. A big window is worth it when the task really requires reading everything at once: comparing two full contracts, reviewing a long report.
Search first: RAG searches for the useful passages and sends only those to the model. It is the better fit if your documents exceed the window, change often, or if each question only concerns a small part of them.
What it means for you
For founders
Context is a cost line. Every token you send is paid for on every request, so "send everything just in case" makes your bill grow with your usage.
For product managers
When users say the assistant "forgot", it often did not forget. The information fell out of the window. Design flows that restate what matters instead of hoping the model remembers.
For tech leads
Treat the context as a budget you allocate between the system prompt, retrieved documents and conversation history. Measure each part, and decide what gets trimmed first when the budget runs out. Prompting: briefing a model well covers what should go into the window, what should be fetched and what should come out.
For business leaders
When a vendor promises an assistant that "remembers everything", ask what it drops once a customer's history gets long. The answer sets what you can promise your own customers.
Go deeperHow tokens are counted
A token is a chunk of text, often part of a word. In English, one token is between half and three quarters of a word, depending on how each model splits text. As of October 2026, Anthropic's documentation says that a million tokens represent about 555,000 words on its current models, against about 750,000 on older ones: the same text therefore takes more tokens than before. French and code usually need more than English for the same meaning.
The limit covers both what you send and what the model writes back, so a long answer leaves less room for your messages and documents. When a conversation exceeds the limit, the application decides what to drop, not the model. The section “Compacting, manual or automatic”, below, explains how.
The classic mistake
fAIling RULE I
The trap: Paste your entire codebase into the prompt
Pasting a whole codebase or document set into the prompt does not make the model understand more. It fills the window with noise, pushes out the parts that matter, and costs more on every request.
Read rule I in the manifestoTry it
Pick one AI feature or workflow your team uses. Write down everything it sends to the model on a typical request, and estimate the share that actually changes the answer. Remove one part that does not, and compare the results for a week.
Around this topic
Go deeperWhat fills a window
Your conversation is only one of the things in the window. It shares the space with the system prompt, the tool definitions, instruction and memory files, loaded extensions (skills, plugins), and a reserve kept free for the answer and for compaction (a summary that replaces the start of the conversation, see below). Part of the window is already taken before you type a word.
An example from October 2026, in a Claude Code session (Anthropic's coding agent) with a one-million-token window: the conversation took 7.4%, and the setup loaded at startup almost as much (7.1%). Every connected tool and every instructions file takes up room on every request, whether it is used or not.
For document Q&A, the conversation is 57% of the window in this illustrative example; it is the largest part.
Go deeperCompacting, manual or automatic
Compacting means replacing the start of a conversation with a summary to free up space. The model does not do it; the application does. Manual compaction runs when you choose, at a good moment such as between two tasks, and you can say what to keep. Automatic compaction kicks in near the limit, often in the middle of a task, and decides on its own what survives.
Either way, the summary leaves information out: the exact wording, a constraint set at the start, a detail that seemed minor. Instructions written in a file that is reloaded on every request survive compaction. Instructions given once in the conversation can disappear. If a rule matters, put it in an instructions file, not in a message.
Go deeperPrompt caching
On every request, the model rereads everything from the start, and the start rarely changes: the system prompt, the tools, the reference documents. Providers can keep that shared start in a cache (a copy ready to reread) for a few minutes. Rereading it from the cache costs less, and the answer arrives sooner.
A cache read costs a small fraction of the normal price: the Tokens and cost page gives the prices as of October 2026. Also as of October 2026, Anthropic's cache expires by default after five minutes without use. The practical rule: put what does not change at the start, and what changes at the end.
What to do with it
An instruction given once can fall out of the window: if it matters, put it in an instructions file, not in a message.
More context is not always better: irrelevant material hides what matters, and every token is paid for on every request.
When documents exceed the window or change often, fetch the useful passages with RAG instead of growing the window.