"It has a million-token context window" gets thrown around like it solves everything. It doesn't. I learned this the slow way, by stuffing an entire codebase into a single Claude call and watching the agent get worse, not better. More context is not free, and it's not always better. Let me explain what a context window actually is and when the big number helps you versus hurts you.
What the window actually is
The context window is the maximum number of tokens the model can attend to in one request — everything you send (system prompt, tools, full message history, files) plus everything it generates. On current Claude models the input window is large: Opus 4.6, 4.7, and 4.8 all run a 1M-token context, Sonnet 4.6 is also 1M, and Haiku 4.5 is 200K. Output is capped separately — up to 128K tokens on the Opus models, 64K on Sonnet and Haiku.
Note those are two different limits. Hit the input limit and you get stop_reason: "model_context_window_exceeded". Hit the requested output cap and you get stop_reason: "max_tokens". They mean different things and need different fixes, so handle both branches. People conflate them constantly and then wonder why bumping max_tokens didn't help — because the problem was the input side.
Why a token is not a word
Quick mechanism check, because it matters for budgeting. The model doesn't see words; it sees tokens, which are sub-word chunks produced by a tokenizer. English prose runs roughly one token per three-quarters of a word. Code tokenizes denser. Non-English text, denser still.
Which means you cannot estimate Claude token counts with tiktoken — that's OpenAI's tokenizer, and it undercounts Claude by 15–20% on plain text and far more on code. If you need a real number, call the count_tokens endpoint with the same model ID you'll run inference on. Tokenizers are model-specific. Opus 4.8 and Fable 5 share one; older models count differently. Don't reuse a budget measured on the wrong model.
The part nobody warns you about: filling the window degrades quality
Here's the counterintuitive bit. Just because you can put a million tokens in doesn't mean you should. Models attend less reliably to information buried in the middle of a very long context, and a noisy context dilutes the signal the model needs for the current step. I've watched an agent with a clean 20K-token context outperform the same agent at 400K tokens on the identical task, because at 400K, 95% of what it was attending to was stale tool output from twelve steps ago.
So the goal isn't to maximize how much you cram in. It's to keep the context relevant. The million-token ceiling is a safety margin for genuinely large single inputs — a whole repo you need analyzed at once, a long document, a deep research trace. It is not a license to never clean up.
Three tools for keeping context relevant
These solve different problems; pick by what's wrong.
Context editing prunes. It clears stale tool results and thinking blocks out of the transcript based on thresholds you set. The cleared content is removed, not replaced. Reach for this when old tool outputs are dead weight and you want a lean transcript without losing conversation structure. It's the right call for keeping tool-call accuracy high in long loops.
Compaction summarizes. As a conversation approaches the window limit, the API condenses earlier history into a summary block server-side. Use it when you're genuinely going to hit the ceiling and you need to keep going. One critical gotcha: you must append the full response.content back to your messages each turn — the compaction blocks live in there, and if you extract only the text and append that, you silently lose the compaction state and the whole thing breaks.
Memory persists across sessions. The other two operate inside one conversation; memory is files the agent reads and writes that survive a process restart. When state has to outlive the session — preferences, accumulated learnings, a running scratchpad — that's memory, not context. And honestly, models like Opus 4.8 perform noticeably better when you give them a place to write things down and tell them to check it next time.
Many long-running agents use all three. Editing prunes the dead turns, compaction catches you near the ceiling, memory holds what matters between runs.
A budgeting heuristic that's served me well
Think of your context as having three zones. The frozen prefix (system prompt, tools) — keep it stable, it never changes, and as a bonus it caches. The working set — the recent turns the model actually needs for the current step. And the tail — the current question or tool result. When the working set starts filling with things the model hasn't referenced in many turns, that's your cue to edit or compact. Not when you hit some token number. When the relevance drops.
One more practical note: large outputs. If you ask for anything near the 128K output ceiling on an Opus model, you have to stream the request. Non-streaming calls hit SDK HTTP timeouts at high max_tokens — the request just dies waiting. Use the streaming helper and pull the final message off it.
The million-token window is a real capability and sometimes exactly what you need. But the skill isn't filling it. The skill is knowing when a tight, relevant 30K context will beat a bloated 600K one — which, in my experience building agents, is most of the time.