A million-token window reads like an invitation to paste the whole project. Usually that is a bad idea, and here is why.
What large context actually does
Three effects you see in practice.
Cost grows linearly. Every request pays for the whole context again. On an agentic run with dozens of calls, that multiplies.
Attention dilutes. The model finds a specific detail less reliably when it sits among a hundred thousand lines of unrelated code. Smaller but precise context often produces a better answer.
Latency rises. Reading a large input takes time too.
When long context is right
- the task needs links between distant parts and you do not know in advance which
- the document is whole and cutting it makes no sense: a contract, a specification, one long session log
- a one-off analysis where quality matters more than cost
When to cut instead
- the question is narrow and you know where the answer lives
- processing runs in batches and the same large chunk would ride along in every request
- the work repeats often while the content barely changes
In the last two cases you are better off caching the stable prefix or retrieving the relevant fragment and sending only that.
A simple heuristic
Ask yourself: solving this by hand, would you read the whole project or open three files?
If three files, give the model three files. It does not get smarter from the other hundred.
What it costs
Take the context size, multiply by number of calls and by the input rate. On a twenty-step agentic task the gap between "whole repository" and "the files that matter" is often an order of magnitude.
The calculator on the site does it on your own numbers.
What to do in practice
- Start with small context and add more when the model asks.
- Move the stable part into a cacheable prefix and keep variable content at the end.
- On repeating work, keep an index and inject only relevant fragments.
In short
A large window is an option, not an instruction. Send what the task needs, not everything that fits.