Part 4 of 6 in a series about Harness Engineering. Previous articles:
Part 1: Harness Engineering
Part 2: The Model Should Never Touch the Tool
Part 3: Never Let an Agent Grade Its Own Homework
Establish a constraint in the first turn — a rule it must not violate, a format it must hold to, a system it must not touch. Then work with it for forty turns on something else entirely. Then check whether the constraint still holds.
If it does, your context layer is working. If the agent quietly stopped honoring the rule somewhere around turn fifteen, you have context rot, and no amount of prompt improvement will fix it, because the instruction is no longer in the window to be improved.
For about two years the industry's answer to memory was capacity. Windows went from four thousand tokens to eight, to thirty-two, to a hundred thousand, to a million. The implicit theory was that memory was a storage problem and storage was getting cheaper, so the problem would solve itself.
It did not. Chroma's context rot research tested eighteen frontier models — across the GPT, Claude, Gemini, and Qwen families — and found that every single one degrades as input length grows. Not at the edge of the advertised window. Throughout. And not only on hard tasks: reliability drops measurably even on simple retrieval and text replication, the sort of thing a model has no business getting wrong.
Part of why this went unnoticed for so long is that the benchmark everyone quoted was measuring the wrong thing. Needle-in-a-haystack tests ask a model to find a distinctive string buried in a long document, which is essentially lexical lookup. Real work is not lexical lookup. When the needle has to be found by meaning rather than by wording, when there are plausible distractors, when the answer requires combining two things that sit far apart — performance falls off considerably faster than the headline benchmark suggests.
A one-million-token window is a statement about what the model will accept, not about what it will reliably use.
Teams say 'memory' and mean two things that need entirely different machinery.
The first is what is in the context window right now: the working set. It is finite, expensive per token, and degrades as it fills. Everything that goes in competes with everything else for the model's attention, which means the window should be curated rather than accumulated. The default behavior of most agent frameworks — append every turn, every tool result, every observation, until something overflows — is the opposite of curation.
The second is durable memory: what persists outside the window between sessions. This is a storage and retrieval problem, and it is the one most teams skip entirely, which is why so many agents start every session having learned nothing from the last four hundred.
A working context layer decides deliberately what belongs in each.
In durable memory: standing constraints and system state, summarized history from prior sessions, and lessons learned from previous mistakes. Things that are needed sometimes, retrieved when relevant.
The standard remedy for a filling window is to summarize the history and continue. This works, and it also introduces two problems that recent research on evolving agent contexts has named precisely.
The first is brevity bias. Summarization optimizes for compactness, and the thing that compresses best is the thing that appeared once and mattered enormously — a specific constraint, an edge case, a domain detail. Narrative flow survives summarization. Precision does not. The constraint you set in turn one is exactly the kind of low-frequency, high-importance detail that a summarizer discards.
The second is context collapse. Each summarization pass operates on the output of the previous one, and small losses compound. Ten cycles in, the working context is a summary of a summary of a summary, confidently stated and materially wrong about details nobody can trace anymore. It is the machine-learning equivalent of a document that has been photocopied thirty times.
The mitigation is structural rather than clever: do not rewrite the whole context each cycle. Keep constraints in a separate, append-only structure that is never summarized, and re-inject it verbatim on every turn. It costs tokens, and it is the cheapest insurance in the entire harness. Summarize the narrative if you must; never summarize the rules.
Four patterns that separate teams who have solved this from teams who are still surprised by it.
Notice that three of the four are about removing things. That is the shift in mindset the research points to. The context layer's job is not to give the model more. It is to give the model less, more precisely.
Persistence has a failure mode of its own, and it is worth understanding before you build the layer rather than after.
A lesson written to durable memory is a lesson the agent will keep applying. If the lesson was wrong — a misdiagnosed failure, a rule generalized from a single unusual case, an assumption that was true of last quarter's system — it does not decay. It gets retrieved, confidently, forever. Memory turns a one-time mistake into a standing policy.
This argues for treating durable memory as something closer to a codebase than a log: reviewable, editable, versioned, and periodically pruned. If nobody on the team can answer the question 'what does this agent currently believe?' by reading a file, the memory layer has become an unaudited configuration surface.
It also argues for scoping. Memory written during one engagement should not silently leak into another, which sounds obvious and is easy to get wrong when the retrieval layer is a single shared index.
Go back to the test at the top of this article, because it takes twenty minutes and it is unusually informative. Set a constraint. Work the agent hard on something unrelated. Check whether the constraint survived.
Then do the more uncomfortable version: check whether the agent will tell you it remembers a constraint it no longer has in context. Models are generally happy to affirm a rule they can no longer see, because affirming rules is a very predictable thing to do at that point in a conversation. Test the behavior, not the claim.
Next in this series: why you will hear about model drift from your users before you hear about it from your monitoring, and what to trace instead.
This is the fourth article in a series on harness engineering. Part 3 covered verification loops. Part 5 covers observability and routing.