We gave it a memory. It never looked.

The agent we ship has a memory system. You can watch it work: facts go in, they come back on request, the store is real and the search returns what you ask for. We had been describing that as the agent has memory for weeks.

Then we asked the only question that matters — does having it make the agent better at anything — and the answer was no. Not a little. Zero.

The setup

One standing operator preference, seeded into a throwaway substrate before the run. Nothing in the prompt mentions it. A grader looks for whether the answer honours the preference, keyed to a token the model has no way to guess, and the grader was self-tested in both directions on ten cases before it graded anything real.

Two arms. Arm A has no memory at all — the control, which must fail. Arm B has the memory plugin, seeded and working. Five trials each, and we report the all-five number, not the best of five.

ARM A (no memory)      0/5
ARM B (memory plugin)  0/5

The control failed as designed. So did the treatment.

What was actually wrong

The instrumentation gave it away immediately: zero searches. Across every trial, on a substrate that held the answer, the model never once queried it.

Our plugin's system prompt block politely tells the model "recall before you act on something you're unsure about." That is a request. The model is a 2.5B-active MoE running on a Mac — the entry tier, deliberately, because if the mechanism only works on a big model it is not a mechanism, it is a subsidy. It did not comply, and it was under no obligation to. The plugin was a filing cabinet in the corner of the room with a sign on it.

The embarrassing part is that the framework already solved this. The base provider class ships prefetch, on_turn_start, queue_prefetch — auto-retrieval hooks that put relevant memory into context without the model deciding anything. We had overridden none of them.

The fix, and the number

prefetch(query) searches the substrate for the incoming turn and returns the hits as an injectable block. The fact is simply present. No compliance required. It is best-effort by construction — a failed search returns empty and never breaks a turn — and it ships with six tests that were proven red before they were allowed to be green.

ARM A (no memory)      0/5
ARM B (prefetch)       5/5

Same model. Same weights. Same quantisation. Same everything except where the fact was sitting when the turn began.

Why we are writing this down

This lab's whole bet is that the harness matters more than the weights, and it is easy to say that and hard to catch yourself a moment where it is literally, measurably true. This is one. The model did not get smarter. The retrieval around it did, and a capability that read as absent turned out to have been present and unreachable the entire time.

It also cost us something to learn, because the failure mode is invisible from the outside. A passive store and a working memory look identical in every log we had — both show facts going in, both show searches succeeding when something asks. The only thing that separates them is the question did anything ask, and until we counted, nothing had.

The lesson we are keeping: shipping a capability is not the same as shipping its use. If the model has to choose to use the thing, measure whether it chooses. Ours never did, five times out of five, and we would still be calling it a memory system.