Keep It InMind: Benchmarking the Implicit-Association Blind Spot in Agent Memory
Ruizhe Li, Mingxuan Du, Benfeng Xu, Zhendong Mao · arXiv · 2026
Abstract
Agent memory systems assume that the memory you need will resemble the query that needs it. A stored nut allergy ought to change the answer to a question about macarons, yet the two share no wording a retriever can match on. This benchmark isolates that failure with paired controls separating never-stored from never-retrieved from missing background knowledge. Handed the right memory directly, the model answers 84 percent of these; made to retrieve it, six memory systems manage at most 14 percent.
Why it matters
The most striking result in this batch. If you are building on a vector-store memory layer, this is the failure mode you have not tested for — and larger embeddings do not close the gap.
https://arxiv.org/abs/2607.24368