Does your agent have to read everything it remembers before answering?

Chin-Lun Fu et al. · 1 authors · Machine Learning Center of Excellence, JPMorgan Chase & Co · 2026 · arXiv:2610.02472

Does your agent have to read everything it remembers before answering?

Opening

Every question your assistant handles pulls the same weight of history into the window, because somewhere in the conversation log might be the one detail the answer needs. That habit holds until the context overflows or the bill arrives, and it spends attention on routine questions a summary could settle. The question this paper asks is whether the depth of reading should depend on the difficulty of the question.

What the paper does

APDMem stores conversation history as a hierarchy of four progressively detailed layers: thematic summaries, personalized key facts, turn-level evidence notes, and raw messages. At inference a controller reads the top layers first and drills down only when the query demands it, and a note synthesizer turns whatever was retrieved into a query-focused structure that consolidates facts, orders events, and flags contradictions before the answer is generated. The evaluation runs on LongMemEval.

Findings

  1. History is stored as layers, not one pile — The store runs from thematic summaries down to raw messages, so each question can choose how deep to read instead of inheriting a fixed depth.

APDMem represents conversation history as four progressively detailed layers: thematic summaries, personalized key facts, turn-level evidence notes, and raw messages

  1. The controller reads the summary first — High-level summaries come before fine evidence, and drill-down happens only when the query actually needs it.

it first reads high-level summaries and drills into finer evidence only when needed

  1. Depth follows difficulty — Simple questions stop at the top layers, while temporal, multi-hop, or exact-evidence questions trigger deeper inspection, which is what makes the cost and fidelity trade adaptive rather than fixed.

simple queries can terminate early, while complex temporal, multi-hop, or exact-evidence queries trigger deeper inspection

  1. Strong results while touching a small slice of history — On LongMemEval the system reports strong long-context memory reasoning while accessing only 8% of the total conversations.

APDMem achieves strong performance for long-context memory reasoning while accessing only 8% of the total conversations

Caveat

Everything here is measured on LongMemEval alone, and the abstract does not separate the controller’s drill-down decisions in an ablation, so how much of the result comes from the disclosure logic itself is not shown.

What this means for 5ac

For a 5ac agent on a budget, progressive disclosure is the cheapest context control available: keep a summary layer on top, let routine questions terminate there, and pay for deeper reads only when a question turns temporal or exact. Build the store in layers from the start, because retrofitting raw messages into a hierarchy later costs more than writing summaries as conversations land. Treat the 8% as a target to beat in your own logs: measure how much history your agent really needs per answer before buying a bigger window.

Source