Should your agent keep the raw events, or only what the events taught it?

Ningning Zhang et al. · 1 authors · Macau University of Science and Technology · 2026 · arXiv:2601.06377

Should your agent keep the raw events, or only what the events taught it?

Opening

A long-horizon assistant accumulates two very different things: the stream of what happened, and the handful of stable facts worth carrying forward. Most memory designs mix both into one store, so retrieval either drowns in old events or misses the detail that made a fact true. The design question is whether those layers should stay separate, and whether a stored fact is allowed to change when later evidence disagrees with it.

What the paper does

HiMem is a hierarchical long-term memory framework for long-horizon dialogues. It segments conversation into Episode Memory with a topic-aware, event-surprise dual-channel strategy, extracts stable knowledge into Note Memory through a multi-stage pipeline, links the two semantically, and revises stored knowledge from retrieval feedback through conflict-aware reconsolidation. Retrieval runs in hybrid or best-effort mode so accuracy and efficiency can be traded instead of fixed.

Findings

  1. Events and notes are built separately — Episode Memory holds segmented interaction events, while Note Memory holds stable knowledge pulled out by a multi-stage extraction pipeline.

HiMem constructs cognitively consistent Episode Memory via a Topic-Aware Event–Surprise Dual-Channel Segmentation strategy, and builds Note Memory

  1. The two are linked into one hierarchy — Semantic links between them bridge concrete interaction events and abstract knowledge, which is what lets retrieval stay efficient without dropping fidelity.

These two memory types are semantically linked to form a hierarchical structure that bridges concrete interaction events and abstract knowledge

  1. Retrieval has two gears — Hybrid and best-effort strategies let you choose where to sit between accuracy and efficiency rather than accepting one fixed cost for every query.

HiMem supports both hybrid and best-effort retrieval strategies to balance accuracy and efficiency

  1. The facts are allowed to change — Conflict-aware reconsolidation revises and supplements stored knowledge from retrieval feedback, so memory keeps evolving instead of only accumulating more of the same.

incorporates conflict-aware Memory Reconsolidation to revise and supplement stored knowledge based on retrieval feedback

Caveat

The abstract reports no numbers and no cost accounting, and its claim of consistently outperforming baselines is made against unnamed baselines, so treat this as a design pattern rather than measured evidence.

What this means for 5ac

The shape here is one a small team can actually maintain: raw events kept apart from distilled notes, with an explicit rule for revising a fact when retrieval shows it conflicts with what is stored. That last part matters most for a memory serving the same customers over months, because a stale fact keeps coming back with confidence attached. Choose the retrieval gear per workload as well: run hybrid where accuracy decides the answer, and fall back to best-effort when speed and token budget dominate the decision.

Source