Context Is a Pipeline, Not a Window: Information Attrition in LLM Agent Harnesses

“LLM agents are judged by two numbers: the size of the context window, and how often the final answer is correct.”

Yuvraj Sehgal, Mahsa Panahandeh, Naser Ezzati-Jivan · 3 authors · Brock University and University of Ottawa · 2026 · arXiv:2610.36114

Measure agents by the context window and you miss the output being cut

Opening

Many teams measure an agent with two numbers: how big the context window is, and whether the final answer is correct. Both numbers miss one thing: how much of a tool’s output was cut before the model ever saw it. This paper measures that lost portion directly on a real tool-using agent, and the loss is bigger than most people expect.

What the paper does

The authors observe an agent that diagnoses injected faults by reading Linux kernel traces, and they measure that across 720 runs the single largest prompt used only 5.75% of the context window. Yet a third of tool results were cut before reaching the model, every cut result became broken JSON, and the cause was the harness cutting each result by slicing its JSON text at a fixed length — a method that deletes whatever data sits near the end instead of shortening every part evenly. The gap between fields was extreme: one field reached the model in only 4.7% of calls while a second field in the very same results reached it in 99.3%, and the agent’s diagnosis accuracy split along exactly that line. The authors model an agent’s context as a five-stage pipeline, replay 864 real tool results, run controlled experiments to separate the cause, and survey seven agent frameworks, finding two that cut tool output the same unsafe way.

Findings

  1. The two familiar numbers ignore a third thing: how much tool output is lost before the model ever sees it. (Both numbers miss a third thing: how much of a tool’s output is lost before the model ever sees it)

  2. Across 720 runs the largest prompt used only 5.75% of the window, yet a third of tool results were cut before reaching the model and every cut result became broken JSON. (Yet a third of all tool results (3,827 of 11,189) were cut before they reached the model, and every cut result became broken JSON)

  3. The controlled experiment shows the cut appears first as extra work rather than wrong answers: more tool calls, more computation, and yield falling from 0.61 to 0.31. (43% more tool calls, twice the computation, and yield falling from 0.61 to 0.31)

Caveat

The limitation the authors state plainly: whether the data cut truly causes the accuracy drop is still an open question. The paper proves the cut causes extra work and lost yield, but it does not close the causal chain that leads to wrong answers, so that part has to be read as an unfinished hypothesis rather than a settled result.

What this means for 5ac

This is why Gbrain should return results in chunks and Hermes should cut tool output at JSON boundaries rather than at a fixed byte length: the production cut described in the paper deletes all 128 records holding the fault’s evidence just because they sit at the end. For a self-hosted stack, the concrete job is to reconcile the number of records a tool returns with the number that actually enter the prompt, instead of watching only how much context window remains. A periodic cron can run that reconciliation every day and alert when the gap crosses an accepted threshold. And when a new skill adds a tool that returns long output, the gate before real runs should check how that tool’s output gets cut, not only how large the window is.

Source