τ-bench: measuring whether tool-using agents follow domain rules, and stay consistent across runs
“Existing benchmarks do not test language agents on their interaction with human users or ability to follow domain-specific rules, both of which are vital for deploying them in real world applications.”
An agent calls tools by the rules once: what happens on the second run?
Opening
Existing benchmarks rarely test two things that decide whether an agent can go into a real application: how it interacts with a human user, and whether it follows the rules specific to a domain. An agent can sound fluent in a demo and then break the rule the moment a customer asks the question in a way the author did not anticipate. An agent that fails that way does not produce a wrong answer in a test set; it produces a wrong action in front of a customer. τ-bench was built to measure both, through real conversations and through the final state of the data.
What the paper does
τ-bench emulates a dynamic conversation between a user, played by a language model, and an agent equipped with domain-specific API tools and a set of policy guidelines. Scoring compares the database state at the end of the conversation directly against an annotated goal state, instead of reading the answers by eye. Alongside it the authors propose a new metric, pass^k, to measure how reliable an agent’s behavior is across repeated runs of the same task. They run experiments with modern function-calling agents and report the results.
Findings
-
τ-bench emulates dynamic conversations between a language-model-simulated user and an agent given domain-specific API tools and policy guidelines, two things prior benchmarks did not test. (a benchmark emulating dynamic conversations between a user (simulated by language models) and a language agent provided with domain-specific API tools and policy guidelines)
-
Scoring compares the database state at the end of the conversation with the annotated goal state, a process the authors describe as efficient and faithful. (We employ an efficient and faithful evaluation process that compares the database state at the end of a conversation with the annotated goal state)
-
The pass^k metric exists to measure the reliability of agent behavior over multiple trials, instead of treating one correct run as enough. (We also propose a new metric (pass^k) to evaluate the reliability of agent behavior over multiple trials)
Caveat
The abstract does not state a limitation of the benchmark itself; the result the authors give is a finding about agents, namely that even the leading function-calling agents such as gpt-4o succeed on fewer than 50% of the tasks and are quite inconsistent, with pass^8 below 25% in retail. Read correctly, these are measured numbers about agent capability and stability at publication time, and they are precisely why the pass^k metric was proposed.
What this means for 5ac
This is the lesson closest to how 5ac’s Kanban agents run repeatedly: a card handled correctly once proves nothing, because the next run may come out different when the prompt, the tool output, or the sampling changes. The pass^k metric translates directly into 5ac practice: for anything important, run the verification several times and only accept the result when every run passes. It also explains why re-scanning cron jobs and gates that loop over the same check are necessary, and why the same card type must be scored by one script instead of a different method each time. While an agent is not stable across runs, every promise about automation is fragile. We treat stability as a property of the process rather than of the model: a card that needs high reliability gets extra scoring passes before it moves to done, while quick trials are not held up by that rule.
Source
- τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains — https://arxiv.org/abs/2406.12045