ReAct: an agent that reasons and acts inside the same loop

“While large language models (LLMs) have demonstrated impressive capabilities across tasks in language understanding and interactive decision making, their abilities for reasoning (e.g. chain-of-thought prompting) and acting (e.g. action plan generation) have primarily been studied as separate topics.”

Shunyu Yao, Jeffrey Zhao, Dian Yu +4 · 7 authors · Princeton University and Google Research (Brain Team) · 2022 · arXiv:2210.03629

Why does an agent have to reason and act at the same time?

Opening

A language model can write an answer that sounds convincing, and still invent details when the user asks about something that just happened. An agent given a real task can plan carefully and then fail on its very first step, because it never checks what its actions returned. Both failures share one root: reasoning and acting have long been handled as two separate problems, while every real task needs the two to follow each other and correct each other. This is the problem of anyone running an agent on their own server: however good the model is, it is useless if it does not know when to stop and look something up.

What the paper does

ReAct fuses Reasoning and Acting: one model writes out reasoning traces and emits concrete actions such as API calls or environment steps, the two interleaved inside a single loop. Reasoning tells the model what information it is missing and how the plan has changed; acting puts it in touch with outside sources, such as a knowledge base or the very page it is operating on, instead of its own memory. Hallucination is then handled by actually looking things up, and exceptions are handled during planning instead of letting error accumulate across later steps. The authors apply ReAct to question answering and fact verification, plus two interactive decision-making tasks, and compare it with the strongest baselines of the time.

Findings

  1. ReAct interleaves reasoning traces and task-specific actions inside one loop rather than splitting them into separate stages; the paper’s claim is that both belong in the same loop. (we explore the use of LLMs to generate both reasoning traces and task-specific actions in an interleaved manner)

  2. Reasoning traces let the model form, track, and update an action plan and handle exceptions, so the plan does not collapse the moment a result comes back different from what was expected. (reasoning traces help the model induce, track, and update action plans as well as handle exceptions)

  3. On two interactive decision-making benchmarks, ALFWorld and WebShop, ReAct beats imitation and reinforcement learning by an absolute success rate of 34% and 10% respectively, while being prompted with only one or two in-context examples. (On two interactive decision making benchmarks (ALFWorld and WebShop), ReAct outperforms imitation and reinforcement learning methods by an absolute success rate of 34% and 10% respectively, while being prompted with only one or two in-context examples)

Caveat

The authors state no limitation in the abstract. The summary describes how the method works and what it measured, but it does not say where the method fails, which conditions it suits, or what a single loop costs in compute. Anyone after ReAct’s real boundaries has to read the full paper: taking the abstract as the whole evaluation gives a wrong picture of what this approach can do, and silence about limits is not evidence that none exist.

What this means for 5ac

ReAct’s loop is the control loop of Hermes: the agent reasons, calls a tool, reads the result, and repeats until the card on the Kanban board is done. When a card fails, the worker does not fine-tune the model; it rewrites its read of the situation in text and retries on the next run, handling the exception inside the runtime exactly the way ReAct repairs a plan with reasoning traces. Gbrain plays the part of the external lookup, standing in for the simple Wikipedia API used in the paper: the agent asks the knowledge base before answering instead of trusting the model’s own memory. Because the loop sits above the model, 5ac can swap models at the model-routing layer without rewriting how reasoning and acting are wired together.

Source