Reading ReAct: Closing the Loop Between Reasoning and Action

How interleaved Thought, Action, and Observation trajectories connect language reasoning to tools—and what production agents still need beyond the prompt.

阅读中文版 →
On this page

Paper: ReAct: Synergizing Reasoning and Acting in Language Models, Yao et al., ICLR 2023.

Language-only reasoning can continue from a false premise. An action-only policy may lack explicit task decomposition. ReAct alternates reasoning, action, and environmental feedback so that a plan can be revised against real observations.

From a single answer to a loop

A ReAct trajectory can be simplified as:

Thought: What information is still missing?
Action: Search[query]
Observation: Evidence returned by the tool
Thought: Is the evidence sufficient, and what comes next?
Action: Finish[answer]

Actions connect the model to search, knowledge bases, or interactive environments. Observations bring the external world back into context. Reasoning becomes an update over new evidence rather than a closed generation process.

Why interaction helps

Chain-of-thought can decompose a problem while still relying on an incorrect memorized fact. ReAct lets a model seek information under uncertainty and condition later steps on the result. On HotpotQA and FEVER, the paper shows how this interaction can improve factual behavior and produce interpretable task trajectories.

On the interactive ALFWorld and WebShop benchmarks, the paper reports absolute success-rate gains of 34 and 10 percentage points over the corresponding baselines, using only one or two in-context examples. Language models could organize instructions, local reasoning, and environmental actions into executable behavior.

From paper pattern to production system

Putting ReAct in a prompt is the easy part. A maintainable agent also needs:

  • small, explicit, and stable tool contracts;
  • parameter, permission, and side-effect validation;
  • step, time, and cost budgets with stopping rules;
  • structured logs for calls and observations;
  • retries, idempotency, and human takeover paths;
  • a distinction between internal model reasoning and auditable user-facing explanations.

A production system does not need to expose raw reasoning traces. Structured plan state, tool arguments, results, and decision evidence are more reliable for debugging than treating free-form text as the only state.

An important limitation

ReAct still depends heavily on prompts, tool descriptions, and environmental feedback. An ambiguous observation or dirty tool response can send the model into another loop from a corrupted state. Long trajectories also consume context and compound early mistakes.

Agent reliability therefore cannot come from a “better reasoning model” alone. Tool contracts, state control, evaluations, and observability determine the system ceiling together.

My takeaway

ReAct frames intelligence as a feedback loop: form a hypothesis, act, observe, and revise. The model chooses under uncertainty; the engineering system limits the choice space and preserves verifiable facts.

A reliable agent does not merely think for longer. It gets reality to answer after each meaningful step.

RELATED WRITING

DISCUSSION

Discussion