Agents Need Explainable Traces, Not Just More Logs

Once an agent calls tools, changes state, and replans, conventional request logs are no longer enough to explain success or failure.

阅读中文版 →
On this page

When a conventional API fails, we inspect the request, response, exception, and latency. An agent fails as a process. It may misunderstand the goal, choose the correct tool with invalid arguments, or complete the task and then take one unnecessary action.

Agent observability must therefore explain not only whether an error occurred, but what the system knew, what it did, and why it continued.

Model a run as state changes

I prefer structured events over one long text log. Each event should retain the run identifier, step, state, model and tool version, input and output summaries, latency, cost, and the state that follows.

{
  "run_id": "run_20261031_0042",
  "step": 3,
  "event": "tool.completed",
  "tool": "search_documents",
  "latency_ms": 428,
  "result_count": 8,
  "state_after": "evidence_ready"
}

Structured events can be aggregated, compared, and replayed. A long log can usually only be searched by eye.

Keep the information that supports a decision

More data is not automatically better. Persisting every prompt and response creates privacy, cost, and noise problems. Prompt versions, tool names, parameter shapes, result summaries, state changes, and explicit error categories are often enough.

Sensitive content can be represented by hashes, lengths, types, or redacted summaries. Observability exists to explain behavior, not to duplicate user data indefinitely.

Give failures useful names

agent_failed is not an actionable category. A practical taxonomy might distinguish goal misunderstanding, planning loops, wrong tool choice, invalid arguments, external service failures, unsupported inference, failure to stop, and permission blocks.

The taxonomy can evolve, but it should tell the team whether the next change belongs in the model, prompt, tool, or system boundary.

Turn traces into evaluation cases

Production traces reveal recurring problems. Those problems become evaluation cases, and the evaluation suite becomes a release constraint. Over time we can compare tool calls, loops, cost, latency, and completion rates across versions.

That feedback loop is often more valuable than adding another layer of apparent intelligence.

RELATED WRITING

DISCUSSION

Discussion