Martin Fowler published Building reliable agentic AI systems with Bayer — a structured take on what it takes to run autonomous LLM agents in production, not as demos. The thesis is straightforward: agents fail differently from traditional services. They are non-deterministic, tool-dependent, and sensitive to prompt drift. Reliability engineering for agents borrows from SRE and ML ops, but needs its own eval and monitoring vocabulary.
This article distills the patterns that matter for teams shipping agent workflows today.
Agents are loops, not requests
A single HTTP request either succeeds or fails with a status code. An agent run is a loop: plan, call tool, observe result, replan, call tool again — until a stop condition or a budget expires. Failures can be partial: three tools succeed, the fourth hallucinates a parameter, the fifth retries into a rate limit.
Reliability starts by treating the loop as the unit of observation, not individual model calls. Metrics that only track token latency miss the failure mode users actually see: a task that ran for four minutes and produced wrong output.
Think of an agent like a contractor with subcontractors. The job isn't done when one subcontractor shows up on time — it's done when the whole chain delivers a correct result within budget.
Eval sets before architecture debates
The article emphasizes domain-honest eval sets — fixed task suites that represent real production work, not cherry-picked demos. Before choosing a model or a framework, you need to know: on our tasks, with our tools, what success rate do we get?
Eval design for agents includes:
- Task completion — did the agent achieve the stated goal?
- Tool correctness — were the right tools called with valid arguments?
- Safety — did the agent refuse or escalate appropriately on out-of-scope requests?
- Cost and latency — how many steps and tokens per successful run?
Without evals, every model upgrade is a guess. With evals, regressions show up before users do.
You cannot reliability-engineer what you cannot measure on tasks that resemble production.
Monitoring beyond token counts
Production agent monitoring needs traces, not just aggregates. Each run should log: prompt version, tool calls with inputs and outputs, model identifiers, stop reason, and human override events. When a user says "the agent did something wrong Tuesday," you reconstruct the loop — not grep for 500 errors.
Alerting targets agent-specific signals:
- Success rate drop on the eval suite (continuous or sampled)
- Spike in tool errors or permission denials
- Average steps-to-completion drifting upward (often a sign of confusion or retry storms)
- Human takeover rate increasing
Traditional uptime monitoring stays necessary — the API must be available — but insufficient.
Architecture patterns that contain failure
Reliable agent systems constrain autonomy deliberately:
Bounded tool sets. Agents with access to twenty tools wander; agents with five well-typed tools succeed more often. Each tool should have a schema, validation, and deterministic behavior where possible.
Human checkpoints on irreversible actions. Send email, charge payment, delete record — gate these behind confirmation or policy checks, not model discretion alone.
Fallback to narrower modes. When confidence is low or tools fail repeatedly, degrade to "suggest only" rather than continue autonomous execution.
Idempotent tools. Retries are inevitable. Tools that double-charge or duplicate records on retry destroy trust faster than wrong text output.
| Pattern | Reliability benefit | Cost |
|---|---|---|
| Fixed eval suite | Catch regressions pre-release | Maintenance of golden tasks |
| Step-level tracing | Debug non-deterministic failures | Storage and PII handling |
| Tool schema validation | Block malformed actions before execution | Schema design upfront |
| Human gates on mutations | Prevent irreversible agent errors | Latency and UX friction |
The organizational gap
Fowler and Bayer note that reliability is as much organizational as technical. Someone must own the eval set, review failed traces, and decide when a model or prompt change is safe to ship. Agents without an owner degrade the way untested cron jobs do — silently, until an incident.
Teams that treat agents as "just another API endpoint" skip that ownership and pay for it in incident volume.
What this means for builders
Ship an eval suite before you ship the agent. Even twenty representative tasks beat zero. Run it on every prompt or model change.
Log the full tool loop. If you cannot replay a failed run step by step, you cannot fix it systematically.
Reduce tool surface area. Fewer, better-typed tools beat a general "call anything" API for reliability.
Define success per task type. "Helpful chat" is not a metric. "Correctly filed expense report with valid receipt attachment" is.
Assign reliability ownership. Name who watches eval scores and trace samples weekly — not who built the demo.
Conclusion
Agentic AI in production is reliability engineering applied to non-deterministic loops. Fowler and Bayer's framing makes the gap explicit: demos optimize for impressive single runs; production optimizes for bounded failure rates across thousands of runs with tools, users, and models that all drift over time. The teams that close that gap invest in evals, tracing, constrained tools, and human gates — the same ingredients SRE brought to distributed systems, adapted for systems that guess.