Before an agent touches production data: the guardrail checklist
The risk is not that the model says something wrong. It is that the agent does something wrong — with credentials, in a system that does not have an undo.
Why agents are a different risk category
A chatbot that hallucinates produces a wrong sentence. An agent with tool access produces a side effect: a record updated, an email sent, a row deleted. The failure surface is no longer the quality of the text, it is the blast radius of the actions. That is why the controls that matter here look more like infrastructure security than like prompt engineering.
Check one: what can it actually reach
Write down every tool the agent can call and what each one can do to state. Most teams discover at this step that a read-only helper shares a credential with something that writes. Scope credentials per tool, not per agent, and prefer tokens that expire.
Check two: is there a gate on irreversible actions
Classify every action as reversible, reversible-with-effort, or irreversible. Anything in the third bucket needs a human confirmation step that cannot be bypassed by a clever instruction in the content the agent is reading. Note that the untrusted input is not only the user prompt — it is also the documents, web pages, and tool results the agent consumes.
Check three: can you replay what happened
You need a trace of every tool call with its arguments and result, retained long enough to reconstruct an incident. When something goes wrong, the question is never what the model said, it is what it did and in what order.
Check four: is there a budget
Cap steps, tokens, wall-clock time, and money per run. Runaway loops are the most common agent incident, and they are cheap to prevent. Surface the cap in the product so a user sees the limit rather than a hang.
Check five: what happens on malformed output
Define the failure path before you need it. If the agent returns an unparseable tool call, does it retry, ask the user, or stop? Silently retrying forever and silently giving up are both bad defaults; pick one deliberately and log it.
Check six: can one person turn it off
There must be a single switch that stops all agent activity without a deploy. This sounds obvious and is frequently missing, because agents tend to be deployed as features inside other systems rather than as a service with its own kill switch.
What to do about it
- Agent risk is blast radius of actions, not correctness of text
- Scope credentials per tool and put a non-bypassable gate on irreversible actions
- Retain tool-call traces with arguments and results — incidents are reconstructed from actions, not outputs
- Cap steps, tokens, time, and money per run, and build one switch that stops everything