Most agents break in production for boring reasons. Someone wired one to a live database with write access "just to test." A retry loop issued the same refund three times. Nobody could explain why it emailed a customer at 2 a.m.
The model is rarely the weak link — the plumbing around it is. What follows are twelve checks worth running before you point an agent at anything that matters: money, customer data, or systems other people depend on.
1. Map Every Permission Before You Grant One
Write out every credential, API key, database role, and file path the agent can reach. Then cross out anything the workflow does not strictly need. A support agent that reads order history does not need the billing admin key. If it must issue refunds, give it a tool that takes an order ID and a capped amount — not raw access to the payment provider.
- Read-only by default. Add write scope per tool, one at a time, with a reason written down.
- One tool, one job. A tool that "runs any query" is a permission bypass wearing a friendly name.
- Separate identities per environment. Staging credentials should be useless against production, and vice versa.
- Time-box secrets. Short-lived tokens beat static keys that sit in a config file for two years.
2. Decide What It Can See and What It May Keep
Context is data. Everything you paste into a prompt — ticket threads, documents, CRM records — becomes something the agent might echo back, store in memory, or send to a third-party endpoint. Redact before the prompt, not after the incident.
Then define retention for each place data lands: prompt logs, tool outputs, vector stores, long-term memory. Set expiry dates and honor them. If you handle health, financial, or legal records, have someone qualified review your setup against the rules that apply where you operate; getting this wrong is expensive in ways a checklist cannot fix.
3. Assume Someone Will Talk Your Agent Into Trouble
Anything the agent reads is untrusted input. A web page, a shared document, a support ticket body — each can carry text that looks like an instruction. You cannot filter your way out of this completely, so design so that a hijacked agent is boring rather than catastrophic.
Treat retrieved content as data, never as commands. Validate tool arguments against a schema before execution. Keep secrets out of anything the agent can read. And for irreversible actions, require a confirmation step the model cannot manufacture on its own.
4. Log Enough to Reconstruct Any Decision
When something goes wrong at 3 a.m., you want to rebuild the run without guessing. Capture who or what triggered it, which tools were called, the exact arguments, what came back, the final output, latency, cost, and the version identifiers for prompt, model, and tool set.
Mask or hash secrets and personal data at write time — a log file is one of the easiest places to leak information by accident. Make the logs queryable by conversation ID so a support agent can pull up a single run in seconds. If a probe takes an hour, people stop probing.
5. Build the Undo Button Before You Need It
Rollback is a design decision, not an emergency plan. Put the agent behind a feature flag. Add a kill switch that stops new runs without breaking in-flight ones. Roll out to a small slice of traffic first and compare behavior before widening.
For side effects, prefer reversible ones. Use idempotency keys so retries do not duplicate work. Queue outbound messages and emails with a short delay so a human can cancel. Where reversal is impossible — a payment sent, a record deleted — say so explicitly in your runbook and gate that action behind approval.
Write down who can flip the switch and exactly how. Then test it in staging, once, on a calm afternoon. An untested kill switch is decoration.
6. Decide Who Is Watching, and When
Match autonomy to blast radius. Reading a dashboard can run unattended. Spending money, changing records, contacting customers, or touching production infrastructure should not. Design approvals so they are quick to review: show the exact action, the arguments, and a one-line rationale. A reviewer who has to dig through raw logs will start approving on reflex within a week.
Name an owner. Name a backup. Give both an escalation path and the authority to pause the agent without asking permission. Rotate reviewers so the queue never becomes a rubber stamp, and track how often approvals get reversed — that number tells you whether oversight is real.
The Night Before Launch
Run this short pass once more with the team in the room:
- Can we revoke every credential in under five minutes?
- Do we know what happens on a timeout, an empty response, or a malformed tool result?
- Is there a named owner, a backup, and a written escalation path?
- Can we replay last week's traffic against the new prompt before swapping it in?
- Does someone outside the build team know how to shut it off?
If any answer is "probably," fix it before you ship. Then start small, watch the first week closely, and keep the checklist in the repository. Re-run it whenever the tool set, the model, the data sources, or the people on call change — because the step that gets skipped is always the one that ends up mattering.
Photo: panumas nikhomkhai / Pexels
