AI agents in production: choosing autonomy without losing control

An AI agent is not simply a chatbot with a longer instruction. It observes a situation, chooses an action, calls a tool, reads the result, and continues until it reaches a goal or asks for help. That loop enables useful behavior. It also creates more places where an error can spread.

The first design question is therefore not “which agent framework should we use?” It is how much decision-making does this task actually require from the model?

Start with the most predictable path

A single model call or deterministic workflow is often enough to extract a field, classify a document, or draft text from known inputs. When the steps are known in advance, application code can sequence them and keep their order explicit.

An agent becomes useful when the route depends on what it discovers: searching several sources, selecting a tool after reading an earlier result, repairing an attempt, or stopping to request missing information. Anthropic recommends starting with simple, composable components and adding agentic complexity only when measurement shows an improvement (Building Effective Agents, 2024).

This distinction prevents a common failure mode: wrapping a fully specified business process in an autonomous loop. The result usually costs more, takes longer, and is harder to debug without making a better decision.

MCP connects systems; it does not govern them

The Model Context Protocol standardizes how an application exposes tools, resources, and capabilities to a model. The July 28, 2026 specification introduced a stateless transport core, self-describing requests, header-based routing, and a formal extension framework (MCP 2026-07-28, 2026).

That common interface can reduce connector-specific code. It does not decide which permissions to grant, which data to disclose, or which actions require approval. Those remain application responsibilities. A production tool should have:

• an unambiguous name and description;
• a strict input schema;
• a short, structured response;
• permissions limited to its job;
• an idempotency key for replayable writes;
• observable timeouts and errors.

A search_orders tool and a refund_order tool do not carry the same risk. The second deserves human confirmation, an amount limit, and a separate audit trail.

Grant little power at first

A useful first release can use a short loop: the model selects from a small set of read-only tools, the orchestrator validates arguments, executes the call, and returns a filtered result. It also enforces maximum steps, wall-clock time, and consumption.

Actions that create a commitment — sending, paying, deleting, publishing, or changing access — go through explicit approval. This is a security boundary, not cosmetic interface work. The NIST generative AI profile recommends managing risk across the lifecycle and documenting governance, measurement, and monitoring (NIST AI 600-1, 2024; updated 2026).

Evaluate trajectories, not only final answers

An agent may produce the right answer after reading the wrong source, making a needless tool call, or attempting an unauthorized action. A test that checks only its final message will miss those failures.

Build scenarios for the normal path, missing data, unavailable tools, hostile input, and requests that require approval. For each scenario, inspect the expected outcome, tool selection, arguments, step count, consumption, latency, and authorization boundaries. Anthropic’s agent evaluation guidance similarly emphasizes examining trajectories and combining evaluation methods (Demystifying evals for AI agents, 2026).

In production, retain enough traces to diagnose behavior while removing unnecessary secrets and personal data. Track technical errors, justified refusals, abandoned runs, and human interventions separately. One overall “success rate” hides too much.

A sensible path to production

A deployment can progress through four levels:

1. a deterministic workflow with no write tools;
2. a read-only agent bounded by time and steps;
3. a few reversible writes with approval;
4. broader autonomy only for scenarios with demonstrated reliability.

At every level, compare the system with a simpler baseline. If the added autonomy does not improve quality, turnaround time, or total processing cost, it has not earned its place.

References

• Model Context Protocol, The 2026-07-28 Specification, July 28, 2026.
• Anthropic, Building Effective Agents, December 19, 2024.
• Anthropic, Demystifying evals for AI agents, January 9, 2026.
• NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, July 26, 2024; updated April 8, 2026.