There’s a pattern we see over and over again. A team builds an impressive AI demo — maybe it drafts emails, summarizes documents, or navigates a browser. The demo works beautifully in a meeting. Then it’s handed off to engineering, and everything falls apart.

The demo-to-production gap

The problem isn’t the model. It’s everything around it.

Most AI demos succeed because they operate in controlled conditions: curated inputs, predictable edge cases, a human watching the output. Production is the opposite. Inputs are messy. Edge cases are the norm. Nobody’s watching.

We’ve found that the model is usually the least interesting part of a production agent. What matters is the harness — the tools, guardrails, evaluation loops, and context architecture that surround it.

Three failure modes

1. The tool-shape problem

Most agent frameworks encourage developers to wrap APIs as “tools” and hand them to a model. But the shape of an API is rarely the shape of an intention. A CRM API has endpoints for CRUD operations on contacts, deals, and activities. But the intention is usually something like “follow up with everyone who attended the webinar but didn’t book a demo.”

When you give a model tools shaped around API structure rather than intention, it has to figure out the mapping itself. That’s where things break.

2. The guardrail gap

Demo agents rarely have guardrails because they don’t need them. Production agents need guardrails everywhere: input validation, output verification, budget limits, rate limits, escalation paths, and fallback strategies.

The hard part isn’t adding guardrails — it’s knowing where to put them. That requires understanding the domain deeply enough to anticipate what can go wrong.

3. The context problem

Models are only as good as the context they receive. Most agent systems dump everything into a prompt and hope for the best. Production systems need progressive disclosure — surfacing the right information at the right time, in the right format.

This is an architecture problem, not a prompt engineering problem.

What actually works

The teams that succeed in production share a few traits:

  • They specify before they automate. They write down the decision heuristics that humans use — the Given-When-Then scenarios — before they try to teach them to a model.
  • They design the harness first. Tools, guardrails, evaluation criteria, and context architecture are designed before the first line of agent code is written.
  • They deploy narrow. They start with a small pilot group on low-risk work, observe how the system performs, and expand only when the evidence supports it.
  • They make reasoning visible. Every agent decision is logged, traceable, and auditable. When something goes wrong, they can see exactly why.

The gap between demo and production is real, but it’s not insurmountable. It just requires treating AI agents as serious engineering projects, not magic tricks.