Production AI is the discipline of running AI features reliably for real users — not the demo, but the system that stays accurate, observable, affordable and safe once real traffic, messy inputs and edge cases arrive. It depends less on which model you choose and more on the engineering around it: evaluation, observability, fallback paths, cost control and security.
Building AI That Works in Production
01
Why demos survive slides but not real users
02
Agents vs workflows: choosing the right pattern
| Reach for a workflow when | Reach for an agent when |
|---|---|
| The steps are known in advance | The path genuinely can't be pre-planned |
| You need predictable cost and latency | Open-ended reasoning is the point |
| Failures must be easy to trace | You can afford looser guarantees |
| The task is bounded and repeatable | The task is exploratory |
03
The boring infrastructure that makes AI reliable
04
Why AI should never be your source of truth
05
What an effective AI team actually looks like
All articles in this guide
01
Engineering Practice
AI model updates: the model you integrated is not the model you are running
Anthropic published a postmortem on six weeks of quality complaints caused by three separate changes, each on a different slice of traffic on a different schedule. The failure mode generalises, and most teams cannot detect it because their evaluation is less reliable than they think.
9 Oct 2026
02
Engineering Practice
Human in the loop: designing review that actually catches things
Three quarters of production AI systems depend primarily on human evaluation, which makes review the main quality control in the industry and one of the least designed. Most implementations are a person clicking approve on a screen that does not show them enough to disagree.
4 Sep 2026
03
Engineering Practice
Why your AI should never be the source of truth
Hallucination is not a bug you tune away. It is a structural property of how language models work. The teams who trust AI in production are the ones who decided, deliberately, which parts of the system the model is never allowed to decide.
29 May 2026
04
Engineering Practice
What working AI teams actually look like in 2026
Most published AI team structures describe how it should be done. This post is about what it actually looks like in the teams that are shipping: the roles, the ratios, the rhythms, and the small operational habits that separate working from theatre.
22 May 2026
05
Engineering Practice
The case for boring AI infrastructure
The teams shipping reliable AI are not winning on model choice. They are winning on evals, observability, cost control, and the fallback paths that fire when the interesting parts break.
24 Apr 2026
06
Engineering Practice
Agents vs. workflows: choosing the right pattern in 2026
Most things sold as agents are workflows wearing a costume. Knowing the difference matters because the trade-offs (cost, latency, debuggability, reliability) go in opposite directions.
10 Apr 2026
07
AI in Production
Why your RAG demo doesn't survive contact with real users
A working demo with five curated documents is not a working system. Here is what breaks when real users, real documents, and real load arrive, and how to build for it from the start.
3 Apr 2026
Frequently asked questions
Why do AI demos work but fail in production?
Demos run on clean, expected inputs. Real users bring messy, unexpected ones, and without evaluation, observability and fallbacks the failures are invisible until a customer hits them. Why your RAG demo doesn't survive real users
Do I need AI "agents" or a simpler workflow?
Usually a workflow. Most problems sold as "agent" use cases are solved more reliably and cheaply by a bounded workflow; reserve agents for genuinely open-ended tasks. Agents vs workflows: choosing the right pattern
What makes an AI feature reliable?
The unglamorous parts: evaluation harnesses, observability, cost controls and a fallback for when the model is wrong — not a cleverer model. The case for boring AI infrastructure
Can AI be our source of truth?
No. AI should draft, suggest and retrieve, but your systems of record stay authoritative and a human stays accountable for the decision. Why your AI should never be the source of truth
Shipping an AI feature that has to hold up in production?
A short call to pressure-test your AI integration — evaluation, observability, fallbacks and cost. No pitch, just a senior read on the risks.
- Format 60-min video call
- Output Honest read
- Commitment None required