There is a moment in most AI projects where a small decision gets made quickly and locks in eighteen months of work. The team agrees to "just stand up a basic gateway, with logging, and a simple eval harness, and a vector store, and a small retrieval service" — and by the time anyone has done the maths on the cost of operating all of that, the decision is past the point where it can be cleanly reversed.
This post is about how to make that decision deliberately, before it gets made by default. The short version of the argument: most AI infrastructure should be bought rather than built, but the question is more useful when you separate "infrastructure" into its parts and ask which parts you are actually competing on. The boring parts you should buy. The unique parts you should build.
That sounds obvious. In practice it is not how most teams approach the decision, and the evidence on what it costs to get this wrong is now sharp.
What the published numbers actually say
MIT's NANDA initiative published a finding in mid-2025 that has not received as much follow-up as it deserves. In their study of how enterprises were adopting AI, they found that buying AI tools from specialised vendors and building partnerships succeeded about 67% of the time, while internal builds succeeded only about one-third as often. That is not a small difference. It is the difference between a strategy and a graveyard.
The pattern is consistent across other published research. A 2026 enterprise survey summarised by CMARIX put the figure at over 70% of enterprises adopting third-party AI platforms to reduce initial engineering effort, with many later facing limits in customisation, system interoperability, and data control. The standard recommendation from these analyses is broadly the same: start with a bought or API-integrated platform, layer your own domain on top, and build in-house only the parts where you genuinely compete.
The total cost of ownership picture, when you actually count, is sobering for the build-everything camp. A widely-cited Hyperion Consulting analysis observes that most build-versus-buy analyses compare upfront costs — developer salaries against licence fees — and miss 60-80% of total cost of ownership. Building AI has hidden costs: infrastructure, MLOps, monitoring, maintenance, retraining, talent retention, opportunity cost. Buying has hidden costs too: customisation, integration, vendor lock-in, data migration. But the build-side hidden costs tend to be larger, in part because they are recurring and operational rather than one-off and capital.
A concrete, specific number on this from a different angle: a recent build-versus-buy analysis on AI gateways alone — the layer that sits between your application and the model providers — found that a self-hosted, production-grade implementation typically runs $2,000 to $3,500 per month in total cost of ownership once you stop excluding the parts that hurt to count. That is for one component, of one team's stack, indefinitely.
These numbers will look different in your specific situation. The pattern they describe will not.
The build-everything instinct is older than AI
Before going further it's worth naming where the instinct to build comes from, because it is older than the current cycle and it is often genuinely well-intentioned.
Software teams tend to overestimate the cost of dependencies and underestimate the cost of their own code. A dependency has a price tag attached. The code we wrote ourselves looks free, even though it is the most expensive code in the system — we will pay for it every quarter for years, in maintenance, in upgrades, in the cognitive overhead of the engineers who have to keep its shape in their heads. The bill never comes in one line on an invoice, so we never see it.
There are also good reasons to build. Differentiation. Data sensitivity that genuinely cannot leave a controlled environment. Regulatory constraints that no vendor satisfies. A specific performance characteristic that no off-the-shelf component meets. These are real, and we will come back to them.
The mistake is when the build-everything instinct overruns the build-deliberately discipline, and a team ends up owning operational complexity that produces no advantage. The instinct says "if we build it, we control it." The reality, two years later, is that the team controls something that takes 30% of an engineer's calendar to keep running, has been duct-taped to a new model provider three times, and is, on every dimension the user cares about, worse than the commercial option that costs less than the engineer.
The "boring vs unique" cut
The clearer cut to make, when looking at any AI infrastructure component, is: is this where we compete, or is this where we operate?
The competitive part of an AI product is your specific data and the specific way you act on it. Your taxonomy, your domain logic, your particular user experience, the proprietary signal you've built up that no one else has. This is what your customers pay you for, and it is where you should invest engineering time deliberately. It is also where you have to build, because nobody can sell you the unique parts of your own business.
The operational part of an AI product is everything underneath that: the gateway between your application and the model providers, the observability that tells you what is happening in production, the cost gates that stop the bill running away, the eval harness that catches regressions, the retrieval infrastructure that holds your knowledge, the audit log that satisfies the regulator, the access controls that keep the right data away from the wrong users. None of this is your differentiator. It is identical to what the company down the road needs. The question is only whether you want to be the one operating it.
The boring-vs-unique cut tends to produce a roughly stable picture across teams. The operational layer — gateway, observability, cost control, eval infrastructure, retrieval plumbing, the audit and governance scaffolding — usually wants to be bought. The application layer — the specific feature your users interact with, the prompts tuned to your domain, the way you compose retrieved knowledge into an answer, the workflow that wraps it — usually wants to be built. The data itself is yours, owned by you, accessed through whatever layer you choose.
This is not a hot take. It is what every mature engineering org concluded about web infrastructure twenty years ago. Nobody runs their own Linux kernel, their own DNS, or their own CDN unless they have specific competitive reasons to. The boring-vs-unique cut for AI looks structurally identical. It is just earlier in the cycle.
A practical framework
For teams making this decision, the version of the framework we use looks like this:
1. List every layer of the AI stack. Application, prompt management, retrieval, vector store, eval harness, observability, cost gates, gateway, access control, audit logging, model fine-tuning if any, data pipelines, feedback collection. Be honest about what you actually need.
2. For each layer, ask: does this differentiate us? Specifically: would a customer notice the difference if we used a generic version of this layer? If the answer is no, that layer is operational. If the answer is yes, ask why — and be sceptical, because most teams overestimate how much their custom-built operational layer differentiates them.
3. For each operational layer, model the three-year TCO of building versus buying. The Hyperion Consulting framework is a useful starting point. Count infrastructure, engineering time at fully-loaded cost (a senior engineer in 2026 is significantly more expensive than the AWS bill), maintenance, monitoring, retraining cycles, and the talent retention risk — AI engineers turn over every 18-24 months in most markets, which means the build option carries a knowledge-loss tax that the buy option does not.
4. For each operational layer where the buy-side total cost is materially lower, default to buying. Reverse the default only when there is a specific reason — regulatory, data-residency, performance, or a genuine differentiation argument that survives scepticism.
5. For each application-layer component, default to building. This is where your engineers should be spending their time. Owning the prompt, the retrieval composition, the user flow, the domain logic — these are the things you sell.
The output of this exercise is a stack diagram with two colours: bought, built. The boring parts are bought. The unique parts are built. Engineering time goes to the unique parts.
The objections worth taking seriously
A few honest counter-arguments deserve naming, because they sometimes apply.
"We need full control of the data." Often true. The response is not "build everything," it is "buy components that respect the data boundary." A reputable AI platform that runs in your VPC, processes data without retaining it, and provides full audit trails is materially different from one that ships your data to a black box. Specify the boundary, then evaluate vendors against it.
"The buy option doesn't do exactly what we need." Sometimes true. More often the buy option does 80% of what you need at 20% of the cost of building. The right question is whether the missing 20% is competitive value or is operational, and whether a thin layer on top of the bought component is cheaper than rebuilding the whole thing. Usually it is.
"We have the engineering talent to build it." Possibly. The cost of having that talent build operational infrastructure is the cost of them not building your application layer. That is the cost most teams forget to count. Engineering capacity is finite. Spending it on the parts of the stack that don't differentiate is a strategic choice, and one that's usually made by default rather than deliberately.
"Vendors will lock us in." A reasonable concern. The mitigation is the standard one: define data and configuration boundaries that can be re-platformed, prefer vendors with open standards and clear export paths, avoid vendor-specific abstractions in your own code, and treat any operational component as replaceable in principle. Lock-in is reduced by architecture choices, not by refusing to use vendors.
What this means for product teams in 2026
The teams pulling ahead on AI in 2026 are not the teams with the most engineering on the operational layer. They are the teams who decided early which parts of the stack were theirs to build and which parts were rented, and who put their engineering time where the unique value sits. That is not a contrarian position any more. It is what the published data is showing, repeatedly.
The framing we keep coming back to with clients: spend engineering time on the parts of the product your customers can see. Buy the parts they cannot. Your customers do not care whether your gateway is self-hosted. They care whether your AI feature actually works for them. The strategic question is which problem you are solving.
There is, finally, a quieter point underneath all of this that matters more than the build-versus-buy calculus on any single component. Most teams discover the cost of building too late — past month nine, past the first model provider migration, past the first regulator question that requires three weeks of forensic log spelunking. The framework above is most useful applied early, when the choices are still cheap to reverse. By month nine the decision has been made by default, and the only honest options are to keep paying for the build or to migrate at a cost that should have been the budget for the migration in the first place.
The teams that ship AI features that earn their place tend to share one operating principle: build the unique, buy the boring, and be honest about which is which.
References
- MIT NANDA Initiative — The GenAI Divide: State of AI in Business 2025, on adoption mode success rates. https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/
- CMARIX — Build vs Buy AI Software: The CTO's 2026 Guide. https://www.cmarix.com/blog/build-vs-buy-ai-software/
- Aisera — Build vs Buy AI Agents: Complete Guide to Adopt AI (2026). https://aisera.com/blog/build-vs-buy-ai/
- Hyperion Consulting — Build vs. Buy AI: The Total Cost of Ownership Framework. https://hyperion-consulting.io/en/insights/build-vs-buy-ai-total-cost-of-ownership
- Tian Pan — Build vs Buy for the AI Gateway (production TCO analysis). https://tianpan.co/blog/2026-05-14-ai-gateway-build-vs-buy-18-month-decision
- Keyhole Software — AI Software Development Costs 2026. https://keyholesoftware.com/ai-software-development-cost-2026/
- Hyperscience — Build vs Buy: Rethinking the Total Cost of Ownership for IDP in the Age of AI and Automation. https://www.hyperscience.ai/blog/build-vs-buy-rethinking-the-total-cost-of-ownership-for-idp-in-the-age-of-ai-and-automation/
- Compyl — Build vs. Buy a GRC Platform in 2026 (TCO analysis methodology, transferable to AI infrastructure). https://compyl.com/blog/build-vs-buy-grc-platform-2026/
Written by JP, Sixees Labs. Last reviewed May 2026.