Netomi runs millions of autonomous customer service interactions per month on GPT-4 and advanced reasoning models. Model ML rebuilt financial services infrastructure from the ground up for agent capabilities. Yet most companies building agentic AI can't reliably execute a three-step workflow without manual intervention. The gap isn't intelligence—it's infrastructure.

The agentic AI wave will follow the microservices pattern with mathematical precision: early adopters will drown in operational complexity while infrastructure winners capture the majority of market value. The companies building concurrency management, governance frameworks, and multi-step orchestration layers—not the agent applications themselves—will define which enterprises actually scale AI autonomy. We're watching 2014's microservices transition replay at 2x speed, and most founders are building in the wrong layer.

The Production Wall: Where Agent Demos Die

The conversation about agentic AI obsesses over capabilities—reasoning depth, tool use sophistication, planning accuracy. This is theater. Netomi's production system reveals what actually breaks at scale: concurrency management when 50 agents simultaneously hit the same rate-limited API, governance failures when one rogue agent's bad decision compounds across interconnected workflows, and silent reasoning failures in multi-step chains that existing observability tools can't even detect.

The math is brutal. If each step in a five-step reasoning chain has 95% accuracy—which would be exceptional for current systems—the compound success rate is 77%. That's a 23% failure rate on tasks that feel trivial in demos. Now add concurrency: when you're running hundreds of agents in parallel, those failures cascade. One agent exhausts an API quota, triggering rate limits that block 47 other agents mid-workflow. The system doesn't just fail—it fails chaotically, in ways that are nearly impossible to debug with traditional monitoring.

Netomi solved this by building custom concurrency arbitration layers that queue and throttle agent API calls, preventing the cascade failures that killed their first production deployment. They built reasoning trace logs that capture every decision point in multi-step workflows, because standard observability tools like Datadog only see the API calls, not the chain of thought that led to them. And they implemented governance checkpoints where human reviewers audit high-stakes decisions before agents execute—not because the agents aren't smart enough, but because 95% accuracy isn't acceptable when you're processing customer requests at scale.

This is the production wall. Every company scaling agents hits it. The difference is whether you build this infrastructure in-house, spend $2M and eighteen months learning what Netomi learned, or wait for the infrastructure layer to mature. Most won't have the luxury of time.

The Missing Stack: What Sits Between GPT-4 and Real Work

There are four infrastructure layers that don't exist yet as mature products but will define the next generation of $10B+ companies:

Stateful orchestration. Agents need persistent context across sessions—not just conversation history, but workflow state, resource allocations, and decision provenance. Amazon Bedrock's new stateful runtime is an early indicator of this category, but the winning platform will handle multi-day workflows where agents hand off tasks, pause for human approval, and resume execution across model updates. This isn't a database problem; it's a distributed systems problem with AI-specific failure modes.

Multi-agent coordination primitives. OpenAI's multi-agent research hints at what's missing: protocol layers for agent communication, resource competition, and collaborative task decomposition. When you have 20 agents working on interconnected workflows, they need something analogous to service mesh for microservices—a coordination layer that handles discovery, communication, and conflict resolution. The company that builds this won't sell agents; they'll sell the substrate that makes multi-agent systems possible.

Governance as code. Enterprises will require programmatic boundaries on autonomous agents: spending limits, approval thresholds for irreversible actions, rollback capabilities for chains of decisions. OpenAI's governance practices documentation provides the blueprint, but the productized version doesn't exist yet. This will look like infrastructure-as-code but for agent behavior—version-controlled policies that define what agents can and cannot do, enforceable at runtime across every interaction.

Agentic observability. Datadog and New Relic trace API calls and model outputs. They don't trace reasoning chains. When a five-step agent workflow fails at step three, current tools tell you that it failed, not why the agent chose that path or where in the reasoning chain the logic broke down. The winning observability platform will instrument the agent's decision-making process itself—capturing tool selection rationale, confidence scores at each step, and the causal graph of multi-agent interactions. This is a $5B+ category that barely exists today.

Each gap represents a venture-scale opportunity. More importantly, these gaps are why most agent applications will fail to scale.

The Financial Services Rebuild: Why Agent Capability Required Infrastructure Overhaul

Model ML's work with financial firms reveals something crucial: increased model capability forced architectural changes, not just API upgrades. Financial services companies aren't swapping GPT-3.5 for GPT-4o in existing workflows—they're rebuilding infrastructure from the ground up because autonomous action requires fundamentally different state management, error handling, and control patterns than co-pilot features.

The specific changes are telling. Financial workflows moved from stateless API calls (where each agent interaction is independent) to persistent agent sessions that maintain context across days or weeks. This isn't about conversation memory—it's about operational state. An agent analyzing loan applications needs to track which documents it's reviewed, which third-party APIs it's queried, and which decisions are pending human approval, all while maintaining audit trails for regulatory compliance.

Error handling transforms when agents act autonomously. With co-pilot tools, users review every suggestion before execution. With autonomous agents, you need rollback capabilities and approval gates after the agent acts—essentially distributed transactions for AI decisions. Model ML built systems where agents can execute 90% of a workflow autonomously but flag high-stakes decisions (loan approvals above $500K, trades exceeding risk thresholds) for human review before finalizing. The infrastructure that enables this—decision provenance, partial rollback, conditional execution—doesn't exist in standard ML platforms.

The control paradox: more capable agents require more sophisticated human override mechanisms, not fewer. As agents handle increasingly complex tasks, the blast radius of mistakes grows, which means enterprises need granular control over agent autonomy levels, real-time intervention capabilities, and the ability to pause agent actions mid-workflow. This is why Model ML's clients invest millions in infrastructure before deploying agents broadly. The alternative is building production systems on scaffolding designed for demos.

The Microservices Parallel: Why History Predicts Infrastructure Winners

The agentic AI transition mirrors 2014-2018 microservices adoption with eerie precision. Everyone wanted the benefits—scalability, modularity, team autonomy. Most ignored the operational tax—service mesh complexity, distributed tracing challenges, chaos engineering requirements. And infrastructure companies captured outsized value.

The pattern was clear: in 2014, every startup wanted microservices. By 2016, those companies were drowning in operational complexity—debugging cascading failures across 50 services, managing deployment orchestration, and implementing distributed tracing just to understand system behavior. By 2018, the infrastructure layer matured: Kubernetes standardized orchestration, Datadog productized observability, HashiCorp built service mesh primitives. The companies that built those infrastructure layers became worth more than most of the applications running on microservices.

Agents will follow the same four-year cycle. We're in 2026—the equivalent of 2015 in the microservices timeline. Early adopters like Netomi and Model ML are building custom infrastructure because the standardized layer doesn't exist yet. By 2027, second-wave enterprise adopters will hit production walls and demand productized solutions. By 2028, the infrastructure winners will be clear, and they'll capture the majority of market value.

The specific parallel is striking: Kubernetes became more valuable than most container-based applications because orchestration is a universal requirement. Agent orchestration platforms—systems that manage stateful workflows, coordinate multi-agent interactions, and handle governance at scale—will follow the same trajectory. Datadog's observability platform became essential because debugging distributed systems required new instrumentation. Agent-native observability platforms that trace reasoning chains and multi-agent causality will be equally critical and equally valuable.

My prediction: by 2028, the leading agent orchestration platform and the leading agentic observability company will each be worth $5B+. The application layer will be fragmented across thousands of vertical-specific solutions, but the infrastructure layer will consolidate around 2-3 dominant platforms per category.

Evaluating Agents vs. Evaluating Models: The Benchmark Gap

Existing AI benchmarks measure model intelligence on isolated tasks. MMLU tests knowledge breadth. HumanEval tests code generation. These benchmarks predict model capability but say nothing about agent production success, which depends on multi-step reliability, resource management under concurrency, and graceful degradation—qualities that require entirely different evaluation frameworks.

IBM Research's Open Agent Leaderboard represents a first attempt at true agent evaluation. Instead of testing single-turn accuracy, it evaluates multi-step task completion: can the agent decompose a complex request, select appropriate tools, handle API failures mid-workflow, and recover from reasoning errors? The results are revealing—models that score 90%+ on traditional benchmarks often achieve 60% or lower on multi-step agent tasks. The delta isn't intelligence; it's operational reliability.

AssetOpsBench takes this further, bridging the gap between AI agent benchmarks and industrial reality. It tests agent behavior in production-like environments: rate-limited APIs, concurrent agent execution, partial tool failures, and ambiguous instructions. This matters because enterprises don't care if an agent is smart—they care if it's reliable under production conditions.

The standardization opportunity is massive. Enterprises will require SOC 2-equivalent certification for agentic systems by 2027, creating demand for independent evaluation frameworks that assess agent reliability, security boundaries, and governance compliance. Whoever defines these standards will shape procurement decisions at every enterprise. This is analogous to what PCI-DSS did for payments—not sexy, but extraordinarily valuable. The company that builds the certification framework for production agents could be worth $1B+ on compliance requirements alone, before considering the broader evaluation platform opportunity.

The current benchmark gap means enterprises are flying blind. They're choosing agent platforms based on model leaderboards that don't predict production success. The first infrastructure company to productize reliable agent evaluation will capture outsized strategic value.

Who Wins: The Business Case for Infrastructure

If 20% of enterprise software workflows become agentic by 2028—a conservative estimate given current adoption trajectories—the infrastructure layer could exceed $50B in total addressable market. The math: enterprise software spend is roughly $750B annually. Agent infrastructure (orchestration, governance, observability) will capture 10-15% of that spend, mirroring what infrastructure captured during the microservices transition.

The Accenture + OpenAI partnership signals something critical: services margins on agent deployment create infrastructure demand. Accenture isn't selling consulting hours to help enterprises fine-tune models; they're selling transformation projects to rebuild operational infrastructure for agentic workflows. Each enterprise deployment requires orchestration platforms, governance frameworks, and observability tools—durable infrastructure spend that recurs annually, not one-time integration fees.

Strategic positioning for founders: build for the second-wave enterprise adopters arriving in 2025-2026. Netomi and Model ML built custom infrastructure because no alternatives existed. The next 1,000 enterprises scaling agents won't have that luxury—they'll buy platforms. The winning infrastructure companies will have reference deployments at tier-one enterprises, integration partnerships with OpenAI and Anthropic, and product depth that handles the concurrency, governance, and observability challenges that first-wave adopters discovered the hard way.

Vulnerable incumbents: Datadog's observability platform doesn't trace agent reasoning chains. ServiceNow's workflow engine isn't built for autonomous agents. Palantir's Ontology handles data modeling but not multi-agent coordination. Each represents a $50B+ company with infrastructure that was state-of-the-art for pre-agent architectures. They'll adapt, but the window for agent-native challengers is wide open.

My bet: the next $10B infrastructure company will be agent-native observability—specifically, platforms that instrument reasoning chains, trace multi-agent interactions, and surface failure modes that are invisible to traditional monitoring. The founder who builds this will sell to every enterprise running agents at scale, which will be most enterprises by 2028.

The Next Eighteen Months

We're at the inflection point where demos become deployments and deployment pain creates infrastructure demand. The technical founders who recognize this pattern—that agentic AI will replay microservices economics with the infrastructure layer capturing majority value—have an eighteen-month window to build the orchestration, governance, and observability platforms that will define the next decade of enterprise AI.

The application layer will be crowded. The infrastructure layer will be consolidated and extraordinarily valuable. History doesn't repeat, but it rhymes with precision.

Key Takeaway: The companies building agent orchestration, governance, and observability infrastructure—not the agent applications themselves—will capture the majority of value in the projected $50B+ agentic AI market by 2028, following the exact pattern where Kubernetes and Datadog became more valuable than most microservices applications.