The Architecture Behind Cost-Effective AI Agents

InnovationThe Architecture Behind Cost-Effective AI AgentsByAruna Veerappan,Forbes Councils Member.for Forbes Technology CouncilCOUNCIL POSTExpertise from Forbes Councils members, operated under license. Opinions expressed are those of the author. | Membership (fee-based)May 22, 2026, 08:45am EDTAruna Veerappan is Senior Director of Engineering at Upwork, leading Developer Enablement to reduce friction and boost team productivity. gettyEngineering leaders are discovering that the hardest part of AI agents isn't the AI—it's the architecture underneath.I learned this firsthand when a quarterly budget disappeared in weeks. Nothing was broken, the models worked and the engineers were strong. But the system hadn't been designed for cost, and the bill arrived before a single workflow reached production.The root cause: we were pointing expensive models at every task. Verifying file existence. Checking ownership against APIs. Routing logic that could have been a single if-statement. Each call seemed reasonable. The cumulative cost was not.I've come to call this the Agent Cost Spiral—and engineering teams across the industry are running into it right now."An Agent Cost Spiral isn't an AI problem. It's an architecture problem. And once you see it, you can't unsee it."This pattern has a precedent. A decade ago, teams migrated to the cloud chasing savings, then watched their bills explode past on-premise costs. The architecture was the problem—not the technology. AI inference costs follow the same arc. The fix: stop treating it like a utility and start treating it like an engineering problem.​Tiered Architecture Every Agentic System NeedsA well-built AI agent isn't a single model receiving a single prompt. It's a choreographed system where each task is matched to the minimum level of intelligence required to complete it well.Tier 1: The Deterministic Skeleton—Just Use CodeIf your process follows a fixed rule—"if a cust...