A significant shift is underway in corporate America's approach to artificial intelligence spending, as CFOs and boards intensify scrutiny over AI budgets that have ballooned beyond expectations. The prevailing strategy of defaulting all tasks to the most advanced and expensive AI models is being replaced by 'model routing,' a method that matches the complexity of a task to an appropriately priced AI model. This change is driven by the realization that most enterprise AI usage—estimated at 95% by Glean CEO Arvind Jain—still relies on costly frontier models, even for simple tasks that could be handled by cheaper alternatives [1].
Scott Wu, CEO of Cognition, highlighted that for routine work, companies can achieve five to ten times better cost efficiency by using less expensive models that are still adequate for the job. He illustrated this with the example of asking an AI model to name the third U.S. president, a task that does not require the most advanced technology. The financial implications are substantial: Jeetu Patel, chief product officer at Cisco, calculated that at $200 of token usage per employee per week, annual costs can reach $10,000 per person. For a company with 90,000 employees, this translates to $900 million annually, a figure that forced Cisco to reallocate resources and prioritize AI token spending [1].
AI vendors are feeling the pressure from this new cost discipline. Cognition has responded by introducing an AI productivity guarantee for its coding agent Devin, promising to fund up to $10 million in usage if the product does not deliver engineering value commensurate with its cost. Wu emphasized the importance of measuring output—such as human engineering hours saved—over mere activity like tokens consumed, stating, 'You can spend billions of tokens and be doing nothing with it.' This approach aims to address industry concerns about return on investment [1].
The adoption of model routing poses a direct challenge to leading AI providers OpenAI and Anthropic. If companies increasingly route simple, high-volume tasks to cheaper open-source models, these vendors risk losing revenue from routine work and will only be compensated for handling more complex jobs [1].
CONCLUSION
The move toward model routing marks a pivotal change in AI spending, with major cost-saving potential for enterprises and significant revenue implications for leading AI vendors like OpenAI and Anthropic. As companies prioritize efficiency and measurable output, the AI market may see a shift in demand toward more cost-effective solutions.