Companies are implementing AI-powered routing systems due to rising agent costs
Companies are increasingly adopting AI model routers—software that selects an AI model for a specific task based on cost, speed, and performance. This trend is driven by the costs associated with autonomous AI agents, particularly programming tools like Claude Code and Codex, which can run for hours and consume millions of tokens, according to Fortune.
Unlike a typical chatbot, an autonomous agent can repeatedly access powerful models while performing a long-running task. This can significantly increase inference costs—the cost of generating responses using the model. According to a study cited by the publication, 62% of organizations reported that unexpected AI costs over the past year had a significant impact on business decisions. In 40% of these cases, the issue had to be escalated to the board of directors; 33% of companies implemented an emergency spending freeze; and 25% postponed or canceled their AI initiatives.
Routers make it possible to avoid sending every request to the most expensive state-of-the-art model. Companies can set selection rules themselves or have the system automatically determine the optimal model at each stage of an agent’s work. Industry representatives claim that this approach can reduce inference costs by double-digit percentages, and in some cases, by up to 30%.
For more breaking news, follow the UA.News Telegram channel.
OpenRouter, Not Diamond, and LiteLLM operate in this market. OpenRouter offers unified access to hundreds of AI models; Not Diamond automatically routes queries to the model it believes is best suited for a specific task; and LiteLLM helps enterprises build their own routing infrastructure. Salesforce and Databricks are also integrating such features into their AI platforms. According to Fortune, Cursor, Ramp, and Meta are working on their own routers, while Runway has already launched such a product.
Not Diamond CEO Thomas Hernando Kofman noted that using the most powerful model for all tasks can be wasteful. At the same time, an incorrectly chosen smaller model can also increase costs if it takes longer to complete the task. A routing system can take into account the complexity of the request, the conversation history, and other signals to select the appropriate model and level of reasoning for the agent’s next step.
Salesforce President and Chief Architect David Ward believes that in the future, routing will encompass more than just cost control. It will also involve trust, compliance, management of access to corporate data, and measurable business outcomes. At the same time, Chris Clark, co-founder and COO of OpenRouter, emphasized that building an industrial-grade routing platform requires partnerships with model providers, continuous monitoring of their performance, and the ability to respond to failures and changes in specifications.
