← All posts
July 5, 20265 min readThe AIFlow Team

Field notes: when the agent costs more than the developer

A single agentic task fires off 5 to 30 model calls. Without per-task tracking, cost becomes a number you only discover on the invoice.

costfinopsobservabilityagentstokens
Field notes: when the agent costs more than the developer
Listen to this article

Last week Gartner dropped a forecast that moved fast through our channels: by 2028, spending on AI coding agents is expected to surpass the average developer's salary (Gartner, 06/24/2026). The number that caught our attention was not the 2028 one, but today's: nearly a quarter of tech leaders already spend between $200 and $500 per developer per month on coding tokens alone, and 6% already exceed $2,000 per developer per month.

We read that as an engineering problem, not a procurement one. And it has a precise technical cause that Gartner itself points to: a single agentic task can fire off 5 to 30 model calls. Multiply that by reprocessing, context reinjected at every step, and tools that return bloated payloads, and cost per task becomes a variable nobody is measuring in the right place.

Unit cost fell, the invoice rose

It is worth separating two things that get conflated. The price per token has collapsed, roughly 10× per year for three years, with drops of about 80% between 2025 and 2026 across several providers (Vantage, DigitalApplied). Even so, invoices keep climbing, because volume runs faster than the price curve. Spending on model APIs roughly doubled between late 2024 and mid-2025.

The mature FinOps read already reflects this: 98% of FinOps teams now manage AI spend, up from 63% in 2025 and 31% in 2024 (FinOps Foundation, State of FinOps 2026). AI cost management is the most sought-after competency on those teams. In other words: it stopped being an emerging concern and became a day-to-day discipline, but instrumentation in the pipelines has not caught up.

Cost per token is an input metric, not an outcome

The classic mistake is optimizing the wrong number. Cost per million tokens (CPM) is useful, but it is an input metric. If you lower CPM by routing everything to a cheap model and, in doing so, the retry rate goes up and the task completion rate goes down, you cut the token bill and raised the cost per business outcome. The metric that matters is cost per successfully completed task, and it only exists if you attribute cost at the task, agent, prompt, and tenant level.

This is where we separate cost observability from performance observability, and why, in AI, they are inseparable. You cannot understand why cost jumped without knowing what the system was doing during that jump. In practice, this requires FinOps and engineering to share the same trace infrastructure, not separate reports.

What we instrument before putting an agent in production

The way we work at AIFlow, cost is a first-class signal in the eval and the trace, not an end-of-month report. A concrete playbook:

Granular attribution by default. Every call carries task, agent, prompt, model, and tenant tags. Without this it is impossible to tell a high-value production load apart from an infinite-loop experiment burning budget.

Cost inside the trace, next to the step. Every node in the agentic flow records input/output tokens, latency, and cost. When a task fires 30 calls instead of 5, it shows up in the trace, not on the invoice three weeks later.

Routing as a measured decision. A routing layer that sends ~80% of routine traffic to economical models and reserves frontier models for high-stakes reasoning cuts 60% to 90% of spend without degrading the perceived quality of the majority (Pristren). But routing is only safe if you have an eval coupled to it, measuring whether quality dropped along with cost.

Prompt cache as a blended-price lever. Reusing already-processed context reduces input token cost by about 90%, which lowers the effective price without depending on an official rate cut.

Budgets with an alert at 80% and anomaly detection. An alert at 80% of budget gives you a week or two to investigate before the next cycle. A cost-per-task anomaly is usually the first sign of an agent entering a loop or a tool returning an inflated payload.

RBAC and per-tenant limits. Access governance is not only security, it is cost containment. Who can invoke which models, with what spending ceiling, with human-in-the-loop on expensive tasks.

The practical point

Gartner's forecast is not about 2028; it is about the instrumentation that is missing today. Worldwide AI spending is expected to reach $2.52 trillion in 2026, up 44% year over year (Gartner, 01/15/2026), and cloud waste rose for the first time in five years, to 29% of IaaS/PaaS spend, precisely because AI workloads make cost forecasting structurally harder.

We treat this the way we treat any other production requirement: if it is not in the trace and the eval, it does not exist. Cost per task is a number you design from the moment you draw the flow, not a scare at the end of the month.

Sources