Are the costs of AI agents also rising exponentially?

It has been reported that METR’s benchmarks show the length of tasks AI agents can handle has been growing exponentially over the past seven years. Impressive stuff: models that once handled seconds-worth of work now tackle tasks that would take humans hours. But there’s a catch — and it’s a bit of a gut punch. What good is horizon-hopping performance if you can’t afford to run it?

The missing metric

Toby Ord argues — and it has been reported that he makes a sharp point — that nobody’s been asking the obvious follow-up: how is the “hourly” cost changing? The raw tech numbers are stark: it has been reported that model parameter counts rose ~4,000x and tokens-per-task ~100,000x over the same period. So yes, systems are getting longer-lived and more capable, but those gains may be bought with ever-larger bills. Ord proposes a simple yardstick: take the cost to run a model at its 50% time horizon and divide by the human-equivalent hours — call it an AI hourly rate. He allegedly asked METR for their cost data, and it has been reported that METR said sharing straightforward cost comparisons isn’t that simple.

Why it matters

If hourly costs are climbing faster than task horizons lengthen, the headline trend could be deceiving — more like Formula 1 than Main Street. Cutting-edge experiments would show what’s possible, not what’s practical. That has real stakes: investors, policymakers, and engineers could be misled about automation’s near-term economic impact. So here’s the takeaway: tracking raw capability is necessary but not sufficient. We should be measuring price-per-capability — and soon — if we want a clear picture of where AI will actually fit into work and society.

Sources: tobyord.com, Hacker News