TokenCast Forecasts How Many Tokens an AI Agent Will Burn
The same agent task can vary by more than ten times in token use across runs. TokenCast updates a cost forecast as the run unfolds and cut token use 21.3 percent in a budget-control replay.
TL;DR
- When an AI agent repeats a task, its token consumption can differ by more than an order of magnitude, making costs hard to predict [^1].
- TokenCast forecasts consumption while the agent runs; in an offline replay it used 21.3 percent fewer tokens than a fixed-budget policy at matched completion [^1].
Background
Agents for tasks like software engineering repeatedly plan, edit code, call tools and check results, and every round's output piles into the context of the next call [^1]. A task may be solved in one edit or need many retries, so total consumption is hard to know in advance and has to be revised as the run unfolds [^1].
What happened
Researchers from Sun Yat-sen University, Rensselaer Polytechnic Institute, Southeast University and the Hong Kong University of Science and Technology built TokenCast [^1]. It learns a cost representation for each segment of an agent's execution that records the segment's own consumption and the context growth it adds. Combining adjacent segments yields a running estimate that captures the extra cost of every later call re-reading earlier context [^1]. As new evidence arrives during the run, the forecast updates, at a mean cumulative prediction time of 32.8 milliseconds per run on SWE-bench Verified and without any additional model calls [^1].
Across four task suites and six agent models, TokenCast reduced mean absolute error by an average of 14.5 percent compared with the strongest alternative, over 96 combinations [^1]. In an offline budget-control replay, it used 21.3 percent fewer tokens on average than a fixed-budget policy at matched completion [^1].
Why it matters
Unpredictable cost is one of the practical barriers to deploying agents: a budget set too low kills runs that would have succeeded, and one set too high wastes money on runs that are going nowhere. A forecast that sharpens as the run progresses lets a system stop a run that is clearly heading over budget, or let a promising one continue.
Practical example
Say you give a coding agent a fixed budget of tokens per task. Some tasks finish quickly and waste nothing; others hit the cap just before succeeding. With a running forecast like TokenCast's, the system can see early that one run will need more and another is spiraling, and allocate tokens accordingly, which in the authors' replay saved about a fifth of the tokens [^1].