How do you budget for tokens?

Every AI feature gets the same question before launch: what will it cost? There’s no history to extend, so the answer has to be built from drivers: how many users, how many requests each of them sends a day, how many tokens each request reads and writes, and what the model charges per million.

The arithmetic is simple. The trouble is that every driver is a guess, and guesses multiply. Twice the users, twice the requests per user and twice the tokens is eight times the bill. Tokens per request are the guess most often wrong, because system prompts, retrieved documents and conversation history all count, as notebook 10 showed.

So the honest estimate is a range, and the useful one shows which driver moves it most. That driver is the one to measure first, usually in a small pilot, before the number goes into anyone’s budget.

Switch the model from small to large. Then find the driver whose bar is widest and halve its uncertainty in your head: how much narrower would the range be?

A cloud forecast, as in notebook 16, extends a trend with seasonality and lands within ten percent. AI spend doesn’t behave like that yet. Adoption grows in steps when a feature catches on, and one product decision, such as letting an agent take five actions instead of one, can multiply tokens per task overnight.

Run many possible years instead of one and the outcomes stop forming a neat band. Most land near the middle, and a few land far above it. The tail is where the budget conversations happen.

That’s why AI forecasts are better stated as percentiles. The P50 is the year you expect. The P90 is the year you could plausibly get. The gap between them is how much reserve, or how many guardrails, you need.

Compare the cloud workload with the AI one. Then launch agents in month 6 and watch the P90 pull away from the median.

Model prices fall fast. One estimate puts the decline at about 10× a year for the same capability. Another measured anywhere from 9× to 900× a year, depending on the task. A forecast that holds today’s price list for twelve months is almost certainly wrong.

It’s also wrong in the other direction. When tokens get cheaper, teams use more of them: longer contexts, more retrieved documents, agents that reason in several steps, newer models that think before they answer. Economists call it the Jevons paradox: efficiency raises consumption.

So forecast price and volume as separate drivers, and don’t let a falling price promise a falling bill. What usually falls is the cost of yesterday’s task. What grows is the task.

Set the price drop to 50% a year and tokens per task to 3×. Which way does the bill go? Now find the growth in tokens that keeps the bill flat.

The scariest AI cost is a loop: an agent that retries the same failing tool call, a batch job pointed at the wrong table, a leaked key. Usage-based pricing has no natural ceiling, so a mistake runs at full price until something stops it.

A monthly budget alert is too slow for that. It fires once the money is mostly gone. An anomaly alert on hourly spend notices within the hour. A hard cap stops the money for certain, but it also stops every other request on that key, so the guardrail becomes the outage.

The combination most teams settle on is an anomaly alert plus a per-key rate limit at a few times normal traffic, with hard caps kept for sandboxes and experiments. The runaway workload slows to a crawl. The customer-facing one keeps going.

Try each guardrail and read the cost of the incident and the hours of outage. Then turn on the per-key limit and try them again.

A fixed annual budget punishes the wrong thing. If the feature catches on and users double, spend doubles too, and the budget report turns red for the best news of the year. If cost per user quietly doubles because of a prompt change, a growing total hides it.

The fix borrows from unit economics: budget a cost per unit, and let the total flex with volume. The budget becomes “$0.40 per active user a month” instead of “$360k for the year”, and the variance report separates the two questions that matter: did we grow, and did each unit get more expensive?

Quarterly re-forecasts help too, but they only move the target. A unit budget tells you which part of the overspend to celebrate and which part to fix.

Compare the fixed budget with the unit budget. Which one flags month 9, when agents launch and cost per user jumps?