The price of a promise

Rent a machine in the cloud and, by default, you pay on demand: a price per second or per hour, no notice, no promises. Start a thousand servers at nine and stop them at five, and you pay for eight hours. That freedom is the whole point of the cloud, and the FinOps Foundation even makes it a principle: take advantage of the variable cost model.

Freedom has a price, though. The on-demand rate is the cloud’s list price, set for a customer the provider knows nothing about. The provider has to keep idle hardware around in case you show up, and it charges for that uncertainty.

Look at a week of real usage and something stands out: however much it wobbles, it rarely drops below a certain level. Night-time, weekends, holidays: there’s always a floor. That floor is the part of your bill that isn’t variable at all, and paying the stranger’s price for it is money left on the table. The rest of this notebook is about what to promise instead.

Switch between the three shapes. Which one has the highest floor compared with its average?

A commitment is a promise: “I will pay for this much compute every hour for the next year, whether I use it or not.” In return the provider cuts the price, by around a third for a one-year promise and by half or more for three years. AWS calls these Savings Plans and Reserved Instances, Azure reservations and savings plans, Google Cloud committed use discounts.

Picture the promise as a waterline drawn across the usage curve. Below the line, usage is paid for at the discounted rate. Above it, usage spills over and is billed on demand. And wherever usage dips under the line, you pay for the promise anyway.

Two numbers describe the line. Coverage is the share of usage the commitment pays for. Utilisation is the share of the commitment you actually used. Raise the line and coverage goes up while utilisation comes down. Chasing 100% of either is the mistake: 100% utilisation means you committed too little, 100% coverage means you committed to your peak. The number that matters is the third one, the effective savings rate: what you saved, as a share of what the same usage would have cost on demand.

Drag the line to the floor, then to the peak. Where is the effective savings rate highest?

Slide the waterline from zero to the peak and plot the savings at every height, and you get a hill. At first every extra dollar of commitment is used every single hour, so savings climb steeply. Past the floor, some of the extra goes unused in quiet hours, and the climb slows. Eventually the waste outweighs the discount and the hill turns down.

There’s a neat rule for the top of the hill. One more dollar of commitment saves the discount, d, in every hour usage is above it, and wastes the remaining 1 − d in every hour usage is below it. So it pays off as long as usage is above it for more than 1 − d of the hours. With a 35% discount, commit up to the level you exceed 65% of the time. With 55% off, up to the level you exceed only 45% of the time.

That’s why the FinOps Foundation’s maturity guidance talks about coverage targets, not utilisation targets: sample targets climb from about 60% coverage for a team starting out to above 80% for a mature one. Those targets assume a mostly steady base, and one that’s already cleaned up. On spiky usage, the best coverage is lower, and that’s fine.

Pick “batch spikes” and compare the best coverage at 35% and at 55% off.

Commitments come in one- and three-year terms, and the longer one is a much bigger discount. The catch is that you can’t renegotiate it. If the workload moves to another region, another instance family, or simply gets smaller after a good optimisation, a three-year promise keeps billing for the usage you used to have.

A one-year promise can be resized every year, so it follows the workload down with a lag of at most twelve months. The question is whether the extra discount pays for that risk, and it usually does, unless the drop is big and early.

This is where FinOps and engineering need to talk. A team about to move half its compute to containers, to another region or to newer chips should say so before the commitment is bought, not after. Flexible commitments soften the bet (plate VII), and buying in small slices every month or quarter rather than one big purchase a year keeps renewal dates spread out, so there’s always something coming up for review.

Drop usage by 50% in month 6. Then move the drop to month 24. When does three years stop winning?

You can pay for a commitment three ways: all upfront, partly upfront or monthly. Paying earlier buys a few more percent of discount, which matters to a company with spare cash and not at all to one watching every month’s burn.

Paying upfront creates a reporting puzzle, though. In the month you buy, the bill shows one huge line. For the next eleven, the usage it pays for looks free. Neither tells a team what its servers really cost. The fix is amortisation: spread the upfront payment across the months it covers, so each month carries its share, and each team’s usage carries the discounted rate it actually got. On the bill, that’s the difference between billed and effective cost.

Who gets the discount is the next question. Commitments are usually bought centrally, because one promise across the whole company has a smoother floor than each team’s separately, and FinOps practice says rates are best managed centrally while usage is everyone’s job. Teams should still see the discounted rate on their own usage, or nobody notices when their changes leave a commitment unused.

Switch “all upfront” between cash and amortised. Which view would you put on a team’s dashboard?

The other way to get a discount is the opposite promise: not “I’ll always be here” but “I’ll leave whenever you ask.” Spot capacity (Google Cloud calls it spot or preemptible VMs) is the provider’s spare hardware, sold at up to 90% off on the understanding that it can be taken back at short notice: two minutes on AWS, thirty seconds on Azure and Google Cloud.

For a web server holding live connections that’s a problem. For a batch job, a test runner, a render farm or a training run, it’s a bargain, provided the work survives being interrupted.

Survival comes from checkpointing: saving progress every so often so a restarted job picks up where it left off. Without checkpoints, every interruption sends a long job back to the start, and if interruptions come often enough it may never finish at all. With them, an interruption costs a few minutes. The same trick keeps GPU training on spot capacity affordable.

Set interruptions to one every three hours without checkpoints. Then checkpoint every hour.