LLM cost engineering: what actually moves the bill

When an AI feature's cloud bill spikes, the first instinct is to swap in a cheaper model. It is usually the wrong first move. Across the cost reviews we have run this year, the biggest savings came from four changes that do not touch model selection at all.

Prompt caching is the largest lever. Most production prompts carry a long, static preamble: system instructions, schemas, few-shot examples. Providers now discount cached prefix tokens heavily, but only if the static content is actually stable and ordered before the dynamic content. Restructuring prompts to be cache-friendly is often an afternoon of work worth 30 to 50 percent of the bill.

Second is output discipline. Teams pay for tokens they immediately throw away: verbose JSON with unused fields, chain-of-thought a parser discards, apologetic boilerplate. Tight output schemas and maximum token limits are free money.

Third, batch what is not interactive. Overnight classification, embedding refreshes, and report generation do not need real-time endpoints. Batch APIs typically price at half the interactive rate.

Only after these three do we look at model tiering: routing easy requests to a small model and hard ones to a frontier model behind a router. It works, but it adds evaluation burden, so it should be earned by traffic volume, not adopted by default.

The pattern to notice: LLM spend behaves like any other cloud spend. The discipline that tamed your AWS bill (visibility, unit economics, and removing waste before renegotiating rates) applies unchanged.