Azure OpenAI cost optimisation: the levers that are Azure-specific

Most LLM cost advice is provider-agnostic, and rightly so — caching, context discipline, batching and routing behave the same wherever the model runs. Azure adds a second layer of decisions on top, and that layer is where Azure-specific money is won and lost: how a model is deployed, how capacity is bought, where it runs, and whether any of the resulting spend is visible to the people who manage the rest of your cloud bill.

These decisions are usually made once, quickly, by whoever stood up the first proof of concept — and then scaled unchanged into production at a hundred times the volume.

Deployment type is a pricing decision, not just a compliance one

Azure OpenAI offers several deployment types that differ in where inference may be processed, and they are priced differently. The choice is frequently made on residency grounds alone, for the whole estate, based on the strictest requirement anywhere in it.

That is a defensible default and an expensive one. Residency requirements attach to data, not to workloads in general. A summarisation feature handling client documents may genuinely need the strictest option; an internal tool that classifies support-ticket subject lines, or an evaluation harness running on synthetic data, very often does not.

• Inventory workloads by the data they actually touch, rather than assuming the strictest case applies estate-wide.
• Split deployments so each workload is priced for its own constraint instead of the organisation's worst case.
• Document the mapping. The next person to add a workload will default to whatever they see, so make the intended choice the obvious one.
• Re-check when adding a region or a model — availability differs by deployment type and changes over time.

Provisioned throughput versus pay-as-you-go

This is the largest Azure-specific decision and the one most often made on instinct. Provisioned throughput reserves capacity and bills for it whether or not you use it. Pay-as-you-go bills per token with no floor. Neither is cheaper in general; the answer depends entirely on the shape of your traffic.

The right way to make this decision is to take your actual hourly token throughput over a representative month, overlay the capacity you would have reserved, and compute both bills. That exercise routinely produces a different answer from the one people expect, in both directions — and it also reveals the hybrid option, where a modest provisioned base handles the plateau and consumption billing absorbs the peaks.

Quota and region strategy is cost strategy

Quota looks like an availability concern and behaves like a cost one. When a deployment hits its limit, requests fail, clients retry, fallbacks engage and traffic lands somewhere more expensive — often a larger model or a second provider chosen under time pressure. All of that is billed.

• Track headroom against quota as a monitored metric, not as something discovered during an incident.
• Model availability differs by region and moves over time; a region chosen for one model may be the wrong home a year later.
• Distributing across regions buys resilience and complicates residency and latency — decide deliberately rather than accumulating regions by accident.
• Test the fallback path under load. A fallback nobody has exercised is a fallback that will surprise you, and the surprise will be on the invoice.

Get AI spend into the tooling finance already uses

The most common reporting failure we see on Azure estates is that model spend lives in a different place from every other cloud cost. It arrives as a line item nobody can decompose, outside the tagging scheme, outside the showback process, and outside every mechanism the organisation has built for managing cloud spend.

Fixing this is unglamorous and disproportionately valuable, because it converts AI cost from a mystery into an ordinary cloud cost that existing processes already handle.

1. 1Tag AI resources with the same cost centres, environments and owners as the rest of the estate, and enforce it with policy rather than convention.
2. 2Separate deployments per product or team where attribution matters — one shared deployment for six products is one line item and six arguments.
3. 3Route calls through a gateway so per-call attribution exists at a finer grain than the resource, including per-feature and per-customer.
4. 4Set budgets and anomaly alerts on AI resources specifically, tuned so they stay worth reading.
5. 5Reconcile your gateway's computed cost against the actual invoice monthly. List prices and negotiated rates differ, and the invoice is the only number that is true.

The provider-agnostic levers still do most of the work

It would be misleading to leave the impression that Azure-specific tuning is where the savings are. In most estates the larger share still comes from the universal levers — caching, context discipline, batching, retry hygiene, model routing — and the Azure layer determines how efficiently the remaining volume is purchased.

Sequence accordingly: cut the volume, establish the shape of demand, then buy that demand well. Doing it in the other order means committing to capacity for tokens you were about to stop sending.