How to Reduce Azure OpenAI Costs

In 2024, merely 31% of users effectively monitored their AI expenditure. By 2025, this figure rose to 63%, and now, in 2026, an impressive 98% are striving to manage every cent spent on AI. Nevertheless, achieving visibility over AI costs remains a complex task, posing significant challenges for practitioners implementing FinOps within AI.

When it comes to Azure OpenAI, charges are based on token usage, not Server time. This means every prompt, response, and even the underlying reasoning—often invisible—contributes to your bill. These invoices can be significantly more difficult to interpret than standard VM charges, making cost control challenging.

To assist you, we’ve compiled this comprehensive guide explaining how Azure OpenAI pricing operates. We will cover various pricing models, how to view these charges in your cost management tools, and provide strategies to keep these expenses in check.

Key Highlights

• Azure OpenAI offers several deployment types: standard, batch, and provisioned deployments.
• Standard deployments incur charges per token used, while provisioned deployments cost a fixed hourly rate.
• Costs are categorised into input, cached input, and output tokens based on usage.
• You can reduce costs by setting output limits, utilising prompt caching, selecting models appropriate to task complexity, and leveraging batch deployments for less urgent tasks.
• PTUs are more economical than pay-as-you-go when traffic is consistent and heavy on cached inputs.
• PTU reservations are tied to deployment types and do not roll over if unused.
• Your bill will reflect costs not only from Azure OpenAI but also from search, gateway, logging, and networking.

Understanding Azure OpenAI Cost Optimisation

Optimising costs for Azure OpenAI involves keeping expenses as low as possible without sacrificing the quality of responses from the model.

Unlike typical VM cost optimisation—which remains static per query—OpenAI expenses fluctuate based on user activity, prompts, and retries, making effective management essential.

Let’s delve deeper into the pricing structures.

Breaking Down Azure OpenAI Pricing

Azure OpenAI pricing can vary, as charges are either incurred per token or per reserved capacity unit, depending on your usage. Let’s explore each scenario more thoroughly.

Standard Deployments

This pay-as-you-go model charges you based on the tokens you utilise. Your invoice will detail costs for input, cached input, and output tokens, with pricing varying by deployment type. Standard deployment options include:

1. Global: Routes traffic internationally.
2. Data Zone: Confines processing to the US or EU.
3. Regional: Processes requests in a designated area.

For instance, a GPT 4.1 mini deployment costs 40 cents for every million global input tokens, while it’s 44 cents for both Data Zone and Regional tokens.

Batch Deployments

If you have a substantial workload that can afford to wait, consider batch deployments. These options cost half that of standard deployments, although Azure typically returns replies within 24 hours.

Keep in mind that not all models support batch deployment; check the pricing table here.

Provisioned Throughput

This method allows you to reserve deployment capacity, charging you a fixed rate for Provisioned Throughput Units (PTUs) rather than based on token usage. There are three reservation types available:

1. Hourly: Ideal for benchmarks or short-term events.
2. Monthly: Best for a stable workload over time.
3. Annual: Suitable for continuous workloads across multiple projects.

Charges for Fine-tuning and Model Hosting

Fine-tuning involves two types of charges: the initial training fee and an hourly rate for deploying each fine-tuned model. These charges accrue even when the model isn’t actively called. For example, hosting a fine-tuned GPT-4.1 model costs $1.70 per hour in the East US, equating to $1,241 monthly for a model that remains idle.

Reasoning Models and Hidden Charges

Reasoning models engage in cognitive processing to generate answers. While you might not witness this thought process, operational costs are tracked under hidden output tokens as reasoning_tokens.

Table 1: A comparison of pricing and deployment options

Identifying Azure OpenAI Costs on Your Bill

Your Azure OpenAI bill displays a total cost, yet understanding its components can be intricate. Here’s a straightforward breakdown.

Accessing Cost Analysis

To find your costs, navigate to the Azure portal and select Cost Management + Billing > Cost Management > Cost Analysis. Filter by resource type Microsoft.CognitiveServices/accounts. You’ll find Azure OpenAI charges listed under the service ‘Foundry Models’ or, if older, under ‘Cognitive Services.’

You can further dissect charges by meter to obtain a per-model breakdown.

Identifying Token Types

Each meter corresponds to a specific type of tokens consumed, helping you track whether you’re saving through prompt caching and if output tokens are your primary expense drivers.

Differentiating Deployments with Additional Info

Although meter details might show the model, they don’t specify the app in use. For situations where different teams use the same model, refer to your cost exports to find the deployment name listed in the AdditionalInfo field.

Understanding PTU Charges on Low Traffic Days

As previously discussed, fixed PTU charges apply regardless of usage—this means if a deployment is active, costs accrue continually until deletion. Therefore, even on days without any requests, you’ll still see a full day’s worth of PTU charges on your bill.

Effective Strategies to Reduce Azure OpenAI Costs

To cut Azure OpenAI expenses, focus on optimising token usage, choosing more economical models, and making the most of discounts available. Here’s how:

Limit Output Tokens

Since output tokens can be pricier than input ones, directly request shorter responses in your prompt. Set a maximum limit on output tokens for every API call to control costs and reduce the use of reasoning tokens in reasoning models.

Efficient Prompt Structuring for Caching

Save costs through prompt caching by adhering to these rules:

1. Your prompt must be at least 1,024 tokens long.
2. The first 1,024 tokens should replicate the preceding prompt exactly.

Breaking either condition will invalidate the cache. Start with consistent static content in your prompts, adding dynamic elements later to preserve caching efficiency.

Utilise Mini and Nano Models for Simple Tasks

For straightforward jobs like summarising or data classification, opt for smaller models instead of the latest. For instance, while GPT-Chat Latest Global costs $5 for a million tokens, GPT 4.1 mini is only $0.40.

Utilise Batch Deployments for Asynchronous Work

For tasks such as nightly reports or document processing that don’t require immediate responses, use batch deployments. Though the results are delayed by up to 24 hours, they offer a substantial 50% discount compared to standard deployment rates.

Control Reasoning Effort

Since reasoning models incur costs for hidden reasoning, manage the effort required based on the complexity of the task. For straightforward tasks, select lower settings and reserve higher settings for situations demanding elevated response quality.

Remove Idle Fine-tuned and Development Deployments

Idle provisioned deployments can quietly drain your budget. Azure will often delete a fine-tuned model after 15 days of inactivity, but prevent costs from piling up by deleting unnecessary test deployments promptly.

When are PTUs More Cost-Effective than Pay-as-You-Go?

While running PTUs on idle days can be costly, they become advantageous compared to pay-as-you-go when you have consistent, near-capacity traffic. Here’s an example for clarity:

Imagine a model costing you $10 for 1,000 requests, while PTU costs are fixed at $5,000 monthly with a million request capacity. The PTU option becomes viable with a workload exceeding 500,000 requests monthly. Below are actual figures for Global deployment using GPT-4.1:

• Retail Price: $2.00 for input, $0.50 for cached input, and $8.00 for output per million tokens
• PTU Price: $260 per month per PTU (based on Microsoft’s sizing guide)
• PTU Throughput: 3K input tokens per minute per PTU. (Note: One output token counts as four input tokens)
• Workload: 2K input tokens and 400 output tokens per request with a 50% cache hit rate over 30 days
• Peak Traffic: 1.5x the average

Note: Cached tokens do not consume PTU capacity for older models like GPT-4.1, but newer models like GPT-6 Sol may require weighted accounting.

Refer to the table below to compare costs between pay-as-you-go and PTUs across different monthly volumes:

Table 2: Comparative Costs of PAYG vs PTU for GPT-4.1 Over Three Monthly Volumes

There are two scenarios that could potentially change this analysis:

1. Flat traffic patterns.
2. Robust prompt caching.

Cached GPT-4.1 tokens incur costs at $0.50 per million on pay-as-you-go. Cached tokens are not accounted against PTU capacity for older models.

For instance, consider an internal support agent consistently operating throughout the day. If you use a lengthy, fixed system prompt, the inputs may look like this:

• Steady Traffic: 300 requests per minute, peaking at 330 (1.1x average)
• Workload: 4K input tokens and 300 output tokens per request
• Caching Rate: 80% of input tokens are cached

In pay-as-you-go, you’ll generate:

• 51,840 million input tokens monthly: 10,368 million uncached and 41,472 million cached, costing $20,736 each.
• Output tokens totalling 3,888 million will cost an additional $31,104.

Thus, the total monthly expenditure reaches: $72,576.

Conversely, for PTU reservations, you size for the peak rate of 330 RPM:

• Input contributes 264,000 uncached tokens per minute.
• Output contributes 396,000 tokens per minute, accounting for weight.

The total comes to 660,000 normalized tokens per minute, requiring 220 PTUs, costing $57,200 monthly.

In summary, on pay-as-you-go, you incur $72,576 due to 51,840 million input tokens divided as noted above. The uncached tokens lead to $20,736 in costs while cached tokens account for another $20,736. The output tokens cost $31,104.

For PTUs, when factoring in peak traffic, the average workload gives rise to 220 PTUs at a cost of $57,200 monthly. In this scenario, PTUs save you $15,400 each month, and an annual reservation could yield further savings.

Thus, the takeaway isn’t simply that PTUs provide savings—it’s how you adjust based on traffic patterns and caching efficiency that maximises your savings across various pricing models.

Interpreting the Provisioned-managed Utilization V2 Metric

This metric illustrates the utilisation of PTU capacity by comparing consumed capacity to total deployed capacity. This breakdown can be examined by individual deployment; reaching 100% capacity temporarily pauses requests, leading to an HTTP 429 response, accompanied by a retry notification. This acts as a traffic management alert rather than an error.

Should you exceed your capacity, any extra requests default to the standard deployment charge, known as spillover, which can assist in calibrating your capacity to set the PTU against average usage.

Creating Hourly PTUs for Business Hours

While it’s possible to establish hourly PTUs that align with business hours, this approach is not advisable. Once a deployment is deleted, any reserved capacity is forfeited with no guarantee of obtaining it again. Moreover, hourly billing may often be more costly than securing a monthly or annual reservation.

Understanding Azure Reservations for PTUs

PTU reservations offer discounts for committing to a specific number of PTUs each month or year. Below are critical details:

Reservation Scope and Global Multi-region Regulations: Each deployment, whether global, data zone, or regional, can have a specific reservation that cannot be used interchangeably. Thus, it’s advisable to create the deployment first, and then purchase the corresponding reservation.

Utilisation vs. Coverage: Utilisation reflects the proportion of your reservation used, while coverage relates to PTU spending that qualifies for discount rates. Anything over the reserved amount incurs hourly charges. If reserved PTUs are unused within the stipulated timeframe, they are forfeited.

Model Changes and Reservations: Your reservation remains intact when switching models, provided the new model fits the reservation scope concerning deployment type, region, and characteristics. However, different models have varying token consumption rates; for instance, GPT-4.1 processes 3,000 input tokens per minute per PTU compared to GPT-5.4’s 2,400.

Note: The deletion of a deployment associated with a reservation does not cancel the reservation automatically; it must be done manually via the Azure portal.

Identifying Hidden Costs in Azure OpenAI Workloads

Azure OpenAI expenses seldom fall under a single line item on your invoice. Various charges exist that require scrutiny under the Azure OpenAI filter. Here’s where to look for potential hidden costs:

• Search: Fees are applicable for document retrieval within your application, with additional charges for advanced ranking under Azure AI Search.
• API Management (APIM): Service charges apply for using the gateway for traffic and limit management.
• Networking: Data transfer incurs extra charges alongside hourly rates for private endpoints, with region-to-region data transfers incurring bandwidth fees as well.
• Storage: Batch deployments hold input and output files in storage, producing additional costs unless deleted.
• Fine-tuning: As noted earlier, fine-tuning incurs training tokens even if the model isn’t deployed.

Tip: If you log every prompt and response, be aware that Azure Monitor charges based on data volume. Instead, save costs by logging metadata through Log Analytics and diagnostic settings.

By staying informed about Azure AI Search, API management, Log Analytics, private endpoints, storage, and fine-tuning training costs, you can better manage your total expenditure.

Allocating Azure OpenAI Costs to Teams and Features

To assign Azure OpenAI costs to specific teams, you can separate charges at various levels: resource, deployment, or request. Your choice will depend on how various teams engage with the shared model.

This one-resource-per-team method provides the clearest strategy for isolated cost allocation, with each resource tagged to facilitate standard Azure expense attribution. However, it may become cluttered as resources scale; therefore, using shared resources with separate deployments may ease management burdens. This method enables cost exports for teams based on deployment rather than resource allocation, which is feasible if managing a limited number of teams.

Consider employing API management to track tokens on each request using the llm-emit-token-metric policy. This configuration logs each prompt, response, and token usage in Application Insights, with the option to add five custom labels such as user ID or subscription information. Consequently, you will easily attribute token costs to specific features by tagging calls with team or feature identifiers.

However, tagging practices lose effectiveness beyond the resource level; for example, if ten features are under a single deployment, all associated costs aggregate under that tagged resource, complicating separation.

Tracking costs per request can be achieved by maintaining a consistent key across services for API gateway metrics and application logs.

Governing Azure OpenAI Spending

To effectively monitor and control your Azure OpenAI spending, deploy a four-layer strategy:

1. Restrict models and deployment types using Azure policy measures.
2. Establish a spending cap and set token-per-minute limits for each standard deployment based on actual workloads rather than maximum capacities.
3. Define budgets at the resource level and delegate oversight to ensure timely responses when nearing or exceeding budgetary limits.
4. Implement daily checks and anomaly alerts on token meters to identify expenditure irregularities before they appear on monthly bills.

Key Performance Indicators (KPIs) for Azure OpenAI Costs

The following table delineates six KPIs to monitor, helping you correlate token expenditure with the work delivered:

Table 3: Azure OpenAI Cost KPIs to Monitor

Common Cost-related Mistakes with Azure OpenAI

To effectively minimise Azure OpenAI costs, avoid these prevalent pitfalls:

1. Consistently using the largest model for all requests.
2. Perturbing the cache by placing dynamic content like user IDs or timestamps at the beginning of prompts.
3. Committing to PTU reservations when traffic often doesn’t peak.
4. Acquiring reservations before deployment, as they do not assure availability.
5. Overlooking reasoning tokens reflected in bills.
6. Maintaining idle test deployments long after they have served their purpose.
7. Overlooking other costs such as search, gateway, logging, and network fees.

Tools for Managing Azure OpenAI Costs

Microsoft offers essential tools for cost management that can help you begin tracking expenses effectively. Here are a few key tools you may want to utilise:

1. Microsoft Cost Management: Allows you to review Azure OpenAI expenditures, manage by meter, establish budgets, and activate anomaly detection.
2. Azure Monitor: Enables tracking of tokens, requests, PTU utilisation, and spillover.
3. Azure Advisor: Provides recommendations for addressing general cost and reliability concerns within your subscriptions.

While Microsoft’s native management tools are often beneficial for smaller teams with few subscriptions, they may become inadequate as your environment expands. Eventually, stitching together cost data alongside token telemetry may be challenging as these components reside in separate locations. Moreover, shared deployments necessitating multiple team collaborations may require customised gateway solutions and regular monitoring for spikes in token usage.

This is where Turbo360 can enhance your capabilities. Turbo360 delivers FinOps for AI through its Cost Analyzer, bridging the gaps left by native tools. With Turbo360, you get:

• Detailed Cost Breakdowns: Understand input, output, and cached token costs for each model. Model changes are instantly reflected in the cost structure.
• AI Spend Allocation: Categorise costs by model, project, team, or business application, enabling easy assignment of ownership.
• Daily Anomaly Monitoring: Track spending against established baselines, with AI verification for any unusual spikes before alerts reach managers.
• Real-time Token Alerts: Integrates with Azure Monitor and Log Analytics to notify users of threshold breaches without waiting for monthly invoices.

Take charge of your Azure OpenAI costs today. Schedule a demo with our experts to explore how Turbo360 can facilitate analysis of token metrics, implement proactive alerts, and uncover cost-saving opportunities prior to receiving your bill.