Unlocking Advanced Fine-Tuning: New Models in Azure AI Foundry

Today, we are pleased to introduce two major updates to model fine-tuning within Azure AI Foundry—Reinforcement Fine-Tuning (RFT) for the o4-mini model, which is launching soon, and Supervised Fine-Tuning (SFT) for the 4.1-nano model, which you can access now.

We’re delighted to reveal three significant improvements to model fine-tuning in Azure AI Foundry: Reinforcement Fine-Tuning (RFT) with o4-mini (arriving shortly), Supervised Fine-Tuning (SFT) for GPT-4.1-nano and the Llama 4 Scout model (already available). These advancements highlight our dedication to providing organisations with state-of-the-art tools to create tailored AI solutions that address genuine business needs. 

With these additions, we are opening up two key pathways for customising LLMs: GPT-4.1-nano serves as a robust compact model, ideal for model distillation, o4-mini stands out as the first customisable reasoning model, and Llama 4 Scout represents a leading open-source solution. 

Reinforcement Fine-Tuning with o4-mini 

Reinforcement Fine-Tuning brings an advanced level of precision in aligning AI behaviour with intricate business workflows. By incentivising correct reasoning and discouraging mistakes, RFT enhances decision-making in fluid or demanding settings.

Soon to be available for the o4-mini model, RFT will broaden what’s possible for scenarios demanding adaptable reasoning, context sensitivity, and sector-specific intelligence—while safeguarding top performance speeds.

Real world impact: DraftWise 

DraftWise, an innovator in legal technology, harnessed reinforcement fine-tuning (RFT) in the Azure AI Foundry suite to strengthen reasoning models calibrated for contract drafting and review. Tasked with delivering highly contextual, accurate guidance to legal teams, DraftWise refined Azure OpenAI models with confidential legal datasets to boost reliability and tackle complex instructions. The result was a 30% boost in quality for search outcomes, empowering lawyers to generate contracts with greater speed and devote more energy to strategic client work. 

Reinforcement fine-tuning for reasoning models is proving invaluable for us. Our models now grasp subtle legal language and respond more effectively to complicated drafting requests, making our service considerably more efficient for lawyers in the moment.

—James Ding, founder and CEO of DraftWise.

When should you use Reinforcement Fine-Tuning?

Reinforcement Fine-Tuning excels where flexibility, iterative improvement and tailored behaviour are crucial. You should consider RFT for scenarios involving: 

1. Custom Rule Integration: RFT is ideal for decision-making that can’t be fully captured by static prompts or traditional datasets. It allows your model to master evolving and exceptionally specific business rules. 

2. Industry or Organisation-Specific Protocols: RFT shines when internal procedures notably differ from sector norms and compliance with these bespoke standards is vital for success. It can encode differences such as longer operational timelines or altered compliance requirements directly into the model workflow. 

3. Complex Decision Trees: In sectors relying on multi-layered logic and variable-heavy decision processes, RFT enables models to navigate intricate subcases and accommodate diverse inputs, supporting more dependable, precise decisions overall. 

Example: Wealth advisory at Contoso Wellness 

Take for example Contoso Wellness, a fictional wealth management company. Leveraging RFT, the o4-mini model adapted to distinct business logic, such as recognising the best times to engage clients based on sophisticated metrics like the ratio of a client’s assets to their liquid capital. This allowed Contoso to accelerate onboarding and make sharper decisions rapidly.

Supervised Fine-Tuning now available for GPT-4.1-nano 

Supervised Fine-Tuning (SFT) is now possible with the GPT-4.1-nano model—a compact yet impressively capable base model, crafted for scenarios demanding quick responses and cost efficiency. SFT allows your AI to mirror your company’s language, terminologies, business routines, and desired styles—all refined for your area of expertise. Fine-tuning for this model is rolling out very soon. 

Why Fine-tune GPT-4.1-nano? 

• Targeted Responses at High Speed: Adjust the model’s replies for accuracy and performance, preserving agility. 

• Professional-Grade Communication: Align every output with your brand’s tone and in-house requirements. 

• Compact and Flexible: Well-suited for use cases where response speed and affordability are critical—perfect for customer service bots, edge-device use, or high-volume document management. 

Unlike bulkier models, 4.1-nano offers faster replies and lower operating costs, making it the top choice for extensive operations, including: 

• Automated customer service, where the AI handles thousands of requests an hour with reliable, brand-consistent messaging. 

• Internal knowledge tools that communicate in line with company protocols when producing summaries or answering staff queries.

How to Fine-tune Models in Azure AI Foundry

If you’d like to customise models for your business, follow these simple steps for fine-tuning within Azure AI Foundry:

1. Prepare a dataset of high-quality, relevant examples reflecting your use case requirements.
2. Access the Azure AI Foundry portal and select your desired base model (e.g., GPT-4.1-nano or o4-mini).
3. Upload your dataset and configure fine-tuning parameters—choose SFT for direct instruction or RFT for iterative feedback and reinforcement.
4. Start the fine-tuning process and monitor training metrics from the dashboard.
5. Evaluate your fine-tuned model using validation prompts or scenarios akin to real-world applications.
6. Deploy your tailored model to your production environment or integrate into your existing workflows.

How to Troubleshoot Common Fine-Tuning Issues

• Training stalls or fails: Ensure your dataset follows formatting guidelines and contains no corrupted records. Check the portal for specific error messages.
• Poor model performance after fine-tuning: Increase data diversity or quality and experiment with different training parameters.
• Deployment issues: Confirm compatibility between your fine-tuned model and your chosen application environment. Consult Azure documentation for supported integrations.