Post-Stream Refinement is now generally available in Microsoft Foundry

This year, we unveiled Post-Stream Refinement in public preview, a significant step in real-time speech processing. It allows you to enjoy instant streaming results while also obtaining a highly accurate final transcription, all without delaying the first-token latency. A second recognition pass operates alongside the streaming, enhancing the final segment with a more precise version once the utterance wraps up.

Today, we’re excited to announce that Post-Stream Refinement is now generally available for Azure AI Speech within Microsoft Foundry, supported by a production SLA. Equally important, it comes with essential capabilities required for effective production transcription: diarization to identify who spoke, phrase lists for your custom vocabulary, and an extended coverage of 19 locales across 22 Azure regions.

Everything you’ve learned about Post-Stream Refinement remains applicable. The real-time streaming contract is unchanged; your partial results will flow as they did before. To activate the refinement, all you need to do is set one property in your existing SpeechConfig. The difference with general availability is that the refined transcript is now ready for production and aware of speakers. Read the Documentation

If you’ve already tried Post-Stream Refinement during the preview phase, here’s what changes with general availability and what stays the same. The streaming path and SDK contract remain intact; however, the refinement pass is now production-ready, boasting new speaker and vocabulary features.

What’s new at GA versus public preview release. Same real-time streaming contract; production-ready refinement with new speaker and vocabulary features.

Real-time and final results cater to different requirements. Quick partial results are essential for maintaining responsive captions, voice interfaces, and agent turn-taking, while final results need ample context for archiving, searching, summarising, and business workflows. Post-Stream Refinement operates both processes simultaneously: a rapid streaming pass along with a more detailed refinement pass of the same audio, running concurrently.

Both passes run in parallel — partial results stream immediately; final results are refined with broader audio context.

Because both processes use the same input stream, enabling refinement doesn’t require setting up a second transcription job or a different client pipeline. Your existing recognition events and handling of partial results will remain just as they are.

With general availability, diarization is now supported in the Post-Stream Refinement pathway. This means that the refined final transcript retains speaker labels. This makes it an excellent choice for meetings, contact centres, interviews, and any situation where it’s crucial to identify speakers in the transcripts. The refinement process not only improves terminology but also ensures every utterance is correctly attributed to the respective speaker.

Phrase lists allow the recognizer to focus on important names and terms relevant to your use case, including product catalogs, medical jargons, technical vocabulary, organization names, and acronyms that typical speech models may struggle with. With general availability, you can combine phrase lists with refinement so that the second pass benefits from both extensive audio context and your specific vocabulary, leading to significant accuracy improvements for named entities.

In our internal tests and evaluations with partners across supported locales, Post-Stream Refinement achieved a notable reduction in the final transcript word error rate, with improvements observed particularly in challenging content: lengthy utterances, proper nouns, and industry-specific language. Using phrase lists alongside refinement further boosts the accuracy of named entities. While the latency for partial results remains unchanged, only the final transcript undergoes refinement.

Relative improvement versus standard real-time transcription. Gains vary by locale, acoustic conditions, and content type.

The refined final result may introduce a slight delay to the last segment since refinement occurs only after segment audio is received. However, it’s important to note that partial results are not affected by this delay.

General availability now includes support for 19 locales. You’ll declare one locale per session, allowing the service to tune into the language you expect. In addition to major languages, we now support Indic locales, including Bengali, Marathi, Punjabi, and Telugu.

19 supported locales at GA. Declare one locale per session.

Post-Stream Refinement is available in 22 Azure regions spanning the Americas, Europe, and Asia Pacific.

Generally available in 22 Azure regions across the Americas, Europe, and Asia Pacific.

The technology behind Post-Stream Refinement currently powers meeting transcription and Microsoft 365 Copilot experiences in Microsoft Teams, serving millions of users daily through various meetings, webinars, and live events. General availability means every Azure AI Speech customer can access this quality via an integrated SDK, rather than just through a research prototype.

Preview users across several industries, including automotive, consumer electronics, and aviation, have reported significant enhancements in transcription quality, particularly for challenging content like proper nouns, lengthy discourse, and domain-specific discussions. Several are in the process of moving these workloads into production with the GA release.

Enabling Post-Stream Refinement is straightforward and requires just a simple configuration adjustment in your existing SpeechConfig. Here’s what you need:

• Speech SDK 1.50 or later: Earlier versions won’t support the refinement pathway.
• A Speech resource in one of the approved regions listed earlier.
• The locale for the session you intend to use, set on the recognizer.

To set the post-processing option, use the following example, which also illustrates how to incorporate a phrase list for your specific vocabulary.

import azure.cognitiveservices.speech as speechsdk

speech_config = speechsdk.SpeechConfig(
    subscription="YourSpeechKey",
    region="YourSpeechRegion")

# Declare one locale for the session
speech_config.speech_recognition_language = "en-US"

# 1) Refine the final transcript (Post-Stream Refinement)
speech_config.set_property(
    speechsdk.PropertyId.SpeechServiceResponse_PostProcessingOption,
    "PostRefinement")

audio_config = speechsdk.AudioConfig(use_default_microphone=True)

recognizer = speechsdk.SpeechRecognizer(
    speech_config=speech_config,
    audio_config=audio_config)

# 2) (Optional) Phrase list for names, acronyms, and domain terms
phrase_list = speechsdk.PhraseListGrammar.from_recognizer(recognizer)
for term in ["Contoso", "Fabrikam", "Foundry", "OAuth"]:
    phrase_list.addPhrase(term)

Your existing recognition events and the handling of partial results will remain unchanged. For speaker attribution, enable diarization through the established real-time diarization path; the refinement applies to the final transcript, while speaker labels are retained.

Now, Post-Stream Refinement has two pathways. They belong to the same product family but cater to different features, so choose the path that meets your customer’s needs.

Are you working with multiple languages? If there’s a need to manage several languages or code-switching without declaring a locale in a single stream, consider using Multilingual Post-Stream Refinement, which is currently in public preview. Conversely, for known session locales with phrase lists and diarization, choose monolingual GA.

Unlock higher-accuracy, language-aware transcription in your Azure AI Speech applications with just one simple configuration update.

Read the Documentation

We welcome your feedback. Try Post-Stream Refinement in your applications and let us know how it enhances your transcription quality.