Microsoft MAI-Transcribe-2-Streaming Reads Speech Mid-Sentence

Microsoft has added three new Microsoft AI (MAI) voice models to Microsoft Foundry, its catalog and platform for apps and agents. The headline model, MAI-Transcribe-2-Streaming, turns live speech into text while a person is still talking, so software can begin to act before a sentence ends.

Naomi Moneypenny, senior director of product development in the Foundry Models division, announced the models on October 1. The other two are MAI-Voice-2.1, an expressive multilingual text-to-speech model, and MAI-Voice-2.1-Flash, tuned for responsive, high-volume applications. According to Microsoft, developers should pick MAI-Voice-2.1 when expressive fidelity matters most, and Flash when they need to balance natural speech with responsiveness and cost at scale.

How incremental recognition works

The streaming transcriber, which includes automatic language detection, produces its first hypotheses, known as partials, within the "low hundreds of milliseconds" of receiving audio. It refines them as more context arrives and commits to a stable transcript once an utterance ends. Microsoft says a support agent could identify a caller's request before the sentence is finished, and a voice assistant could start reasoning or preparing a tool call sooner. Microsoft also says the model debuts at number one for accuracy on both partial and final transcripts on the Artificial Analysis leaderboard.

The models are available through Azure Speech, which Microsoft has offered for years. Target uses include customer-service agents, multilingual assistants that reply in a consistent voice, language-learning tools and live captions.

Latency is only half the problem

Will Bodewes, CEO of voice-agent company Phonely, called the launch a sign that voice AI is ready for wider use. He said his company's own model reaches 120-millisecond responses and that critical steps, such as collecting a name and date of birth before a refund, should be enforced in code rather than left to the LLM.

Phonely machine learning researcher Amit Kiran Rege cautioned that the hard part is the pause in the middle, when a system must decide whether a caller has finished or is just taking a breath. At Phonely's scale, he said, the agent jumped in too early two-thirds of the time, but callers rarely noticed because they resumed before the first syllable and the reply was canceled. Halve the latency, and those become audible interruptions. His conclusion: latency and turn-taking have to improve together.