OpenAI released three new audio models through its Realtime API, each targeting a distinct capability in live voice applications: GPT-Realtime-2 for voice agents with reasoning, GPT-Realtime-Translate for live speech translation, and GPT-Realtime-Whisper for streaming transcription. Alongside the model releases, the Realtime API officially exits beta and is now generally available — a meaningful signal for developers who held off building production systems on it. All three models are available immediately through the OpenAI API and can be tested in the Playground.
Together, they push voice applications past the basic question-and-answer loop — toward systems that can listen, reason, translate, transcribe, and act within a single conversation.
GPT-Realtime-2: Voice Reasoning with a 128K Context Window
The flagship release is GPT-Realtime-2, which OpenAI team describes as its first voice model with GPT-5-class reasoning. GPT-Realtime-2 can process harder requests, manage interruptions, and continue conversations naturally. OpenAI expanded the model’s context window from 32K to 128K tokens, allowing longer conversations and more complex tasks without losing context.
Previous voice models frequently stalled on multi-step requests or dropped earlier context during longer sessions. GPT-Realtime-2 is specifically designed to keep the conversation moving while it reasons through a request.
Developers can enable short preamble phrases — like “let me check that” or “one moment while I look into it” — so users know the agent is working on the request. The model can also call multiple tools at once and narrate what it’s doing while it does — so instead of dead air during a multi-step task, the user gets a running commentary. These features directly address one of the most common failure modes in deployed voice agents: awkward silence that makes the system feel broken.
A particularly useful control for production builders is adjustable reasoning effort. Developers can dial reasoning intensity across five levels: minimal, low, medium, high, and xhigh. The default is “low” to keep latency down for simple requests, while tougher tasks can tap into more compute. This means teams can tune the performance-latency tradeoff at the session level depending on the use case — a quick customer lookup doesn’t need the same reasoning depth as a multi-step travel booking workflow.
GPT-Realtime-2 also adds tone control. The model can adjust its speaking style depending on the situation — staying calm during problem-solving, shifting to empathetic when users are frustrated, and turning upbeat after a successful outcome. The model is also better at understanding industry-specific terminology, including healthcare vocabulary and proper nouns.
On benchmarks, the gains are measurable. GPT-Realtime-2 with high reasoning scored 96.6% on Big Bench Audio, compared to 81.4% for GPT-Realtime-1.5 — a 15.2 percentage point improvement. GPT-Realtime-2 with xhigh reasoning scored 48.5% on Audio MultiChallenge instruction following, compared to 34.7% for GPT-Realtime-1.5.
Big Bench Audio evaluates challenging reasoning capabilities in language models that support audio input. Audio MultiChallenge evaluates multi-turn conversational intelligence in spoken dialogue systems, including instruction following, context integration, self-consistency, and handling natural speech corrections.
Pricing: GPT-Realtime-2 is priced at $32 per 1M audio input tokens ($0.40 for cached input tokens) and $64 per 1M audio output tokens.
GPT-Realtime-Translate: Live Speech Translation Across 70+ Languages
GPT-Realtime-Translate is a new live translation model that translates speech from 70+ input languages into 13 output languages while keeping pace with the speaker. Unlike GPT-Realtime-2, this model is a dedicated translation pipe — speech goes in one language and comes out in another. It is not a conversational agent; it is designed to convert one audio stream into another in real time.
The distinction is important for developers choosing the right tool. If your application needs a bilingual customer support flow or a live interpreter for an in-person event, GPT-Realtime-Translate is the purpose-built option. If you need the model to also reason, call functions, or hold context across turns, GPT-Realtime-2 handles that.
Pricing: GPT-Realtime-Translate is priced at $0.034 per minute.
GPT-Realtime-Whisper: Streaming Transcription as People Speak
GPT-Realtime-Whisper is a new streaming speech-to-text model built for low-latency speech-to-text — transcribing audio as people speak, so live products can feel faster, more responsive, and more natural.
The original Whisper model was designed for completed chunks of audio, making it better suited for post-session transcription. GPT-Realtime-Whisper is the streaming counterpart, purpose-built for applications that need live output. For realtime transcription, gpt-realtime-whisper gives you controllable latency — lower delay settings produce earlier partial text, while higher delay settings can improve transcript quality.
Use cases include live broadcast captions, meeting notes generated during the conversation, and voice agents that need to continuously understand the user rather than wait for turn-by-turn input.
Pricing: GPT-Realtime-Whisper is priced at $0.017 per minute.
Architecture Patterns and New Voices
Developers can choose between three session types depending on the use case: a voice-agent session when the application needs an assistant that responds to the user, a translation session when the application needs an interpreter, and a transcription session when text from audio is needed without model-generated responses.
On the voice output side, two new voices, Cedar and Marin, join the API roster exclusively with this release.
All three models — GPT-Realtime-2, GPT-Realtime-Translate, and GPT-Realtime-Whisper — are available now through the OpenAI Realtime API, which is generally available starting today.
Key Takeaways
- GPT-Realtime-2 brings GPT-5-class reasoning to voice with a 128K context window, five-level adjustable reasoning effort, tone control, parallel tool calls, and interruption recovery
- On Big Bench Audio, GPT-Realtime-2 (high) scores 96.6% vs. 81.4% for GPT-Realtime-1.5; on Audio MultiChallenge, the xhigh variant scores 48.5% vs. 34.7%.
- GPT-Realtime-Translate handles live speech translation across 70+ input languages into 13 output languages at $0.034/min
- GPT-Realtime-Whisper streams transcription in real time with controllable latency at $0.017/min
- The Realtime API exits beta and goes generally available today alongside two new voices, Cedar and Marin
Check out the Full Technical Details here. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
Credit: Source link
