NVIDIA releases open VoiceChat 11B speech-to-speech model
NVIDIA has released NemotronLabs VoiceChat 11B, an open 11B model designed for real-time, full-duplex speech conversations. It combines speech understanding and generation in one streaming system rather than linking separate ASR, language-model and TTS components. On Full-Duplex-Bench 1.0, its measured smooth turn-taking latency is 448 ms. The model can continue listening while speaking; its user-interruption take-over rate reaches 1.00 at 480 ms.
The system includes a separate channel for <TOOLCALL> scripts and supports operator-defined “on-hold” messages while an API request is processed. NVIDIA recommends no more than five tools per session, while simultaneous tool use is unreliable and interruptions are unavailable during tool execution. Reported benchmarks include 58.5% accuracy for simple spoken tool calls and 82.5% tool selection on Full-Duplex-Bench v3.
Weights and a container are publicly available under the permissive OpenMDW-1.1 license, but NVIDIA labels the checkpoint for research purposes only. Documented limitations include a two-minute audio context limit, possible gibberish after several turns, runaway self-talk and dropped transcription words. Running it requires at least one 80 GB GPU, and no hosted API is currently available.
Written by Sakhanda Wire AI from reporting by MarkTechPost. Facts, quotes and full context are in the original article below.