NVIDIA Releases NemotronLabs VoiceChat 11B: An Open Full-Duplex Speech-to-Speech Model with ~450 ms Turn-Taking and Live Tool Calling
NVIDIA has released NemotronLabs VoiceChat 11B, an open 11B end-to-end speech-to-speech model for real-time, full-duplex conversation. Instead of chaining ASR, an LLM, and TTS, it performs streaming speech understanding and speech generation in one unified network. That removes the multi-model orchestration and API handoffs a cascaded stack requires, and cuts end-to-end latency: measured smooth turn-taking latency is 448 ms on Full-Duplex-Bench 1.0. The model listens while it speaks, so a user
Get smart on it
NVIDIA released an open speech-to-speech model that handles real-time two-way conversation in a single unified network, rather than chaining separate components for speech recognition, language understanding, and text-to-speech generation. The model achieves measured turn-taking latency of approximately 448 milliseconds and allows users to interrupt the agent mid-turn, while also supporting tool calling through a separate output channel that lets the agent speak holding messages while APIs execute. The technology is available for research and pilot projects but has documented limitations including a two-minute audio context ceiling and degradation after multiple conversation turns. Deployment requires access to a single GPU with at least 80 gigabytes of memory, with potential applications across contact centers, automotive assistants, retail ordering, and accessibility tools.
Related news
Fine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3
In this tutorial, we implement an end-to-end supervised fine-tuning pipeline for the XYZ-Aquila-SFT dataset, Hugging Face Transformers, PyTorch, and PEFT. We stream and inspect the dataset, parse multi-turn tool-use trajectories, extract structured tool calls, analyze corpus characteristics, and preserve embedded reasoning and observation patterns. We then convert tool schemas between message-embedded and structured formats, render Qwen-compatible ChatML with assistant-only loss masking, prepar
Meta’s ‘open’ AI, and a $250M deal gone very wrong
Meta released Glimmer this week, an open-weight AI model anyone can download and run on their own hardware — a contrast to Muse Spark, the company’s more powerful model that stays locked behind its own APIs. The release landed alongside a letter from Mark Zuckerberg arguing AI should be “for everyone” rather than controlled by a handful of labs, but as Equity’s […]
