Lowest-Latency Inference APIs for Voice and Realtime Agents: A Time to First Token TTFT-First Benchmark
Time to first token (TTFT) is the metric teams use to pick an inference API for voice. It is also the metric that misleads them. TTFT marks when generation starts; a text-to-speech model cannot speak until a full clause arrives. Between those two points sits the difference between an agent that feels conversational and one that gets interrupted. This piece benchmarks every layer of the voice stack including LLM, speech-to-text, text-to-speech, and speech-to-speech. Why TTFT Is the Right Entr
Make your prediction
Will a voice AI inference API achieve sub-100ms median time to first token by November 30, 2026?
Resolves by Nov 30, 2026
Your prediction
50% · 50/50 coin flip
NOYES
Get smart on it
Time to first token (TTFT) measures the delay between sending a request to an AI system and receiving the first response token back. For voice agents, TTFT alone is a misleading metric because text-to-speech models cannot produce audio until they receive a complete clause or sentence, making the speed of subsequent tokens equally important to user experience. A complete voice interaction involves delays from speech-to-text transcription, language model processing, text-to-speech conversion, and network transmission, with natural conversation requiring total latency under 1.5 seconds and ideally around 700 milliseconds for the language model portion. Benchmark comparisons of TTFT across different AI providers vary significantly based on factors including input prompt length, server location, how measurements are taken, and provider infrastructure choices, making it critical to understand these methodological differences rather than comparing headline numbers directly.
Related news
Google says its AI weather model is getting better
Google is rolling out an updated AI weather model that's supposed to be more accurate, especially when it comes to predicting rain and snowfall. In the announcement today, the company says it's now able to make forecasts with "unprecedented resolution" using its new WeatherNext 3 AI model. It can produce a global picture that's five times sharper than Google's previous model by learning from real-time weather observations, according to the company. "One of the main developments
BenchMIRT: What are LLM benchmarks actually measuring?
Researchers question whether popular AI tests truly capture real-world language abilities or merely reflect memorization and training data overlap.
