MLPerf Endpoints measures inference in production conditions
MLCommons: What is MLPerf Endpoints, and why it matters to buyers
What MLPerf Endpoints measures MLPerf® Endpoints measures what inference systems deliver in production-like conditions. Instead of benchmarking a tuned configuration in isolation, Endpoints measures a served endpoint: the model, hardware, serving software, and networking together, under realistic load, the way customers actually consume inference. What buyers get The full operating frontier, not a point. An Endpoint result is a Pareto frontier of operating points across concurrency lev
Get smart on it
MLPerf Endpoints is a benchmark that measures how inference systems perform in production-like conditions with realistic user loads, rather than in isolated tuned configurations. Unlike other benchmarks that test fixed scenarios at predefined operating points, Endpoints publishes a full curve showing performance across different concurrency levels, allowing buyers to see how systems behave at their actual workload demands. The benchmark tracks five primary metrics: concurrency, time to first token, system throughput, power efficiency, and interactivity per user. This matters to buyers because it reduces procurement risk by measuring the complete serving stack as it would actually be purchased and deployed, making visible the tradeoff between throughput and user experience at different operating points.
Explainer: How to Actually Read an AI Benchmark (Without Being Fooled)
Relevant courses
Want to go deeper than the news? Explore live, cohort-based AI courses taught by practitioners.
Browse AI courses on MavenRelated news
The AI race is back on
Last week, Anthropic and OpenAI looked like they had the AI race to themselves, pulling ahead with increasingly powerful models that left their rivals in the dust. Google seemed hopelessly far behind, while Meta, despite spending billions to revive its floundering AI ambitions, had yet to establish itself as a serious contender. Even the latest models coming out of China, impressive as they were, appeared to be playing catch-up. Then, in the space of a few days, everything chan
Grok Voice 3.0 from SpaceXSI outscores OpenAI and Google on speech-to-speech index
Elon Musk’s artificial intelligence company SpaceXSI, formerly xAI, today released Grok Voice 3.0, a new version of the speech-to-speech model behind its Voice Agent API. SpaceXSI is pitching the model to businesses that handle customer support and sales calls with voice agents. Independent testing firm Artificial Analysis ranks it first among models of its type. Artificial […] The post Grok Voice 3.0 from SpaceXSI outscores OpenAI and Google on speech-to-speech index appeared first
Also reported by MarkTechPost
Google Deepmind's AlphaProof Nexus solved dozens of open math problems and actually got peer reviewed
Google Deepmind's AlphaProof Nexus solved dozens of open math problems, including nine Erdős problems two of which were unsolved for 56 years, per a study in Science. The publication comes as OpenAI faces a boycott for dumping AI proofs on GitHub without peer review. Deepmind had its results reviewed before publishing. The article Google Deepmind's AlphaProof Nexus solved dozens of open math problems and actually got peer reviewed appeared first on The Decoder.
How well can AI play Slay the Spire?
Months ago, frontier models could barely beat Slay the Spire. Now they win 90%+ at the easiest setting and up to 20% at the hardest. Results from Epoch AI’s new SpireClimb benchmark.
AI guardrails designed to stop hackers are now blocking defenders, and Orion-1 aims to fill the gap
Artemis Security's new Orion-1 model is purpose-built for cyber defense and beats Claude Opus 5.5 and GPT-6 Sol on the company's own benchmark. The model fills a growing gap as general-purpose frontier models lock down their cyber capabilities, blocking defenders in the process. The article AI guardrails designed to stop hackers are now blocking defenders, and Orion-1 aims to fill the gap appeared first on The Decoder.
Open, peer-reviewed measurement for today’s AI buyers: MLPerf in the EU’s AI Gigafactories tender, and Endpoints v1.0 opens for submissions
As AI adoption is growing across the economy, AI buyers are asking a more complex set of questions about AI systems than simply a chip’s speed. They want to know what performance a deployed service will deliver, what it will cost for given workloads, and how much energy it will use. MLPerf has been the standard-bearer for measuring AI system performance since 2018; it’s evolving to address these more complex questions for every AI buyer. First, out of Europe: MLPerf has been chos