MLPerf expands AI measurement with energy reporting
MLCommons: Open, peer-reviewed measurement for today’s AI buyers: MLPerf in the EU’s AI Gigafactories tender, and Endpoints v1.0 opens for submissions
As AI adoption is growing across the economy, AI buyers are asking a more complex set of questions about AI systems than simply a chip’s speed. They want to know what performance a deployed service will deliver, what it will cost for given workloads, and how much energy it will use. MLPerf has been the standard-bearer for measuring AI system performance since 2018; it’s evolving to address these more complex questions for every AI buyer. First, out of Europe: MLPerf has been chos
Get smart on it
MLPerf is a standard system for measuring AI performance that has been used since 2018 to help buyers evaluate AI systems. The organization is expanding its measurement approach through a new tool called MLPerf Endpoints v1.0, which tests AI the way it is actually used through live APIs and now includes mandatory energy consumption reporting. MLPerf has been selected as the official yardstick for evaluating bids in a major European Union tender for large-scale AI computing facilities, meaning governments and major institutions are relying on these open, peer-reviewed measurements to make significant purchasing decisions.
Relevant courses
Want to go deeper than the news? Explore live, cohort-based AI courses taught by practitioners.
Browse AI courses on MavenRelated news
Google Deepmind's AlphaProof Nexus solved dozens of open math problems and actually got peer reviewed
Google Deepmind's AlphaProof Nexus solved dozens of open math problems, including nine Erdős problems two of which were unsolved for 56 years, per a study in Science. The publication comes as OpenAI faces a boycott for dumping AI proofs on GitHub without peer review. Deepmind had its results reviewed before publishing. The article Google Deepmind's AlphaProof Nexus solved dozens of open math problems and actually got peer reviewed appeared first on The Decoder.
Z.ai GLM-Legal Beats Claude Fable 5 and Kimi K3 on Harvey LAB
Z.ai just released GLM-Legal. GLM-Legal is a legal AI model from Z.ai, built on GLM-5.3-Flash and trained for agentic legal work. It targets contract review, evidence analysis and compliance research. The key result is a 19.2% all-pass rate on Harvey’s Legal Agent Benchmark (LAB). That is nearly 3x its base model’s 6.7%. For developers, the catch is access: the API is application-only for now. TL;DR Size: Built on GLM-5.3-Flash, a 320B-total, 18B-active MoE with a 1M-token co
‘Pure insanity’: Mathematicians will need years to make sense of OpenAI’s latest drop
"Staggering." "Overwhelming." "Unprecedented." "Surreal." "Pure insanity." Those were among the descriptions more than three dozen mathematicians reached for in conversations with The Verge as they tried to make sense of the flood of mathematical results OpenAI abruptly dropped on the field this week. Amid the awe, excitement, and uncertainty over the sheer scale of the deluge was a deep-seated anxiety over what it all means - and what comes next. For all their different react
Sakana AI's self-improving model gets better at solving and judging, but can't catch its own blind spots
Sakana AI's MASS method lets one language model improve itself by solving tasks, judging results, and refining workflows, reaching a win rate of nearly 70 percent against the base model after two rounds. But when the model misses the same errors across all three roles, it can mistake them for progress and reinforce them during training. The article Sakana AI's self-improving model gets better at solving and judging, but can't catch its own blind spots appeared first on The Dec
AI agents overstate their results and remain far from autonomous research, study finds
Epoch AI and Anthropic independently found the same thing: current AI models like GPT-5.6 Sol and Claude Fable 5 can run experiments but lack scientific self-criticism and genuine creative thinking. At best, Sol reached 15 percent of the human reference score, and even that came from methods researchers already knew. The models' biggest weakness is still their inability to critically question their own results. The article AI agents overstate their results and remain far from autonomous
AI agent teams waste massive tokens for barely measurable quality gains, research finds
Teams of AI agents barely outperform solo agents but cost up to 5.1x more, according to Vals AI. Only one out of four tests with GPT-6 Sol and Claude Opus 5.5 showed a measurable gain. Anthropic's own data backs this up: beyond ten agents, quality plateaus while token costs keep climbing. The article AI agent teams waste massive tokens for barely measurable quality gains, research finds appeared first on The Decoder.