Deployed AI systems need their own safety tests
MLCommons: Don’t assume composability in AI safety. Deployments need their own testing.
Practitioners know that a deployed system behaves differently from its base model, and that its behavior keeps changing as they change the system prompt, retrieval layers, fine-tuning, and so on. But safety evaluation often assumes the opposite, with teams citing the base model’s safety score from the system card for whatever they’ve done on top of the model. Many in the field end up holding both beliefs at once: expecting behavior to shift when they build on a model, yet basing purchasing and
Get smart on it
Safety scores from base AI models do not reliably carry over to deployed systems that have been modified with fine-tuning, system prompts, guardrails, or other changes. Recent research shows that such modifications shift safety scores unpredictably in both directions, and many existing safety benchmarks are too noisy to detect whether score changes reflect real safety differences or just measurement error. MLCommons is addressing this by developing benchmarking tools that test complete deployed systems rather than base models alone, and by introducing margins of error in safety measurements so that deployers can reliably determine whether changes to their systems actually affect safety performance. This matters because teams currently make deployment decisions based on base model safety scores while knowing that deployed systems behave differently from their base models.
Relevant courses
Want to go deeper than the news? Explore live, cohort-based AI courses taught by practitioners.
Browse AI courses on MavenRelated news
Mistral Says Its AI Model ‘Le Chonk’ Has Leapfrogged China in the Open-Weight Race
The French AI lab claims its latest release matches or outperforms even the best models from DeepSeek and Z.ai in coding and cybersecurity.
OpenAI announces three big science discoveries and a new way to work with academics
Less than a month after its mass release of hundreds of manuscripts that it said contained solutions to long-standing mathematics questions, OpenAI is back with three more "major" discoveries in math, computer science, and physics. What's different this time is the choice to have external experts that OpenAI describes as "shepherds" guiding the results in computer science and physics, while for the math problem it says that they advised it to release the AI output for researche
Also reported by The Decoder
WorldOracle Tests Whether World Models Can Reliably Predict Real-World Executions
Key Takeaways WorldOracle is a benchmark for action-conditioned world models (ACWMs) that scores each model’s predicted rollout against the paired real-world execution, with both starting from the same initial observations and following the same action sequence. Seven representative models were evaluated; arm motion was predicted relatively well, but object positions and movements often deviated from reality, and performance dropped over complete operation sequences. The platform opened t
Frontier AI labs say their models refuted the Exponential Time Hypothesis. What does that mean?
TechCrunch spoke to the MIT computer scientist charged with explaining OpenAI's latest mind-bending discovery
Working with academic communities to build scientific understanding
OpenAI shares model results on the Kakeya conjecture, Exponential Time Hypothesis, and quantum chromodynamics—and how researchers can verify and build on them.
Nace AI Releases Drex for Search for AI Agents: 92.05% on BrowseComp-Plus With 12.6 Search Calls
Nace AI has released Drex for Search, which brings its Drex decision models to document search. Drex for Search is a retrieval layer that helps people and agents find evidence. Coding agents such as Claude Code, Codex, Cursor and OpenCode call it through CLI, SDKs, MCP or REST. In Nace’s evaluation, it scores 92.05% on BrowseComp-Plus with GPT-5, using 12.6 search calls per question. TL;DR Size: a hosted search system built to search over 1M+ files. It pairs with an answering model