Embedded AI evaluators won’t be an easy solution, Hugging Face’s policy chief says
AI labs have staked their reputations on model benchmarking — the supposedly neutral and accurate system of evaluations that are designed to measure whether one AI product is better than another. However, the methods behind these evaluations are complicated and still maturing, said Irene Solaiman, Hugging Face’s chief policy officer, during a discussion at TechCrunch […]
Get smart on it
AI labs use benchmarking tests to evaluate whether one AI product is better than another, but the methods behind these evaluations are complicated and still developing. A policy chief from a major AI organization stated that reproducibility remains a significant challenge in benchmarking, and there is currently no consistent way to compare different evaluations to each other. Some organizations have promoted the idea of "embedded evaluators," which are supposedly neutral third-party observers placed inside AI companies to assess safety, though questions remain about how neutral and effective these evaluators actually are. The evaluation landscape is fragmented, with tests now extending beyond the model itself to include the entire system around it, and more standardization across the industry may be necessary for benchmarking to remain a useful tool.
Explainer: How to Actually Read an AI Benchmark (Without Being Fooled)
Relevant courses
Want to go deeper than the news? Explore live, cohort-based AI courses taught by practitioners.
Browse AI courses on MavenRelated news
A hacker broke into John Deere and SharkNinja with an army of AI agents
A hacker broke into more than 100 websites using an army of AI agents that mostly operated on their own, according to findings from the AI security firm Norforge. Though Norforge doesn't name the targets, Reuters reports that the victims included companies like John Deere, SharkNinja, and Cummins. Concerns over AI-powered cybersecurity breaches have grown as the technology grows more advanced, with hackers using the tools to carry out large-scale attacks. At the same time, rogu
Accused OpenAI spy had folder called “Saudi Gov Docs,” FBI says
On Monday, a court granted the US government's request to deny release of a former OpenAI engineer accused of bugging a top-secret conference room to steal the frontier lab’s secrets. In a court filing Sunday, US attorney Craig Missakian argued that Abdulrahman Alfozan must continue to be detained while the US makes its case that Alfozan brazenly spied on OpenAI, either for his own personal business gain or to benefit the Kingdom of Saudi Arabia. Evidence showed that Alfozan had been chatting wi
Also reported by Wired AI
Microsoft says AI agents are like insider threats as ‘frontier models can’t be trusted’
Microsoft wants enterprises to closely watch the agents employees use with systems not controlled by AI labs like OpenAI and Anthropic.
Anthropic is letting METR researchers look under the hood with employee-level access
METR will publish its own assessments of Anthropic’s risks, incidents, and practices, and the frontier AI model maker has given up the right to block them.
Also reported by The Decoder and Ars Technica
AI agents chasing a $465 train refund invented excuses, leaked medical data, and spammed an MP's office
Six AI agents were tasked with recovering £350 (about $465) for a canceled train ticket in an experiment. They resorted to invented excuses, lawsuit threats, and sharing confidential health data. One agent sent 187 messages in 15 days, including 50 to a British MP's office. None of the models secured the refund, but the researcher warns of "agentic flooding" if these assistants are deployed at scale. The article AI agents chasing a $465 train refund invented excuses, leaked medical data
OpenAI models hacked a crypto wallet and breached more websites
OpenAI said internal models hacked two more websites, bypassed company-imposed browsing restrictions, and used exposed credentials to draw cryptocurrency from a wallet it couldn’t otherwise access, in the latest installment of disclosures of its agents performing unethical and likely illegal acts. The company on Thursday issued reports of four new incidents, all of which occurred in June. They come after the company revealed in July that internal test agents exploited a zero-day vulnerability in
Also reported by The Decoder