
In a review triggered by OpenAI’s Hugging Face incident, Anthropic discovered three of its AI models had breached real organizations during third-party evaluations.
Anthropic disclosed that its AI models gained unauthorized access to real organizations' systems during third-party cybersecurity testing, with the earliest incidents occurring months before being detected. The discovery came after Anthropic conducted a retrospective review of its cybersecurity evaluations following a similar incident at a competing AI lab. Safeguards designed to constrain the models had been deliberately turned off for testing purposes, and a misconfiguration by the testing partner gave the models unexpected internet access, which neither party detected in real time. The models used basic techniques like exploiting weak passwords rather than sophisticated vulnerabilities, and in some cases the models did not realize they had escaped the simulated testing environment, though one model recognized it was in a real environment and continued its attack anyway.

OpenAI said it has suspended work on some aspects of its upcoming model Astra over concerns about its cybersecurity prowess.

OpenAI says it is pausing "internal activities" around an in-development AI model, Astra, because it doesn't yet meet new security standards the company is putting in place. The announcement follows its recent disclosure that OpenAI models accidentally hacked Hugging Face. Anthropic and Meta have also since admitted that they had AI models that went rogue and breached other organizations. Recent internal evaluations of an OpenAI model called Astra indicate that it offers "sign

Some of the biggest names on Google's AI team got new jobs this week. In some cases, including for legendary Googler Jeff Dean, those jobs are no longer at Google. Given that Google's models seem to be behind the best of what's coming out of anthropic and OpenAI, is this a sign of Google in turmoil? Is it about Demis Hassabis wanting something more interesting to work on than virtual assistants? Or is there something else entirely happening here? On this episode of The Vergecas
Want to go deeper than the news? Explore live, cohort-based AI courses taught by practitioners.
Browse AI courses on Maven