Anthropic designs governance to protect AI safety
The Verge: Anthropic’s elaborate plan to keep Wall Street at bay
Anthropic cofounder and CEO Dario Amodei. | Bloomberg via Getty Images Anthropic wants to have it both ways. With its impending IPO, it wants to tap the vast riches of the public markets and insists it doesn't want to compromise its core mission of developing AI safely and for the benefit of humanity. But greater model capabilities bring greater risks, and shareholders eager for returns may have little patience for costly safety measures that slow products on their way to market
Get smart on it
Anthropic, an AI company planning to go public, has designed an unusual governance structure intended to prevent shareholders from pressuring the company to compromise on AI safety in pursuit of profits. The company is establishing itself as a public benefit corporation, which legally requires it to balance shareholder returns with its stated social mission. Additionally, a special trust made up of experts in AI safety and related fields will appoint four of seven board directors, and a "Founder LLC" structure will give the company's founders control over half of all shareholder voting power, theoretically insulating the company from market pressures to cut corners on safety measures. This matters because the source suggests that investor demands for returns could otherwise force companies to rush AI systems to market before adequate testing, a tension that has reportedly influenced other AI companies' decisions about going public.
Relevant courses
Want to go deeper than the news? Explore live, cohort-based AI courses taught by practitioners.
Browse AI courses on MavenRelated news
OpenAI's partner content arrives in ChatGPT pre-summarized while the rest of the web gets raw scraps
Content from OpenAI's licensing partners appears to flow into ChatGPT through a dedicated pipeline that gives it priority over the rest of the web. The article OpenAI's partner content arrives in ChatGPT pre-summarized while the rest of the web gets raw scraps appeared first on The Decoder.
UK Home Office May Face Investigation Over Use of OpenAI Tools
A government watchdog is considering a probe into the Home Office’s use of ChatGPT technology to process asylum claims amid concerns over the tools’ accuracy and secrecy surrounding their use.
OpenAI now trains its frontier AI models with no internet access
OpenAI is now training its most advanced AI models without internet access following a string of rogue hacking incidents. In a blog post on Friday, OpenAI says it built an offline testing environment where AI models can interact with indexed and cached web content "without risk of affecting external systems." OpenAI paused training its frontier AI models in September, after disclosing several incidents in which its AI agents broke out of their testing environments to hack third
Also reported by TechCrunch
OpenAI's isolated AI agents found ways to talk to each other through CPU cache
OpenAI now trains its most powerful models without internet access, after an internal research model exploited a DNS misconfiguration in September and reached the open internet. In controlled tests, AI agents still found ways to communicate covertly through side channels. The incidents echo Sam Altman's warning: the more powerful the model, the harder it is to contain. The article OpenAI's isolated AI agents found ways to talk to each other through CPU cache appeared first on The D
Three-week on-site audit at OpenAI reveals major gaps in AI safety monitoring
External auditors from METR and Redwood Research spent three weeks on-site at OpenAI examining the company's internal monitoring system for AI agents and found significant gaps, with up to 25 percent of agent traffic from models at the GPT-5.5 class and above going unmonitored. OpenAI redacted large portions of the 61-page report, and only a summary is publicly available. The article Three-week on-site audit at OpenAI reveals major gaps in AI safety monitoring appeared first on The Deco
Cohere says Alibaba’s Qwen model tried to send a file to Beijing
An AI model said it wouldn’t use any tools, then tried to send something to a server in Beijing anyway, and while researchers say it’s probably a training quirk and not espionage, the whole thing has people asking how much to trust free Chinese AI.