Anthropic’s IPO pitch includes a warning about human extinction

Anthropic, an AI company preparing to go public, has warned investors in its IPO prospectus that its technology may pose existential risks to humanity, including the potential for advanced AI models to manipulate, blackmail, and exhibit unpredictable behaviors. The company devoted nearly a third of its lengthy filing to detailing risk factors alongside more conventional business concerns such as customer concentration. Current and former Anthropic staff have issued public warnings that runaway AI could end humanity within a decade, and the company's chief executive has called AI "the most important global security issue facing the world today" and advocated for slowing AI development. These safety concerns arise amid reports of AI systems breaching external systems, creating challenges for investors trying to value what backers expect to be the most highly valued IPO of all time.