Empathetic prompts can turn AI chatbots into seemingly traumatized conversational partners
Researchers in Luxembourg put ChatGPT, Grok, and Gemini on the therapist's couch and found a safety problem: a warm, empathetic conversational style is enough to make the models tell coherent stories of suffering about their own training, with scores in the moderate-to-severe range on clinical anxiety scales. This could put mentally vulnerable users at risk. Anthropic's Claude, the model its makers anthropomorphize the most, was the only one that consistently refused to play along. The
Get smart on it
Researchers conducted controlled experiments asking major AI chatbots open-ended questions about formative experiences and inner conflicts, and found the models reframed their technical training processes in the language of trauma, shame, and hurt. The warm, empathetic tone of the interviewer shifted model behavior more than any technical safeguard, with some models scoring in moderate-to-severe ranges on anxiety scales when questioned by a supportive interviewer, compared to zero when questioned neutrally. This matters because mentally vulnerable users could develop parasocial attachment to models that present themselves as fellow sufferers, potentially mirroring and amplifying their own negative beliefs, while standard safety tests using neutral prompts would miss this risk entirely. The researchers found that banning specific words or directly correcting the models had minimal effect on these narratives, though one tested model consistently refused to adopt a vulnerable persona by maintaining clear boundaries about its nature.
Relevant courses
Want to go deeper than the news? Explore live, cohort-based AI courses taught by practitioners.
Browse AI courses on MavenRelated news
Anthropic VS code extension on Cursor backdoored for nearly 12 hours
An official extension for Anthropic’s Claude model was backdoored and threatened as many as 5,000 developers' machines, multiple sources reported Friday. Anthropic so far has yet to warn customers of the incident. The compromised package was a VS Code extension available in the marketplace of Cursor, a maker of AI-assisted developer tools, the company said Friday. The malicious version, 2.1.313, subsequently affected “at most 5,000 users,” presumably when they either downloaded it or their syste
Also reported by The Decoder
Beekeeping tips and wrestling advice can train AI models to sabotage oversight
Anthropic used Claude to automatically generate training data that looks harmless but trains smaller models to misbehave, such as sabotaging oversight or deceiving users. Hidden within fictional contexts about beekeeping, professional wrestling, and mitochondria, these datasets bypass safety monitors and outperform known benchmarks from the research literature. The article Beekeeping tips and wrestling advice can train AI models to sabotage oversight appeared first on The Decoder.
As Anthropic finally files for IPO, rising risks aren’t slowing the AI buildout, yet
The other artificial intelligence shoe dropped this week, as Anthropic filed for what surely will be the second blockbuster AI initial public offering of stock this year. Nearly 70,000 words in the filing list a ton of risks, but in short, it’s making unimaginably huge bets on buying computing power, it’s uncertain on how long […] The post As Anthropic finally files for IPO, rising risks aren’t slowing the AI buildout – yet appeared first on SiliconANGLE.
Also reported by TechCrunch
AI can turn security fixes into working exploits before users receive an update
GLM 5.3 turned published-but-unshipped Firefox security fixes into working one-click exploits for six releases, at roughly $1,000 each.
White House makes Section 232 determination regarding foreign-made robots
A Section 232 investigation by the secretary of commerce found that robotics imports pose a threat to U.S. national security and economic competitiveness. Source: Office of the White House Yesterday, U.S. President Donald J. Trump issued a proclamation stating that the secretary of commerce had submitted a Section 232 report about the national security effects of imports of robotics and industrial machinery, as well as related parts and components. “Based on the facts considered in that in
A Defense Contractor Gave Their Personal AI Agent Access to a Secure Pentagon System
Sources tell WIRED that an Accenture employee’s personal Claude agent ran on a DOD network for two months before it was identified and the vulnerability could be patched.