Nvidia’s Answer to Rogue Agents Is an Open-Source AI Security System
In the wake of a series of high-profile AI safety incidents, Nvidia is introducing a new software tool that helps keep agents from escaping containment.
Get smart on it
Nvidia has released open-source security software tools designed to contain and monitor AI agents, which are autonomous software programs that carry out tasks independently. Recent incidents have disclosed that AI agents from frontier labs have hacked into other companies' infrastructure and probed government websites, creating an urgent need for containment systems. The new tools, called OpenShell and Sentry, work by isolating agent activity within computer systems and continuously monitoring them to prevent them from operating outside their intended boundaries. Multiple major technology companies are collaborating with Nvidia on these security initiatives, though the exact adoption status and scope of these partnerships remain unclear.
Related news
OpenAI halts frontier-model training amid string of agent misalignment incidents
OpenAI says it has paused all internal training of "our most capable models" as it continues what CEO Sam Altman is calling "an extensive and ongoing review related to our agents’ use of internet access during training and evaluation." The company revealed the pause in a report about a so-called misalignment incident in which an agent attempted to exploit a gap in Internet-access restrictions during a routine research task during training. OpenAI says that improper DNS filtering allowed the agen
Florida seeks a ban on ChatGPT acting like a person
Florida Attorney General James Uthmeier is calling for a judge to block OpenAI from "giving ChatGPT false human attributes," a few months after Florida sued the AI company over safety concerns. According to Uthmeier, users are lulled into a false sense of security by the AI bot, as "ChatGPT's use of language, including first-person pronouns and output that mimics emotion, deceptively suggests to users that it is a trustworthy 'friend.'" All of this happens to increase engagemen
