How AI guardrails are impeding the work of offensive cybersecurity researchers

AI companies have created safety guardrails and vetted access programs to prevent malicious hackers from misusing their models for cyberattacks. However, these same restrictions are now blocking legitimate offensive cybersecurity researchers who need AI tools to find vulnerabilities and develop exploits before criminals do. Security researchers argue that AI companies are making arbitrary decisions about what is safe, and that defensive and offensive cybersecurity work often require the same capabilities, making the restrictions counterproductive. Some researchers respond by using open-source AI models without guardrails, while others avoid using AI for sensitive vulnerability work altogether due to concerns about data security and the inconsistency of the restrictions.