OpenAI Details GPT-Red: An Internal Automated Red-Teaming Model That Beat Human Red-Teamers 84% To 13% On Prompt Injection

GPT-Red is an automated model designed to find security vulnerabilities in AI systems by attempting prompt injection attacks, where malicious instructions are hidden in data that the AI reads. The model was trained using self-play reinforcement learning, where it learns to craft increasingly sophisticated attacks while defender models learn to resist them. In testing, GPT-Red succeeded in finding vulnerabilities at much higher rates than human red-teamers and discovered a previously unknown attack method involving fake chain-of-thought entries. This matters because AI agents increasingly access external data through browsers, apps, and tools, creating expanding attack surfaces that human testing cannot efficiently cover, making automated red-teaming essential for safety before deployment.