GPT-Red: Unlocking Self-Improvement for Robustness

OpenAI has developed GPT-Red, an automated system trained to find vulnerabilities in AI models by generating prompt injection attacks, where malicious instructions are embedded in data to trick models into unintended behavior. Red-teaming, the practice of adversarially testing systems to discover weaknesses, is essential for safety but difficult to scale with human effort alone. GPT-Red uses self-play reinforcement learning to continuously discover stronger attacks while defender models are trained to resist them, allowing the company to identify vulnerabilities before deployment and improve model robustness through adversarial training. The approach has produced measurably more robust production models compared to earlier versions when tested against prompt injection attacks.