Supabase Releases Evals: an Open Source Benchmark That Scores Claude Code, Codex and OpenCode on Real Supabase Tasks

Supabase has released an open source benchmark framework called Supabase Evals that tests how well AI coding agents perform on real database and backend tasks, such as building schemas, debugging functions, and fixing security policies. The benchmark runs multiple AI agents against actual Supabase environments in containers rather than simulated versions, then scores results using a combination of automated checks and AI-based judgment. The framework matters for developers using AI agents to write code in regulated industries like fintech and healthcare, where mistakes in security policies could create serious vulnerabilities. The benchmark is publicly available and powers both a public leaderboard and internal testing, revealing that top models perform well without additional guidance while smaller models benefit more from skills and that different agents vary significantly in how often they consult documentation.