Discord admits AI moderation bug wrongfully banned users over harmless images
The company confirmed that the issue had been affecting accounts since May, with an additional 200 users banned over the weekend before its team identified and fixed the problem.
Get smart on it
Discord acknowledged that a bug in its AI moderation system incorrectly banned more than 8,000 users over two months after harmless images including spreadsheets, chessboards, game textures, and transparent backgrounds were mistakenly flagged as harmful content. The company's automated safety system works by matching uploaded content against databases of known harmful material, but a bug caused accounts to be immediately banned instead of waiting for human review as intended. This incident highlights challenges with AI-assisted moderation systems that platforms increasingly rely on to identify illegal or abusive material at scale, as similar problems have affected other major social platforms where users faced unexplained account suspensions.
Related news
OpenAI’s Chief Futurist Is Leaving the Company
Joshua Achiam spent nearly nine years at OpenAI researching AI safety and made a memorable appearance in the Musk v. Altman trial.
Zscaler finds autonomous agents succumb to IPI traps
Hidden instructions on websites con agents into falling for scams that a human would see through.
