The Open ASR Leaderboard Adds Its First Global South Language

The Open ASR Leaderboard is adding evaluation datasets for Hindi and Indian English, marking the first languages from the Global South on a multilingual benchmark that previously contained only European languages. ASR (automatic speech recognition) leaderboards measure how well speech-to-text models perform, but existing benchmarks record only what was said, not details about who said it, which means performance disparities across different populations remain invisible. The new datasets were deliberately built to vary across nine dimensions including geography, age, gender, vocabulary, devices, and acoustic environments, so that models cannot score well on average while performing poorly for specific groups. This approach addresses research showing that ASR error rates are not evenly distributed across people, with documented racial, gender, age, and accent-based disparities in commercial systems.