OpenAI’s Jalapeño chip is built for fast inference at scale, benchmarks show

OpenAI has developed a chip called Jalapeño, created in collaboration with Broadcom, that is designed to handle AI inference tasks more efficiently than current state-of-the-art processors. In benchmark testing, the chip demonstrated better performance in tokens per user and throughput per kilowatt compared to existing inference processors. The chip addresses specific bottlenecks in the inference process, particularly during prefill and communication phases, by minimizing data movement and keeping model state local to reduce delays. Jalapeño is planned for limited deployment at the end of 2026, with broader deployment expected in 2027.