Google updates Android Bench with new LLMs, but Gemini still lags behind
Code generation is emerging as one of the most popular applications for large language models (LLMs), but not all agents are equally good at all development tasks. Google created a benchmark earlier this year to evaluate how LLMs perform in Android app development, and Android Bench is getting a big update today. The leaderboard now includes a raft of new models, and Google has adopted a new framework that should be easier to use. Developers are invited to run their own tests and submit feedback
Get smart on it
Android Bench is a testing tool that measures how well large language models perform at Android app development tasks, evaluating them across 100 different coding problems. Google recently updated the benchmark to include new models and a more user-friendly testing framework, while also adding metrics for cost and efficiency. The results show that Google's own Gemini model ranks in fifth place for accuracy, behind competitors like Claude Fable 5 and GPT 5.4, though Gemini is more cost-effective to run than some top performers. Google is inviting developers to contribute their own test cases and feedback to help shape how Android Bench evolves, reflecting the company's broader push toward using AI agents for software development.
Related news
Datalab Lift vs the Field: How a 9B Schema-First Extractor Compares with NuExtract3, LlamaExtract, Marker, and Docling
Datalab’s Lift is a focused document extraction tool with a specific promise: give it a PDF or image plus a JSON Schema, and it returns schema-shaped JSON directly. Instead of converting a document to Markdown first and then asking another model to extract fields, Lift reads rendered page images and attempts to emit the final structured object in a single pass. According to Datalab, Lift is a 9B vision model for structured JSON extraction from PDFs and images, supports schema-constrained
Separating signal from noise in coding evaluations
A new analysis from OpenAI reveals issues in SWE-Bench Pro, a popular coding benchmark, raising concerns about reliability and accuracy in evaluating AI models.