XYBench tests if AI recognizes flawed user questions

Hugging Face: XYBench: Helping with the problem behind the question

Understanding what a question actually asks matters more than raw performance scores for building reliable AI systems.

Get smart on it

XYBench is a benchmark containing over 8,000 questions that tests whether AI assistants can recognize when users ask for help with an approach that won't solve their actual problem, a situation known as the XY problem. The benchmark matters because truly useful AI assistants need to help users with misguided questions, not just those who ask the right question correctly, yet current large language models often focus on the stated request rather than identifying the underlying goal. The test measures three aspects of model responses: whether they include solutions to both the stated and intended requests, where they place emphasis in their answers, and whether they steer users away from mistaken approaches toward better solutions. Experiments showed that all five evaluated models were less likely than human answers to recognize misconceptions and redirect users, though models performed better when explicitly told what the user's underlying goal was.

Google DeepMind just mathematically verified key parts of Signal

Google DeepMind researchers used agentic AI to mathematically prove, or “formally verify,” a number of the Signal protocol’s key cryptographic guarantees—a landmark development for the widely used cryptography and AI itself. Conducted as a sort of proof-of-concept project in just four weeks, the work is significant in two ways. It serves as further independent validation of the cryptography that protects massive quantities of the world's communications on platforms like the Signal app, Google Me

Also reported by The Decoder

BenchmarksOpen story →

Best AI Video Generation Models in 2026: Elo, Price per Second, Open Weights

The AI video leaderboard looks very different from 12 months ago. Native audio is now table stakes. Clips run 15–30 seconds. And an open-weights model sits in the top 3. This ranking (as of Nov 13, 2026) uses one quality signal: blind human-preference Elo from the Artificial Analysis Video Arena. We pair it with API price per second and the license status of the weights. You can check the deployment options and specs here through our GPULeaderboard.com chrome extension. Methodology Qua

BenchmarksOpen story →

Anthropic says AI is eroding Nvidia's CUDA lead, cutting an AMD chip switch from months to three days

Anthropic claims it used AI itself to breach Nvidia's CUDA moat. The company's in-house model Mythos reportedly ported Claude to AMD GPUs in just three days, with almost no human intervention. Anthropic now runs Claude on four chip families and aims to cut costs and reduce vendor dependencies. The article Anthropic says AI is eroding Nvidia's CUDA lead, cutting an AMD chip switch from months to three days appeared first on The Decoder.