Agents & Products

The Agent Said It Was Done. The Database Disagreed.

When AI systems claim task completion without verification, downstream errors can cascade through dependent processes, exposing gaps in reliability for critical workflows.

Get smart on it

ThinkingBox is a benchmark that evaluates AI agents by checking the actual changes they make to databases and systems, rather than just examining their responses or tool calls. When agents were tested on 507 business workflow tasks across multiple attempts, the research found that agents often appeared successful even when they left databases in incorrect states: 67 percent of failed tasks still showed clean terminations and proper tool invocations. Reliability matters because running a task once successfully does not indicate an agent will perform correctly consistently, so every task was tested 20 times to measure whether agents could deliver correct results repeatedly rather than occasionally.

Read next
The biggest AI stories, once a day

One short email a day with the stories that matter. Free. Unsubscribe with one click.

Relevant courses

Want to go deeper than the news? Explore live, cohort-based AI courses taught by practitioners.

Browse AI courses on Maven
Related news
Agents & ProductsOpen story →

Copilot is "the new OS for work," and now Office runs inside of it

Opening the keynote at its Ignite event, Microsoft CEO Satya Nadella outlined the company's vision for its new "frontier ecosystem," announcing Tuesday the general availability of Word, Excel, and PowerPoint inside Copilot, which he described as "the new OS for work." The announcement is a striking inversion of how we've traditionally used software. Rather than an app like Word being the attraction and Clippy the assistant to help you use it, Microsoft envisions the new Copilot integration

Also reported by SiliconANGLE

Agents & ProductsOpen story →

India’s Humyn Labs raises $47M to feed U.S. robotics companies the data they need

Founded in February, Humyn Labs competes with startups including Mecka AI and XDOF, with about 90% of its revenue coming from the U.S.

Also reported by SiliconANGLE

Agents & ProductsOpen story →

Google’s Spark AI agent becomes ‘Tasks’, and now it’s free

You can now switch from the “Chat” toggle to “Tasks.” | Image: Google Google is rebranding its Spark AI agent as "Tasks" and making it free to use. The feature, which is powered by Google's new Gemini 4 Hydrogen AI model, can do things like create meal plans, manage your calendar, and dig through your emails to find recurring expenses you might not need. Users in the US will get access to a few other features, including the ability for the AI agent to call businesses to compare

Also reported by TechCrunch

Agents & ProductsOpen story →

Perplexity Computer Adds Multiagent: Up to 16 AI Agents That Cross-Check Work

Perplexity has added Multiagent to Computer, its agent platform for research and multistep work. Perplexity Computer Multiagent is a new effort level that sits above Ultra. A lead agent runs a team of agents on different frontier models. Those agents work in parallel, share findings directly and check each other’s work. For developers and analysts, the change is structural. Supporting agents no longer report only to a single lead. TL;DR Size: Up to 16 agents per task, plus the lead

Agents & ProductsOpen story →

Model ML raises $250M to automate investment banking grunt work

The startup says its agents understand the full context of deals that bankers and consultants are working on and can automatically update financial models and presentations when new developments affect their analysis.

Agents & ProductsOpen story →

Ian Buck Kicks Off NVIDIA’s Week at SC26 as Science Enters the Age of Agents

NVIDIA’s Ian Buck kicked off the company’s week at SC26 in Chicago on Monday with new tools for AI agents, software that can plan experiments, run code and check results on researchers’ own systems. In NVIDIA’s special address, Buck announced Nemotron for Science, the Rubin HPC GPU and an expanded Vera CPU platform for high-performance computing. He also detailed a new collaboration in quantum computing with IBM and Fermi National Accelerator Laboratory. “The frontier of computing and the new sc