Benchmarks

Black Forest Labs Releases FLUX 3 Action: A 7B Open-Weights World Action Model That Tops RoboLab-120

Black Forest Labs (BFL), the lab behind the FLUX image models, has released FLUX 3 Action. It is a 7B open-weights World Action Model (WAM) for robot control. The model reads camera frames, robot state and a text instruction. It then predicts future video frames and the next chunk of actions together. On the RoboLab-120 leaderboard, it ranks first at 42.92% task success. Is it deployable? Yes, with conditions. The DROID policy needs about 32 GB of GPU memory in BF16 on an H200. It fits 24 GB

Get smart on it

Black Forest Labs released FLUX 3 Action, a 7 billion parameter open-weights model designed to control robots by reading camera frames and text instructions, then predicting future video frames and robot actions together. The model ranks first on the RoboLab-120 benchmark at 42.92% task success, outperforming larger competing models while running 1.5 to nearly 4 times faster. This matters because it addresses a tradeoff in robot control systems: models that predict video tend to be accurate but slow, while faster models sacrifice accuracy. The model can be deployed on consumer and professional GPUs with quantization techniques and is available under a non-commercial license with code and documentation for fine-tuning on custom robot demonstrations.

Explainer: How to Actually Read an AI Benchmark (Without Being Fooled)

Read next
The biggest AI stories, once a day

One short email a day with the stories that matter. Free. Unsubscribe with one click.

Relevant courses

Want to go deeper than the news? Explore live, cohort-based AI courses taught by practitioners.

Browse AI courses on Maven
Related news
BenchmarksOpen story →

MLCommons Releases MLPerf Training v6.1 Results, Spotlighting Recent Advances in AI Training

Today, MLCommons® announced new results for the MLPerf® Training v6.1 benchmark suite. The two new benchmarks added in this round, and the submissions received, demonstrate the impact of recent architectural advances in AI training systems and the increasing diversity of the hardware used to train them. “We’ve reached an exciting new threshold for the MLPerf Training benchmark suite, with a higher level of participation in this round, the representation of both established and emerging hardw

Also reported by NVIDIA Blog

BenchmarksOpen story →

MLPerf Training v6.1: Bigger Clusters, Broader Hardware, and the Rise of Software-Driven Gains

The round at a glance MLPerf® Training v6.1 expands the benchmark suite and features even larger systems from our members. Two benchmarks join the suite: Qwen3.5-397B GRPO for agentic reinforcement learning, and DLRMv4-HSTU, a generative recommender. Google’s Ironwood and NVIDIA’s Rubin make their debut as Available systems. The largest jobs reached 10,240 GPUs (NVIDIA on GB300 and Azure on GB200, both training DeepSeek-V3) [1, 2], multi-node systems rose to 76% of the total, and the hardwar

BenchmarksOpen story →

Research acceleration: The jagged frontier

OpenAI measures how coding agents are accelerating AI research, where their performance remains uneven, and what this signals about recursive self-improvement.

BenchmarksOpen story →

Agentic inference capacity grew 4.6× in a year, versus 2.6× for compute capacity

Our Inference Performance Index and chip sales data show agentic inference capacity grew nearly twice as much as 8-bit compute.

BenchmarksOpen story →

New LoRA Adapter Lifts Qwen3.8 Coding Success From 39% to 65%

Hugging Face’s harness-hub team has released Qwen3.8-27B-pi-adapted, a LoRA adapter for Qwen3.8-27B. It was trained with reinforcement learning to work inside the pi coding agent. The Qwen3.8-27B-pi-adapted LoRA adapter targets one harness instead of general chat. For developers running local coding agents, that is the point. You keep the open base model and swap in a small, harness-specific adapter. TL;DR Size: LoRA rank 32 (alpha 64) on Qwen3.8-27B, a 27B dense hybrid with 262,14

BenchmarksOpen story →

How good are Fable 5.2 and GPT-6.2 Astra at real work?

We tested Fable 5.2, GPT-6.2 Astra, and four other models on real Epoch work tasks. None met our employee standard, including the two that top our ECI.