NVIDIA Cosmos-H-Dreams: Bringing Real-Time Generative Simulation to Surgical Robotics

Surgical robotics systems are moving toward using vision-language-action policies, but training and evaluating them is difficult because physical robots are expensive to operate and failures can damage instruments. A world foundation model called Cosmos-H-Dreams addresses this by creating a real-time, action-conditioned generative simulator that learns surgical dynamics directly from video and robot movement data rather than manually modeling every object and interaction. The system distills a larger teacher model into a faster student model and uses an accelerated inference library to run interactive simulations at speeds fast enough for real-time closed-loop control on a single GPU. This allows safer evaluation and synthetic data generation for surgical policies without repeated physical robot execution.