The Next Evolution of AI Is Learning From Your Dodgy Gaming Skills

A British startup is converting video game controller inputs and visual data into training material for artificial intelligence models designed to navigate and interact with the physical world. Large language models trained only on text are considered limited in their ability to perform precise physical tasks like piloting autonomous vehicles or operating robotic arms, so researchers are developing "world models" that require both visual and action data to understand real-world physics. Video game data is abundant and varied enough to potentially provide the massive quantities of training material needed, though some researchers worry that video game physics are oversimplified and may not capture the precise requirements of tasks involving fine-grained motor control. The approach remains unproven, with experts divided on whether gaming data will ultimately be suitable for training world models that perform reliably in real-world conditions.