How Generalist uses human demonstration data for robot learning

Generalist is a robotics startup that uses human demonstration data to train robot foundation models, allowing collaborative robots to learn tasks more quickly. The company's approach involves collecting video demonstrations of people performing real-world tasks using puppet-like end effectors, then converting these demonstrations into training data that enables robots to autonomously perform those same tasks. What distinguishes Generalist's method is its use of simple one-degree-of-freedom grippers and the combination of human-collected data with additional robot-collected data to refine the models. The company has demonstrated its models working across different robotic arms and tools on tasks ranging from folding boxes to repairing vacuums, with the ability to recover in real time when errors occur.