
Inside NVIDIA’s IsaacTeleop: From Hand and Controller Tracking to Robot Actions with the Graph-Based Retargeting Engine
In this tutorial, we work through the retargeting engine at the core of NVIDIA IsaacTeleop, the framework that turns XR hand tracking and motion-controller input into commands for simulated and real robots. Rather than plugging in a headset, we build every input ourselves in NumPy, so each step runs on a plain Colab CPU and prints what it computes. We start with the type system every node speaks, generate synthetic hand and controller data, write our own retargeter with live-tunable parameters,
IsaacTeleop is a framework that converts hand tracking and motion controller input into commands for simulated and real robots using a graph-based retargeting engine. The system works by translating physical hand positions and controller data into robot actions through a structured pipeline built on typed data containers and tensor operations. Understanding this matters because it enables remote robot control by mapping human movements to robot movements. The technical foundation relies on a type system where different input types, such as hand joint positions and controller poses, are validated at write time to ensure data integrity before being processed into robot commands.
One short email a day with the stories that matter. Free. Unsubscribe with one click.
Want to go deeper than the news? Explore live, cohort-based AI courses taught by practitioners.
Browse AI courses on Maven
California DMV bars Waymo AVs from operating on the state’s freeways
Waymo said the DMV order only applies to freeways and will not affect its operations on city streets. | Source: Waymo The California Department of Motor Vehicles has barred Waymo autonomous vehicles (AVs) from operating on the state’s freeways. The DMV issued the order on Nov. 11, following a number of incidents in the state, it confirmed to The Robot Report. Waymo said it will restrict freeway operations across all of its operating areas, as part of its safety-first approach. The ban comes R

Boston Dynamics’ Atlas humanoid lands first non-automotive trial
GlobalFoundries will test Boston Dynamics’ Atlas humanoid (right) in semiconductor manufacturing tasks in 2027. | Credit: Boston Dynamics GlobalFoundries, a leading semiconductor manufacturer, will become the first customer outside of the automotive industry to test Boston Dynamics’ Atlas humanoid. Atlas testing will begin in early 2027, marking a key step in Boston Dynamics’ efforts to commercialize its humanoid. GlobalFoundries is already a customer of Boston Dynamics, having deplo

Robo-soldier startup backed by Eric Trump misses payroll, may collapse
A humanoid robotics company backed and advised by President Donald Trump’s son, Eric Trump, has reportedly missed payroll and may be on the brink of collapse after a funding shortfall. The Silicon Valley startup, Foundation Future Industries, claims it is developing and testing robots for military roles on battlefields from Ukraine to Gaza. Foundation failed to pay employees for the payroll period ending on October 30, anonymous sources told CNBC. The startup’s staff is especially angry because

Six AGIBOT G2 Robots Complete 1,730 Tasks in 24-Hour Shanghai Convenience Store Trial
Four of the six robots pass a performance assessment as the trial puts customer service, retail operations and embodied AI capabilities to the test in a working store. Six AGIBOT G2 wheeled humanoid robots completed a 24-hour trial at an operating convenience store in Shanghai’s Xuhui District, performing a combined 1,730 tasks ranging from customer assistance and coffee preparation to shelf restocking and cleaning. The trial ran from 10 a.m. on November 2 to 10 a.m. on November 3 an

How Rhoda AI is using video data to create generalized AI models for robots
Rhoda pretrains its models on internet-scale video to build a strong prior on motion, physics, and physical interaction. | Source: Rhoda AI Earlier this year, Rhoda AI emerged from 18 months of stealth with $450 million in Series A funding. The company is working with customers in automotive, manufacturing, logistics, and e-commerce to deploy its direct video action, or DVA, model for robotics. “We’re focused on deploying in the real world, and that means you have to deal with the messiness of t

Learn how AI is changing supply chain robotics
From left to right: Lior Elazary, the co-founder and CEO of inVia Robotics, Josip Cesic, the co-founder and CEO of Gideon, and Jeff Mahler, the co-founder and CTO of Ambi Robotics. Supply chain robotics is undergoing rapid transformation. Companies hope to address labor shortages, improve efficiency, and build resilience against disruptions. Increasingly, they’re turning to robotics to solve these challenges. Currently, available technologies include warehouse automation, picking and packi