Clarity for AI buildersSponsored by AI Routing
adoption

Nvidia’s GR00T 2 uses human demonstrations and video prediction to train robots

Nvidia’s GR00T 2 uses human demonstrations and video prediction to train robots
Nvidia’s GR00T 2 aims to reduce robot teleoperation by learning from human demonstrations, The Decoder reports. According to the outlet’s account of Nvidia’s research, the model predicts both a robot’s next movement and a short video of what it should see immediately afterward. Nvidia calls this a “world-action model,” distinguishing it from the vision-language-action approach used by its predecessor, GR00T 1, which translates images and language commands into movements. The intended benefit is to transfer actions demonstrated by people to robots with different physical designs. The Decoder reports that Nvidia trained GR00T 2 on a cluster with more than a thousand GB300 chips, using about 400,000 hours of footage of people performing tasks while wearing head-mounted cameras. For new tasks, people demonstrate movements using data gloves, specialized hand grippers or a finger exoskeleton. Training combines those recordings with general movement data from the target robot. The model then predicts a robotic arm in place of the human arm and derives corresponding control commands. This is not a claim that any ordinary video can immediately produce a reliable robot skill: the described process still requires robot-specific motion data and training. In laboratory tests described by The Decoder, a robot with five-fingered hands lit a match, sliced a banana and tossed packages onto a conveyor belt without task-specific teleoperation. Success rates ranged from 10 to 80 percent, depending on the task. A separate demonstration used a single video of a human hand movement to teach nail-hammering in simulation; that result should not be read as a physical-robot test. Additional conventional training improved results on a selected group of tasks. For 50 tasks on which the model already performed well after a few examples, less than three minutes of training raised success from 40 to 95 percent. The selection matters: this result shows rapid improvement on promising tasks, rather than establishing that the same training time would work across unfamiliar activities. For longer assembly jobs, Nvidia paired GR00T 2 with a higher-level language model that divided work into steps, marked grip points and triggered recovery routines. The combined system assembled toy models from loosely arranged parts in at least 97 percent of trials. Without the corrective language model and additional fine-tuning, toy-car assembly succeeded in 23 of 100 attempts. Because both planning support and fine-tuning differed, those figures do not isolate the contribution of either change alone. The report also describes GR00T 2 Turbo, a reduced-detail variant running on a humanoid robot’s Jetson Thor onboard computer. Tests included scooping popcorn, clearing tables and wiping liquids with a towel under the robot’s foot. The wiping task succeeded in just under half of attempts, underscoring the gap between demonstrating a capability and performing it reliably. Important limitations remain. The model does not yet process direct touch-sensor data and can miss subtle errors such as partially turned screws. Higher-level planning also adds delays: during trash sorting, the robot spent roughly one-third of its time idle, waiting more than seven seconds on average for the planning model. The reported research therefore points to a way to reduce task-specific teleoperation, not to established readiness for dependable real-world deployment.