Humanoid Robots

Gathering extensive, high-quality, real-world datasets for this can be challenging, costly, and time-consuming. Synthetic data, generated from physically accurate simulations, addresses this challenge by accelerating data collection and providing the diversity needed to generalize robot learning models,

NVIDIA Isaac GR00T-Dreams is a blueprint that helps generate vast amounts of synthetic motion to teach robots new behaviors and how to adapt to changing environments.

Developers can first post-train NVIDIA Cosmos Predict world foundation models (WFMs) for their robot. Then, using a single image as input, GR00T-Dreams can help generate multiple videos of the robot performing new tasks in new environments. The blueprint then curates ‘dreams’ using Cosmos Reason and extracts action tokens — compressed, digestible pieces of data that are used to teach robots how to perform these new tasks.

The GR00T-Dreams blueprint complements the Isaac GR00T-Mimic blueprint. While GR00T-Mimic uses Omniverse and Cosmos to augment existing data, GR00T-Dreams uses Cosmos to generate entirely new data.

Humanoid robot grasping functionality requires human-like dexterous object manipulation skills, capable of performing both gross and fine-grained manipulation tasks. GR00T-Dexterity is a comprehensive suite of models and policies built using a reinforcement learning-based approach, combined with reference workflows, to enable the development of these advanced capabilities.

General-purpose navigation in complex and dynamic environments requires extensive tuning. With the GR00T-Mobility reference workflow, you can create a mobility generalist for navigating across varied settings and robot embodiments.

Achieving whole-body control in humanoid robots is challenging, demanding both stable manipulation and robust locomotion. GR00T-Control addresses this with a suite of advanced motion planning and control models, policies, and reference workflows, streamlining the development of effective control systems. 

By using imitation learning and teleoperated datasets, GR00T-Control facilitates training for robust, whole-body motion policies, enabling humanoid robots to learn dexterous manipulation and locomotion skills.

To improve situational awareness and interaction efficiency, humanoid robots require long-term memory for events, spaces, personalized settings, and context-aware responses. 

GR00T-Perception enables this with a robust suite of perception libraries, foundation models, and reference workflows built on Isaac Sim and Isaac ROS. These tools integrate advanced technologies like vision-language models and retrieval-augmented memory to enhance perception, cognition, and adaptability in humanoid robots.

Robot hardware is also crucial for running an ensemble of multimodal AI models that power humanoids with the right performance, latency, and functional safety in diverse conditions. 

NVIDIA Jetson AGX Thor, based on NVIDIA’s Blackwell GPU architecture, delivers ultra-high-performance AI compute and a new transformer engine. This delivers the necessary AI superpower at the edge to enable the new generation of humanoids.