Загрузка видео...

Не удалось загрузить видео

На главную

Boston Dynamics collaborated with NVIDIA to demonstrate DextrAH-RGB, a workflow for dexterous grasping from stereo RGB input. The end-to-end policy for Atlas robot, trained entirely in NVIDIA Isaac Lab, transfers zero-shot from simulation to the real robot.

TheHumanoidHub's profile picture

The Humanoid Hub

112,691 subscribers

79,743 просмотров • 1 год назад •via X (Twitter)

Новости и политикаНаука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 6

Фото профиля The Humanoid Hub
The Humanoid Hub1 год назад

Trains a fabric-guided teacher policy with RL in simulation, then distills it into a stereo-RGB student policy via an imitation learning framework for zero-shot grasping on novel objects. Technical blog:

Фото профиля The Information
The Information1 год назад

Meta AI researchers are fretting over the threat of Chinese AI, whose quality caught American firms, including OpenAI, by surprise.

Фото профиля Reborn
Reborn1 год назад

NVIDIA’s Isaac platform has quietly become one of the most comprehensive robotics dev stack out there.

Фото профиля KecksbeLit
KecksbeLit1 год назад

i would love to see an uncut version of like 20 minutes to see where we rly are this video has a lot of cuts in it still amazing though

Фото профиля Cornelius Ong
Cornelius Ong1 год назад

Yo that's so sickkk

Фото профиля ChatableApps
ChatableApps1 год назад

So basically, we’re one step closer to having robots that can steal our snacks without us noticing? 🍕🤖

Похожие видео

Next step in dynamic dexterous grasping from NVIDIA: DextrAH-RGB! No more depth. We’re now consuming RGB stereo pairs, and the resulting perceptual system is much more robust. Trained entirely in sim (IsaacLab), leveraging fast tiled rendering, and deployed zero-shot to real.
0:47

Next step in dynamic dexterous grasping from NVIDIA: DextrAH-RGB! No more depth. We’re now consuming RGB stereo pairs, and the resulting perceptual system is much more robust. Trained entirely in sim (IsaacLab), leveraging fast tiled rendering, and deployed zero-shot to real.

Nathan Ratliff

47,247 просмотров • 1 год назад

We talked to Ritvik Singh about how you can train sim-to-real dexterous manipulation policies using NVIDIA Isaac. This robot is grasping objects using pure RGB stereo: take in images from a camera pair and predict what to do, all without training in the real world.
1:08

We talked to Ritvik Singh about how you can train sim-to-real dexterous manipulation policies using NVIDIA Isaac. This robot is grasping objects using pure RGB stereo: take in images from a camera pair and predict what to do, all without training in the real world.

Chris Paxton

20,067 просмотров • 11 месяцев назад

Boston Dynamics taught Atlas a Rabona kick. Learned from human mocap, retargeted to Atlas, trained in sim via RL, deployed zero-shot to the real robot. Soccer skills demand whole-body coordination, and similar recipe transfers to warehouse work.
0:58

Boston Dynamics taught Atlas a Rabona kick. Learned from human mocap, retargeted to Atlas, trained in sim via RL, deployed zero-shot to the real robot. Soccer skills demand whole-body coordination, and similar recipe transfers to warehouse work.

The Humanoid Hub

11,268 просмотров • 2 месяцев назад

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)
0:33

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)

Chris Paxton

20,486 просмотров • 1 год назад

NVIDIA announces the first open humanoid robot reference design built for robotics research. The NVIDIA Isaac GR00T Reference Humanoid Robot combines the Unitree H2 humanoid robot, Sharpa Wave five-fingered hands for dexterous manipulation, Jetson Thor onboard compute, and Isaac GR00T open software and models, giving researchers a full-stack platform from data capture to model deployment. Read the #NVIDIAGTC Taipei announcement:
0:20

NVIDIA announces the first open humanoid robot reference design built for robotics research. The NVIDIA Isaac GR00T Reference Humanoid Robot combines the Unitree H2 humanoid robot, Sharpa Wave five-fingered hands for dexterous manipulation, Jetson Thor onboard compute, and Isaac GR00T open software and models, giving researchers a full-stack platform from data capture to model deployment. Read the #NVIDIAGTC Taipei announcement:

NVIDIA Robotics

169,453 просмотров • 2 месяцев назад

What if we can simulate an *interactive 3D world*, from a single image, in the wild, in real time? Introducing PointWorld-1B: a large pre-trained 3D world model that predicts env dynamics given RGB-D capture and robot actions. 🌐 from Stanford University NVIDIA
2:08

What if we can simulate an *interactive 3D world*, from a single image, in the wild, in real time? Introducing PointWorld-1B: a large pre-trained 3D world model that predicts env dynamics given RGB-D capture and robot actions. 🌐 from Stanford University NVIDIA

Wenlong Huang

275,348 просмотров • 7 месяцев назад

Physics AI 🌊 Jensen just unveiled the NVIDIA Isaac GR00T Reference Humanoid Robot at GTC Taipei--the world’s first open-source humanoid robot reference design Built on Jetson Thor edge computing and the Isaac GR00T open platform, it features a Unitree H2 Plus body with Sharpa five-finger dexterous hands and 75 degrees of freedom. Even before Physical AI truly explodes, NVIDIA has built a complete end-to-end full-stack infrastructure: >chips (Jetson Thor + Blackwell), >synthetic data (GR00T-Dreams), >world models (Cosmos), >foundation models (GR00T N-series VLA), >high-fidelity simulation platforms (Isaac Sim + Isaac Lab + Omniverse). Dozens of humanoid robot companies ,including Agility Robotics, Aglie robotics,Boston Dynamics, Figure, 1X, NEURA Robotics, XPENG Robotics, and more are already running large-scale daily simulation training on this platform, rapidly iterating perception, reasoning, and actions in virtual environments, speeding up the jump from lab prototypes to real factory deployment. By providing an open reference design and full-stack tools, NVIDIA has dramatically lowered the barrier, letting every robotics team join the Physical AI wave with low cost and high efficiency.
4:31

Physics AI 🌊 Jensen just unveiled the NVIDIA Isaac GR00T Reference Humanoid Robot at GTC Taipei--the world’s first open-source humanoid robot reference design Built on Jetson Thor edge computing and the Isaac GR00T open platform, it features a Unitree H2 Plus body with Sharpa five-finger dexterous hands and 75 degrees of freedom. Even before Physical AI truly explodes, NVIDIA has built a complete end-to-end full-stack infrastructure: >chips (Jetson Thor + Blackwell), >synthetic data (GR00T-Dreams), >world models (Cosmos), >foundation models (GR00T N-series VLA), >high-fidelity simulation platforms (Isaac Sim + Isaac Lab + Omniverse). Dozens of humanoid robot companies ,including Agility Robotics, Aglie robotics,Boston Dynamics, Figure, 1X, NEURA Robotics, XPENG Robotics, and more are already running large-scale daily simulation training on this platform, rapidly iterating perception, reasoning, and actions in virtual environments, speeding up the jump from lab prototypes to real factory deployment. By providing an open reference design and full-stack tools, NVIDIA has dramatically lowered the barrier, letting every robotics team join the Physical AI wave with low cost and high efficiency.

CyberRobo

17,092 просмотров • 2 месяцев назад

Humanoid robot development is too fragmented to scale. 🤖 With NVIDIA Isaac GR00T, developers unify simulation, data, and training in one open platform, cutting setup time from days to hours. See how Isaac GR00T helps research teams move faster from robot bring-up to skill development and real-world validation. Watch now 📺
0:38

Humanoid robot development is too fragmented to scale. 🤖 With NVIDIA Isaac GR00T, developers unify simulation, data, and training in one open platform, cutting setup time from days to hours. See how Isaac GR00T helps research teams move faster from robot bring-up to skill development and real-world validation. Watch now 📺

NVIDIA Robotics

22,573 просмотров • 1 месяц назад

Boston Dynamics has demonstrated its new Atlas humanoid robot lifting and carrying a 50-pound fridge. The robot uses its whole body to balance and move the heavy object in a natural, human-like way. Before doing it in the real world, Atlas practiced the task for millions of hours in a virtual simulation.
0:26

Boston Dynamics has demonstrated its new Atlas humanoid robot lifting and carrying a 50-pound fridge. The robot uses its whole body to balance and move the heavy object in a natural, human-like way. Before doing it in the real world, Atlas practiced the task for millions of hours in a virtual simulation.

Space and Technology

180,505 просмотров • 2 месяцев назад

Historically, RL policies for robots have been trained in synthetic, untextured environments, limiting perceptive policies to depth images where the sim-to-real gap is manageable. RGB has always had more potential, but leveraging it to train policies in simulation remained an open problem. Partnering with Niantic Spatial 🌎 and NVIDIA Robotics, we built a pipeline that addresses exactly that. We can now scan a real deployment site with off-the-shelf hardware, reconstruct it into a photorealistic Gaussian splat, and run massively parallel RL training. The policies trained in our Gym environment then transfer zero-shot to the real robot and environments they were trained for. This enables faster deployment of more capable and robust policies for the end user. The new resulting capabilities are a big step towards solving sim-to-real and also apply well beyond navigation. Read the full technical breakdown on our blog; link in the comments. #HumanoidRobots #Flexion #NianticSpatial #NVIDIA
1:21

Historically, RL policies for robots have been trained in synthetic, untextured environments, limiting perceptive policies to depth images where the sim-to-real gap is manageable. RGB has always had more potential, but leveraging it to train policies in simulation remained an open problem. Partnering with Niantic Spatial 🌎 and NVIDIA Robotics, we built a pipeline that addresses exactly that. We can now scan a real deployment site with off-the-shelf hardware, reconstruct it into a photorealistic Gaussian splat, and run massively parallel RL training. The policies trained in our Gym environment then transfer zero-shot to the real robot and environments they were trained for. This enables faster deployment of more capable and robust policies for the end user. The new resulting capabilities are a big step towards solving sim-to-real and also apply well beyond navigation. Read the full technical breakdown on our blog; link in the comments. #HumanoidRobots #Flexion #NianticSpatial #NVIDIA

Flexion

70,906 просмотров • 23 дней назад

Real → Sim → Robot — fast. With Gaussian splats, GRID brings physical assets into simulation tools like AirGen and NVIDIA Isaac Sim, enabling AI skills to be tested in sim and deployed to real robots in minutes. #Robotics #PhysicalAI #Simulation
0:58

Real → Sim → Robot — fast. With Gaussian splats, GRID brings physical assets into simulation tools like AirGen and NVIDIA Isaac Sim, enabling AI skills to be tested in sim and deployed to real robots in minutes. #Robotics #PhysicalAI #Simulation

General Robotics

12,257 просмотров • 1 год назад

Trained a humanoid entirely in a 3D scan of the office. Zero real-world fine-tuning. It just walked in and worked. RL needs hundreds of thousands of attempts, and real robots can't afford to crash. A misjudged gap or a glass door collision breaks hardware and costs hours resetting. So you train in a sim. But sim policies usually train on randomized, untextured geometry; depth is easy to fake. The robot learns structure, not the real world: no materials, no lighting, no idea what anything actually is. RGB cameras carry all of that but training RGB policies in generic fake worlds won’t generalize to the real world. Niantic Spatial 🌎 Scaniverse reconstructs your scan of the real deployment site. One 360° camera walkthrough → photorealistic 3D Gaussian splat at metric scale → collision mesh pulled from the same reconstruction, so vision and physics match exactly. Drops straight into NVIDIA Isaac Sim/Lab, no manual conversion. Flexion simulation-first approach then seamlessly enables the training of RGB-only nav policies inside that reconstruction. With added domain randomization + large image encoders for robustness, this deploys straight to hardware. No real-world fine-tuning. Deployment: months of on-site adaptation → days. Tune into the NVIDIA livestream on 12 August to hear how these companies are closing the sim2real gap: NVIDIA Robotics ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:47

Trained a humanoid entirely in a 3D scan of the office. Zero real-world fine-tuning. It just walked in and worked. RL needs hundreds of thousands of attempts, and real robots can't afford to crash. A misjudged gap or a glass door collision breaks hardware and costs hours resetting. So you train in a sim. But sim policies usually train on randomized, untextured geometry; depth is easy to fake. The robot learns structure, not the real world: no materials, no lighting, no idea what anything actually is. RGB cameras carry all of that but training RGB policies in generic fake worlds won’t generalize to the real world. Niantic Spatial 🌎 Scaniverse reconstructs your scan of the real deployment site. One 360° camera walkthrough → photorealistic 3D Gaussian splat at metric scale → collision mesh pulled from the same reconstruction, so vision and physics match exactly. Drops straight into NVIDIA Isaac Sim/Lab, no manual conversion. Flexion simulation-first approach then seamlessly enables the training of RGB-only nav policies inside that reconstruction. With added domain randomization + large image encoders for robustness, this deploys straight to hardware. No real-world fine-tuning. Deployment: months of on-site adaptation → days. Tune into the NVIDIA livestream on 12 August to hear how these companies are closing the sim2real gap: NVIDIA Robotics ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

116,936 просмотров • 3 дней назад

Excited to introduce StereoPolicy, led by Evans Han. 📷📷🤖StereoPolicy is an effective way to add geometric cues to modern robot policy models while keeping the strengths of pretrained 2D encoders. ⁉️Why stereo for robot manipulation? Monocular RGB often lacks the depth cues needed for precise manipulation, while RGB-D and point clouds can be noisy or brittle, especially on reflective and transparent objects in real-world deployment. Instead of explicitly reconstructing disparity, depth, or point clouds, StereoPolicy directly fuses synchronized left/right RGB views to learn implicit stereo cues, avoiding extra reconstruction latency that can make real-time manipulation difficult. Project Page:
0:50

Excited to introduce StereoPolicy, led by Evans Han. 📷📷🤖StereoPolicy is an effective way to add geometric cues to modern robot policy models while keeping the strengths of pretrained 2D encoders. ⁉️Why stereo for robot manipulation? Monocular RGB often lacks the depth cues needed for precise manipulation, while RGB-D and point clouds can be noisy or brittle, especially on reflective and transparent objects in real-world deployment. Instead of explicitly reconstructing disparity, depth, or point clouds, StereoPolicy directly fuses synchronized left/right RGB views to learn implicit stereo cues, avoiding extra reconstruction latency that can make real-time manipulation difficult. Project Page:

Ruohan Zhang

850,936 просмотров • 2 месяцев назад

Big news! World Labs' Marble is officially compatible with NVIDIA Robotics' Isaac Sim. Instead of showcasing their highlight reel, our team at Lightwheel created a full environment from a 360 pano using Marble and showed you can train directly in sim. The only non-generated assets are the robot arms, coffee, and bowl. Everything else was generated with Marble. Not sure how to do this yourself? NVIDIA wrote a technical blog with full instructions. You will learn how to: 1. Export scenes from World Labs' Marble as Gaussian splats 2. Convert to USD using NVIDIA Omniverse NuRec 3. Import into NVIDIA Isaac Sim 4. Add a robot and run the simulation Read the guide ➡️ #WorldModels #Robotics
0:32

Big news! World Labs' Marble is officially compatible with NVIDIA Robotics' Isaac Sim. Instead of showcasing their highlight reel, our team at Lightwheel created a full environment from a 360 pano using Marble and showed you can train directly in sim. The only non-generated assets are the robot arms, coffee, and bowl. Everything else was generated with Marble. Not sure how to do this yourself? NVIDIA wrote a technical blog with full instructions. You will learn how to: 1. Export scenes from World Labs' Marble as Gaussian splats 2. Convert to USD using NVIDIA Omniverse NuRec 3. Import into NVIDIA Isaac Sim 4. Add a robot and run the simulation Read the guide ➡️ #WorldModels #Robotics

Jonathan Stephens

13,783 просмотров • 7 месяцев назад

NVIDIA's Jim Fan has verified from reliable sources that the video by EngineAI is a real robot demo and not CGI. The controller is a neural net trained in the NVIDIA Isaac simulator using reinforcement learning. There’s also another video from a different angle.
0:06

NVIDIA's Jim Fan has verified from reliable sources that the video by EngineAI is a real robot demo and not CGI. The controller is a neural net trained in the NVIDIA Isaac simulator using reinforcement learning. There’s also another video from a different angle.

The Humanoid Hub

148,535 просмотров • 1 год назад

Reality of robotics: humanoid kung fu is solved before they can open doors with RGB. Here we are. Introducing the frontier of sim2real at NVIDIA GEAR. 100% sim data. RGB input only. Code name: 𝗗𝗼𝗼𝗿𝗠𝗮𝗻. We are opening the sim-to-real door. 🧵
1:01

Reality of robotics: humanoid kung fu is solved before they can open doors with RGB. Here we are. Introducing the frontier of sim2real at NVIDIA GEAR. 100% sim data. RGB input only. Code name: 𝗗𝗼𝗼𝗿𝗠𝗮𝗻. We are opening the sim-to-real door. 🧵

Haoru Xue

373,193 просмотров • 8 месяцев назад

Dex is ready for the factory floor. 🏭 Learn how Richtech Robotics built this mobile humanoid robot, powered by NVIDIA Jetson AGX Thor and trained in Isaac Sim, to handle light to medium industrial tasks. Read the blog ➡️
0:36

Dex is ready for the factory floor. 🏭 Learn how Richtech Robotics built this mobile humanoid robot, powered by NVIDIA Jetson AGX Thor and trained in Isaac Sim, to handle light to medium industrial tasks. Read the blog ➡️

NVIDIA Robotics

25,463 просмотров • 8 месяцев назад

Netflix funded $100 million to make Atlas, a film where JLO is forced to team up with an AI robot in order to prevent an apocalypse In the end JLO and the AI Robot expressed their feelings for each after winning the final battle, yes this is real.
2:19

Netflix funded $100 million to make Atlas, a film where JLO is forced to team up with an AI robot in order to prevent an apocalypse In the end JLO and the AI Robot expressed their feelings for each after winning the final battle, yes this is real.

Ahmed/The Ears/IG: BigBizTheGod 🇸🇴

829,508 просмотров • 2 лет назад

Boston Dynamics has released a new video of their next-generation humanoid robot, Atlas. "We are excited to share some of our progress on developing Large Behavior Models for Atlas. This work is part of a collaboration between AI research teams at Toyota Research Institute (TRI) and Boston Dynamics. We have been building end-to-end language-conditioned policies that enable Atlas to accomplish long-horizon manipulation tasks."
2:02

Boston Dynamics has released a new video of their next-generation humanoid robot, Atlas. "We are excited to share some of our progress on developing Large Behavior Models for Atlas. This work is part of a collaboration between AI research teams at Toyota Research Institute (TRI) and Boston Dynamics. We have been building end-to-end language-conditioned policies that enable Atlas to accomplish long-horizon manipulation tasks."

Sawyer Merritt

200,744 просмотров • 11 месяцев назад

Robot foundation models are limited by costly real data, while simulation data is plentiful but visually mismatched to reality. We present Point Bridge, a method that enables zero-shot sim-to-real transfer for robot learning with minimal visual alignment.
0:50

Robot foundation models are limited by costly real data, while simulation data is plentiful but visually mismatched to reality. We present Point Bridge, a method that enables zero-shot sim-to-real transfer for robot learning with minimal visual alignment.

Siddhant Haldar

19,953 просмотров • 6 месяцев назад