Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

New paper release: Learning Agile Perceptive Traversal of Sparse 3D Structures for Humanoids Paper Videos We study how to do learning and sim2real for 3d traversal, such as monkey bars and overhanging obstacles.

C. Zhang

7,663 subscribers

83,433 views • 2 months ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

14 Comments

kache2 months ago

great work

C. Zhang2 months ago

The work demonstrates 1) hardware design, with a passive hook and a head-mounted solid-state lidar; 2) learning pipeline with teacher student learning; 3) sim2real designs for perception and actuation, including lidar noise patterns, temperature modelling and voltage modelling.

Henrique Ferrolho2 months ago

Really cool work, @ChongZzZhang! @CarlosMastalli, this is getting very close to doing the giant (gymnastics) manoeuver on the horizontal bar.

Jason Liu2 months ago

So much to learn from this paper, really cool chong!!!

lauw 🧢2 months ago

👀

Noiz Ai2 months ago

The sparse 3D traversal setup is a nice test because it forces perception and action to stay coupled when the terrain changes. The sim to real gap becomes much more visible than on flat ground demos.

cetusian2 months ago

head mounted solid state lidar is a nice call. does the noise model hold when the head is moving fast

C. Zhang2 months ago

By principle it's not dependent on speed unless it's a rocket...

cetusian2 months ago

rocket humanoid is the next paper then

Pabs • Robot Everest2 months ago

working on bringing robots to Everest let's collab

Aryan Dhawan2 months ago

How much recovery behavior survived sim-to-real without hand-tuned transitions?

Chandra Shekhar 📟2 months ago

W

Carlos DP 🤖🇺🇸2 months ago

lots of monkey bar robots lately, hmmm

Nabil Ess2 months ago

do you have any advice for anyone that wants to get into ETH zurich

Related Videos

🚀Create complex, modular 3D assets and scenes.🚀 Introducing #MoCA (Mixture-of-Components Attention), a new sparse attention method for compositional 3D generation. It scales to *many * components. paper: code:

🚀Create complex, modular 3D assets and scenes.🚀 Introducing #MoCA (Mixture-of-Components Attention), a new sparse attention method for compositional 3D generation. It scales to many components. paper: code:

Tengfei Wang

27,212 views • 11 months ago

Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusion Paper: (not yet) Project: Code:

Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusion Paper: (not yet) Project: Code:

MrNeRF

20,087 views • 2 years ago

The million-dollar question in humanoid robotics is: Can humanoids tap into Internet-scale training data such as online videos due to their human-like physique? Our #CoRL2024 oral paper showed the promise of humanoids learning new skills from single video demonstrations. (1/n)

The million-dollar question in humanoid robotics is: Can humanoids tap into Internet-scale training data such as online videos due to their human-like physique? Our #CoRL2024 oral paper showed the promise of humanoids learning new skills from single video demonstrations. (1/n)

Yuke Zhu

69,582 views • 2 years ago

Check out our new paper that turns (text, sparse images, videos) => (dynamic 3D scenes)! I can't get over how cool the interactive demo is. Try it out for yourself on the project page:

Check out our new paper that turns (text, sparse images, videos) => (dynamic 3D scenes)! I can't get over how cool the interactive demo is. Try it out for yourself on the project page:

Aleksander Holynski

37,520 views • 1 year ago

Native Action-Prior Learning from Videos for World Action Models paper:

Native Action-Prior Learning from Videos for World Action Models paper:

AK

19,186 views • 1 month ago

DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision paper page: We have witnessed significant progress in deep learning-based 3D vision, ranging from neural radiance field (NeRF) based 3D representation learning to applications in novel view synthesis (NVS). However, existing scene-level datasets for deep learning-based 3D vision, limited to either synthetic environments or a narrow selection of real-world scenes, are quite insufficient. This insufficiency not only hinders a comprehensive benchmark of existing methods but also caps what could be explored in deep learning-based 3D analysis. To address this critical gap, we present DL3DV-10K, a large-scale scene dataset, featuring 51.2 million frames from 10,510 videos captured from 65 types of point-of-interest (POI) locations, covering both bounded and unbounded scenes, with different levels of reflection, transparency, and lighting. We conducted a comprehensive benchmark of recent NVS methods on DL3DV-10K, which revealed valuable insights for future research in NVS. In addition, we have obtained encouraging results in a pilot study to learn generalizable NeRF from DL3DV-10K, which manifests the necessity of a large-scale scene-level dataset to forge a path toward a foundation model for learning 3D representation.

DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision paper page: We have witnessed significant progress in deep learning-based 3D vision, ranging from neural radiance field (NeRF) based 3D representation learning to applications in novel view synthesis (NVS). However, existing scene-level datasets for deep learning-based 3D vision, limited to either synthetic environments or a narrow selection of real-world scenes, are quite insufficient. This insufficiency not only hinders a comprehensive benchmark of existing methods but also caps what could be explored in deep learning-based 3D analysis. To address this critical gap, we present DL3DV-10K, a large-scale scene dataset, featuring 51.2 million frames from 10,510 videos captured from 65 types of point-of-interest (POI) locations, covering both bounded and unbounded scenes, with different levels of reflection, transparency, and lighting. We conducted a comprehensive benchmark of recent NVS methods on DL3DV-10K, which revealed valuable insights for future research in NVS. In addition, we have obtained encouraging results in a pilot study to learn generalizable NeRF from DL3DV-10K, which manifests the necessity of a large-scale scene-level dataset to forge a path toward a foundation model for learning 3D representation.

AK

49,969 views • 2 years ago

[1/N] Current visual geometry prediction models primarily rely on labeled 3D data. Our CVPR26 paper, Flow3r, allows additionally leveraging unlabeled videos (using flow supervision) for scalable visual geometry learning, enabling accurate multi-view 3D reconstruction in-the-wild.

[1/N] Current visual geometry prediction models primarily rely on labeled 3D data. Our CVPR26 paper, Flow3r, allows additionally leveraging unlabeled videos (using flow supervision) for scalable visual geometry learning, enabling accurate multi-view 3D reconstruction in-the-wild.

Shubham Tulsiani

17,487 views • 8 months ago

Tutorial paper squishy kue tart 3D - How to Make 3D tart paper squishy #papersquishy #tutorialsquishy #diy #fypn

Tutorial paper squishy kue tart 3D - How to Make 3D tart paper squishy #papersquishy #tutorialsquishy #diy #fypn

Squishy Kingdom ID

48,300 views • 3 years ago

📸 TOEM 2 GAMEPLAY PREMIERE 📸 The world is now 3D, more options for movement and traversal, and plenty of new friends to meet. Wishlist: ---- Huge thanks to Day of the Devs who let us be a part of their show, it's deeply appreciated!

📸 TOEM 2 GAMEPLAY PREMIERE 📸 The world is now 3D, more options for movement and traversal, and plenty of new friends to meet. Wishlist: ---- Huge thanks to Day of the Devs who let us be a part of their show, it's deeply appreciated!

TOEM 📸

17,966 views • 1 year ago

We are thrilled to share that our first paper from my new lab, Spateo ( for spatiotemporal modeling of molecular holograms, is now online in Cell: Spateo is a comprehensive analytical framework for 3D whole-embryo spatiotemporal modeling. Its advanced features include: • 3D alignment and reconstruction at the whole-mouse-embryo scale (see the animation). • 3D spatial domain digitization and cell-cell communication analysis to understand spatial gene expression gradients and both inter- and intracellular communication. • 3D morphometric and volumetric analyses along with 3D morphogenesis vector field modeling to quantify dynamics such as surface area, volume, and cell density across organs, and to dissect the interplay between morphogenesis factors and cell migration. • A “Google Earth”-like browser, Spateo-viewer ( and for interactive and intuitive exploration of 3D spatial data. • Additional features, such as RNA signal-based single-cell segmentation. We are also honored that Nature “News and Views” has highlighted this work as well: This is really an amazing outcome after two years' heroic revision process that rewrite the entire paper using a new data ( for whole mouse embryos.

We are thrilled to share that our first paper from my new lab, Spateo ( for spatiotemporal modeling of molecular holograms, is now online in Cell: Spateo is a comprehensive analytical framework for 3D whole-embryo spatiotemporal modeling. Its advanced features include: • 3D alignment and reconstruction at the whole-mouse-embryo scale (see the animation). • 3D spatial domain digitization and cell-cell communication analysis to understand spatial gene expression gradients and both inter- and intracellular communication. • 3D morphometric and volumetric analyses along with 3D morphogenesis vector field modeling to quantify dynamics such as surface area, volume, and cell density across organs, and to dissect the interplay between morphogenesis factors and cell migration. • A “Google Earth”-like browser, Spateo-viewer ( and for interactive and intuitive exploration of 3D spatial data. • Additional features, such as RNA signal-based single-cell segmentation. We are also honored that Nature “News and Views” has highlighted this work as well: This is really an amazing outcome after two years' heroic revision process that rewrite the entire paper using a new data ( for whole mouse embryos.

evo-devo

58,572 views • 2 years ago

Sharing our new work: UniForge3D generates detailed 3D meshes from sparse-view inputs with high fidelity! The code has been released and welcome to try it! Paper: Project Page: Code:

Sharing our new work: UniForge3D generates detailed 3D meshes from sparse-view inputs with high fidelity! The code has been released and welcome to try it! Paper: Project Page: Code:

Yuan Liu

29,511 views • 1 month ago

SAM 3D enables accurate 3D reconstruction from a single image, supporting real-world applications in editing, robotics, and interactive scene generation. Matt, a SAM 3D researcher, explains how the two-model design makes this possible for both people and complex environments. 🔗 Read the SAM 3D Objects research paper: 🔗 Read the SAM 3D Body research paper:

SAM 3D enables accurate 3D reconstruction from a single image, supporting real-world applications in editing, robotics, and interactive scene generation. Matt, a SAM 3D researcher, explains how the two-model design makes this possible for both people and complex environments. 🔗 Read the SAM 3D Objects research paper: 🔗 Read the SAM 3D Body research paper:

AI at Meta

18,031 views • 11 months ago

Very excited to announce: Keypoint Action Tokens! We found that LLMs can be repurposed as "imitation learning engines" for robots, by representing both observations & actions as 3D keypoints, and feeding into an LLM for in-context learning. See: More 👇

Very excited to announce: Keypoint Action Tokens! We found that LLMs can be repurposed as "imitation learning engines" for robots, by representing both observations & actions as 3D keypoints, and feeding into an LLM for in-context learning. See: More 👇

Edward Johns

32,586 views • 2 years ago

HACK: Learning a Parametric Head and Neck Model for High-fidelity Animation abs: paper page:

HACK: Learning a Parametric Head and Neck Model for High-fidelity Animation abs: paper page:

AK

90,160 views • 3 years ago

Can humanoids perform agile, autonomous, long-horizon parkour—based on what they see in the world? We present 𝗣𝗲𝗿𝗰𝗲𝗽𝘁𝗶𝘃𝗲 𝗛𝘂𝗺𝗮𝗻𝗼𝗶𝗱 𝗣𝗮𝗿𝗸𝗼𝘂𝗿 (𝗣𝗛𝗣): a framework that chains dynamic human skills using onboard depth perception for long-horizon traversal. 1/6

Can humanoids perform agile, autonomous, long-horizon parkour—based on what they see in the world? We present 𝗣𝗲𝗿𝗰𝗲𝗽𝘁𝗶𝘃𝗲 𝗛𝘂𝗺𝗮𝗻𝗼𝗶𝗱 𝗣𝗮𝗿𝗸𝗼𝘂𝗿 (𝗣𝗛𝗣): a framework that chains dynamic human skills using onboard depth perception for long-horizon traversal. 1/6

Zhen Wu

165,184 views • 8 months ago

New paper from our lab Nature Communications! Aakash Saha and colleagues reveal the dynamics and mechanism of target DNA traversal in #CRISPR-Cas12a, a conundrum in the field! #compchem We thank the amazing #HPC resources PSC (#Anton2) and San Diego Supercomputer Center

New paper from our lab Nature Communications! Aakash Saha and colleagues reveal the dynamics and mechanism of target DNA traversal in #CRISPR-Cas12a, a conundrum in the field! #compchem We thank the amazing #HPC resources PSC (#Anton2) and San Diego Supercomputer Center

Giulia Palermo

17,905 views • 2 years ago

EditHero A Benchmark for Long-Horizon Part-Level 3D Editing and Vibe Modeling paper:

EditHero A Benchmark for Long-Horizon Part-Level 3D Editing and Vibe Modeling paper:

AK

19,121 views • 1 month ago

--Towards Scalable Sidewalk Autonomy (2/4)-- Introducing UrbanVerse — turning city-tour videos into physics-aware, interactive simulations for scalable robot learning in NVIDIA Omniverse. 🌆 Real-world scene layouts 🧱100K+ 3D assets ICLR’26 paper and **Open-source** assets at 🔗 We thank Coco Robotics for robot support and NVIDIA Academic Grant support NVIDIA Robotics

--Towards Scalable Sidewalk Autonomy (2/4)-- Introducing UrbanVerse — turning city-tour videos into physics-aware, interactive simulations for scalable robot learning in NVIDIA Omniverse. 🌆 Real-world scene layouts 🧱100K+ 3D assets ICLR’26 paper and Open-source assets at 🔗 We thank Coco Robotics for robot support and NVIDIA Academic Grant support NVIDIA Robotics

Bolei Zhou

15,081 views • 7 months ago

We are releasing uCO3D! Built to supercharge 3D GenAI and digital-twin models, this evolution of CO3D features more and higher-quality object videos from 1k categories, 3D Gaussian Splats, and streamlined OSS tools. 💻Data&code: 📄Paper:

We are releasing uCO3D! Built to supercharge 3D GenAI and digital-twin models, this evolution of CO3D features more and higher-quality object videos from 1k categories, 3D Gaussian Splats, and streamlined OSS tools. 💻Data&code: 📄Paper:

David Novotny

16,742 views • 1 year ago