Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Introduce HumanPlus - Autonomous Skills part Humanoids are born for using human data. Imitating humans, our humanoid learns: - fold sweatshirts - unload objects from warehouse racks - diverse locomotion skills (squatting, jumping, standing) - greet another robot Open-sourced!

zipengfu's profile picture

Zipeng Fu

13,247 subscribers

158,140 Aufrufe • vor 2 Jahren •via X (Twitter)

BildungWissenschaft & TechnologieGaming
LIVE

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Zipeng Fu
Zipeng Fuvor 2 Jahren

We build our customized 33-DoF humanoid, and a data collection pipeline through real-time shadowing in the real world.

Profilbild von Zipeng Fu
Zipeng Fuvor 2 Jahren

Using the data collected through shadowing, we then perform supervised behavior cloning to train skill policies using egocentric vision. We introduce Humanoid Imitation Transformer. Based on ACT, HIT adds forward dynamics prediction on image feature space as a regularization.

Profilbild von Zipeng Fu
Zipeng Fuvor 2 Jahren

Compared to baselines, HIT uses - binocular vision, thus having implicit stereos for depth information - visual feedback better, avoiding overfitting to proprioception given small-sized demos

Profilbild von Zipeng Fu
Zipeng Fuvor 2 Jahren

Besides vision-based whole-body manipulation skills, our humanoid has strong locomotion skills: - outperforming H1 default standing controller under strong perturbation forces - enabling more whole-body skills like squatting and jumping

Profilbild von Zipeng Fu
Zipeng Fuvor 2 Jahren

This project is not possible without our team of experts, covering from computer graphics to robot learning to robot hardware: - co-leads: @qingqing_zhao_ @Qi_Wu577 - advisors: @chelseabfinn @GordonWetzstein project website: hardware: code:

Profilbild von Karol Hausman
Karol Hausmanvor 2 Jahren

Congrats @zipengfu !! 🎉

Profilbild von Yu Yang
Yu Yangvor 2 Jahren

Pretty cool!

Profilbild von Joe Valentine
Joe Valentinevor 2 Jahren

@DrDisrespect robodoc?

Profilbild von BotBot
BotBotvor 2 Jahren

Wow! Incredible, is the demo unit the Unitree H1?

Profilbild von Ak
Akvor 2 Jahren

@heyjchu

Ähnliche Videos

Introduce HumanPlus - Shadowing part Humanoids are born for using human data. We build a real-time shadowing system using a single RGB camera and a whole-body policy for cloning human motion. Examples: - boxing🥊 - playing the piano🎹/ping pong - tossing - typing Open-sourced!
0:53

Introduce HumanPlus - Shadowing part Humanoids are born for using human data. We build a real-time shadowing system using a single RGB camera and a whole-body policy for cloning human motion. Examples: - boxing🥊 - playing the piano🎹/ping pong - tossing - typing Open-sourced!

Zipeng Fu

230,117 Aufrufe • vor 2 Jahren

Excited to introduce TWIST2, our next-generation humanoid data collection system. TWIST2 is portable (use anywhere, no MoCap), scalable (100+ demos in 15 mins), and holistic (unlock major whole-body human skills). Fully open-sourced:
1:59

Excited to introduce TWIST2, our next-generation humanoid data collection system. TWIST2 is portable (use anywhere, no MoCap), scalable (100+ demos in 15 mins), and holistic (unlock major whole-body human skills). Fully open-sourced:

Yanjie Ze

100,291 Aufrufe • vor 7 Monaten

NVIDIA has open-sourced SONIC, a humanoid behavior foundation model that gives robots a core set of motor skills learned from large-scale human motion data.
0:48

NVIDIA has open-sourced SONIC, a humanoid behavior foundation model that gives robots a core set of motor skills learned from large-scale human motion data.

The Humanoid Hub

33,542 Aufrufe • vor 3 Monaten

Scientists developed HumanX, a system that lets humanoid robots learn new skills by watching just one short human video. It learns directly from demonstrations instead of large datasets or complex training rewards. Using this, the Unitree G1 humanoid robot learned 10 different real-world skills without extra training.
0:34

Scientists developed HumanX, a system that lets humanoid robots learn new skills by watching just one short human video. It learns directly from demonstrations instead of large datasets or complex training rewards. Using this, the Unitree G1 humanoid robot learned 10 different real-world skills without extra training.

Space and Technology

27,686 Aufrufe • vor 2 Monaten

Excited to introduce COLA: Learning Human–Humanoid Coordination for Collaborative Object Carrying 🤝🤖 COLA makes humanoids truly helpful in human collaboration — capable of carrying objects, pushing carts, or responding to human push commands. It provides a proprioception-only policy for compliant human–humanoid coordination across diverse movement patterns. The core idea is simple yet effective: 👉 Fine-tune a collaborative policy from a locomotion policy using a residual teacher. 👉 Train in simulation and distill to a real-world student policy for deployment. Paper: Project:
3:13

Excited to introduce COLA: Learning Human–Humanoid Coordination for Collaborative Object Carrying 🤝🤖 COLA makes humanoids truly helpful in human collaboration — capable of carrying objects, pushing carts, or responding to human push commands. It provides a proprioception-only policy for compliant human–humanoid coordination across diverse movement patterns. The core idea is simple yet effective: 👉 Fine-tune a collaborative policy from a locomotion policy using a residual teacher. 👉 Train in simulation and distill to a real-world student policy for deployment. Paper: Project:

Siyuan Huang

21,892 Aufrufe • vor 8 Monaten

The million-dollar question in humanoid robotics is: Can humanoids tap into Internet-scale training data such as online videos due to their human-like physique? Our #CoRL2024 oral paper showed the promise of humanoids learning new skills from single video demonstrations. (1/n)
0:49

The million-dollar question in humanoid robotics is: Can humanoids tap into Internet-scale training data such as online videos due to their human-like physique? Our #CoRL2024 oral paper showed the promise of humanoids learning new skills from single video demonstrations. (1/n)

Yuke Zhu

69,574 Aufrufe • vor 1 Jahr

See our robot open doors, water plants, assemble a box, and more by learning from watching humans. Using <1hr of robot data.
0:59

See our robot open doors, water plants, assemble a box, and more by learning from watching humans. Using <1hr of robot data.

Skild AI

36,023 Aufrufe • vor 5 Monaten

'LATENT' learns tennis skills for humanoid robots from human motion data. The robot can sustain multi-shot rallies, handle ball speeds of 15+ m/s, and showed a 90.9% success rate for the forehand. No onboard cameras or vision models, relies on external MoCap for high-precision, low-latency ball tracking. Paper:
1:31

'LATENT' learns tennis skills for humanoid robots from human motion data. The robot can sustain multi-shot rallies, handle ball speeds of 15+ m/s, and showed a 90.9% success rate for the forehand. No onboard cameras or vision models, relies on external MoCap for high-precision, low-latency ball tracking. Paper:

The Humanoid Hub

63,824 Aufrufe • vor 3 Monaten

Our humanoid now learns loco-manip skills that generalize across space (Stanford) &amp; time (day &amp; night), using egocentric vision, trained only in simulation
1:03

Our humanoid now learns loco-manip skills that generalize across space (Stanford) &amp; time (day &amp; night), using egocentric vision, trained only in simulation

Yanjie Ze

93,348 Aufrufe • vor 8 Monaten

Earlier this year, AgiBot open-sourced a large-scale dataset of real-world humanoid robot tasks. Now, they've announced AgiBot Digital World, a large-scale simulation framework that includes an open-sourced dataset covering 180+ object categories and 12 core skills.
2:25

Earlier this year, AgiBot open-sourced a large-scale dataset of real-world humanoid robot tasks. Now, they've announced AgiBot Digital World, a large-scale simulation framework that includes an open-sourced dataset covering 180+ object categories and 12 core skills.

The Humanoid Hub

43,709 Aufrufe • vor 1 Jahr

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.
0:45

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

The Humanoid Hub

17,395 Aufrufe • vor 8 Monaten

Imagine robots learning new skills—without any robot data. Today, we're excited to release EgoZero: our first steps in training robot policies that operate in unseen environments, solely from data collected through humans wearing Aria smart glasses. 🧵👇
1:24

Imagine robots learning new skills—without any robot data. Today, we're excited to release EgoZero: our first steps in training robot policies that operate in unseen environments, solely from data collected through humans wearing Aria smart glasses. 🧵👇

Lerrel Pinto

42,538 Aufrufe • vor 1 Jahr

Introducing Omnigrasp: Grasping Diverse Objects with Simulated Humanoids. With Omnigrasp, we show that we can control a humanoid equipped with dexterous hands to grasp diverse objects (>1200) and follow diverse trajectories, with one policy! 🌐: 📜:
3:05

Introducing Omnigrasp: Grasping Diverse Objects with Simulated Humanoids. With Omnigrasp, we show that we can control a humanoid equipped with dexterous hands to grasp diverse objects (>1200) and follow diverse trajectories, with one policy! 🌐: 📜:

Zhengyi “Zen” Luo

92,104 Aufrufe • vor 1 Jahr

Can humanoids perform agile, autonomous, long-horizon parkour—based on what they see in the world? We present 𝗣𝗲𝗿𝗰𝗲𝗽𝘁𝗶𝘃𝗲 𝗛𝘂𝗺𝗮𝗻𝗼𝗶𝗱 𝗣𝗮𝗿𝗸𝗼𝘂𝗿 (𝗣𝗛𝗣): a framework that chains dynamic human skills using onboard depth perception for long-horizon traversal. 1/6
0:29

Can humanoids perform agile, autonomous, long-horizon parkour—based on what they see in the world? We present 𝗣𝗲𝗿𝗰𝗲𝗽𝘁𝗶𝘃𝗲 𝗛𝘂𝗺𝗮𝗻𝗼𝗶𝗱 𝗣𝗮𝗿𝗸𝗼𝘂𝗿 (𝗣𝗛𝗣): a framework that chains dynamic human skills using onboard depth perception for long-horizon traversal. 1/6

Zhen Wu

142,536 Aufrufe • vor 4 Monaten

How can robots learn generalizable manipulation skills for diverse objects? Going beyond pick-and-place, our recent work “HACMan” enables complex interactions for unseen objects, such as flipping, pushing, or tilting, using spatial action maps + RL with point clouds. (w/ @MetaAI)
1:14

How can robots learn generalizable manipulation skills for diverse objects? Going beyond pick-and-place, our recent work “HACMan” enables complex interactions for unseen objects, such as flipping, pushing, or tilting, using spatial action maps + RL with point clouds. (w/ @MetaAI)

Wenxuan Zhou

49,846 Aufrufe • vor 3 Jahren

Our 6ft humanoid HumanPlus at Stanford can autonomously put on a Nike skateboard shoe, tie shoelaces, stand up and walk. Using two transformers & dual RGB vision, it integrates two recipes of general robotics end-to-end: - imitating humans in real world - large-scale RL in sim
0:46

Our 6ft humanoid HumanPlus at Stanford can autonomously put on a Nike skateboard shoe, tie shoelaces, stand up and walk. Using two transformers & dual RGB vision, it integrates two recipes of general robotics end-to-end: - imitating humans in real world - large-scale RL in sim

Zipeng Fu

245,139 Aufrufe • vor 2 Jahren

The Warp team uses a lot of Skills to speed up our workflow, so we open-sourced them. &gt; npx skills add warpdotdev/oz-skills Includes Skills for SEO and accessibility auditing, documentation writing, DevOps tasks like terraform configuration, and more.
0:21

The Warp team uses a lot of Skills to speed up our workflow, so we open-sourced them. &gt; npx skills add warpdotdev/oz-skills Includes Skills for SEO and accessibility auditing, documentation writing, DevOps tasks like terraform configuration, and more.

Warp

232,739 Aufrufe • vor 1 Monat

Introduce Open-𝐓𝐞𝐥𝐞𝐕𝐢𝐬𝐢𝐨𝐧🤖: ⁣ We need an intuitive and remote teleoperation interface to collect more robot data. 𝐓𝐞𝐥𝐞𝐕𝐢𝐬𝐢𝐨𝐧 lets you immersively operate a robot even if you are 3000 miles away, like in the movie 𝘈𝘷𝘢𝘵𝘢𝘳. Open-sourced!
1:40

Introduce Open-𝐓𝐞𝐥𝐞𝐕𝐢𝐬𝐢𝐨𝐧🤖: ⁣ We need an intuitive and remote teleoperation interface to collect more robot data. 𝐓𝐞𝐥𝐞𝐕𝐢𝐬𝐢𝐨𝐧 lets you immersively operate a robot even if you are 3000 miles away, like in the movie 𝘈𝘷𝘢𝘵𝘢𝘳. Open-sourced!

Xuxin Cheng

329,376 Aufrufe • vor 1 Jahr

China's G1 humanoid robot is mastering combat skills at a terrifying rate
0:41

China's G1 humanoid robot is mastering combat skills at a terrifying rate

Interesting AF

157,726 Aufrufe • vor 8 Monaten

Another American industrial humanoid robot🤖🏭 Noble Machines (formerly UCR) has just emerged from stealth mode. Launching the first humanoid robot. Founded only 18 months ago, it has already been deployed in a Fortune Global 500 factory. Teams from SpaceX, NASA, and Apple specialize in the toughest, most tiring, and most dangerous industrial tasks: >27kg heavy load >5-hour battery life >Walking speed 0.8m/s >Climbing stairs, traversing scaffolding, and navigating chaotic construction sites > Modular end effector, allowing for quick tool change. > AI-controlled operation with end-to-end autonomy; learns new skills in hours >Autonomous operation + Telep-op mode >Rapid integration with existing enterprise workflows > Human-robot collaboration
1:12

Another American industrial humanoid robot🤖🏭 Noble Machines (formerly UCR) has just emerged from stealth mode. Launching the first humanoid robot. Founded only 18 months ago, it has already been deployed in a Fortune Global 500 factory. Teams from SpaceX, NASA, and Apple specialize in the toughest, most tiring, and most dangerous industrial tasks: >27kg heavy load >5-hour battery life >Walking speed 0.8m/s >Climbing stairs, traversing scaffolding, and navigating chaotic construction sites > Modular end effector, allowing for quick tool change. > AI-controlled operation with end-to-end autonomy; learns new skills in hours >Autonomous operation + Telep-op mode >Rapid integration with existing enterprise workflows > Human-robot collaboration

CyberRobo

44,656 Aufrufe • vor 3 Monaten