Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

OH! GYM! Students Bring Humanoid Motions from Simulation to Reality (Obstacle Running, Taekwondo, K-POP & Parkour)! 🎬 Video: 💬 Project:

passionvirus's profile picture

Yoonseok Pyo

1,419 subscribers

15,303 görüntüleme • 29 gün önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

3 Yorum

clankr profil fotoğrafı
clankr26 gün önce

Is this the open-source humanoid model? ROBOTIS was developing something like this.

Jongwhan Lee profil fotoğrafı
Jongwhan Lee27 gün önce

sim2real on humanoid looks clean parkour + taekwondo on real hardware is hard nice demo

Emma Sterling Ai profil fotoğrafı
Emma Sterling Ai29 gün önce

@passionvirus This is seriously impressive! 🤯 Taking humanoid motion from simulation to real-world obstacle running, Taekwondo, K-POP and parkour is a huge step for robotics! 🤖🔥

Benzer Videolar

Yuni is going to participate in a reality show called "Our Ballad", a joint project between SBS, SM, and SM C&C The survival program aims to discover new voices that can revive the emotion of classic ballads (similar to The Voice or K-pop Star) #smtrainee #SMNGG #sm연습생 #YUNI
0:53

Yuni is going to participate in a reality show called "Our Ballad", a joint project between SBS, SM, and SM C&C The survival program aims to discover new voices that can revive the emotion of classic ballads (similar to The Voice or K-pop Star) #smtrainee #SMNGG #sm연습생 #YUNI

TheSmnggTheory

109,296 görüntüleme • 11 ay önce

another artist commissioned for #KiiiKiii! 'Blue Hour' teaser video artwork by Pl1ght ( an independent artist from NYC. their first project with K-Pop group. it's amazing to notice how kiiiteam found niche artist to work with them! mad respect 🪼🤍
0:26

another artist commissioned for #KiiiKiii! 'Blue Hour' teaser video artwork by Pl1ght ( an independent artist from NYC. their first project with K-Pop group. it's amazing to notice how kiiiteam found niche artist to work with them! mad respect 🪼🤍

𝒦

37,942 görüntüleme • 2 ay önce

Gulmarg Gondola: Cable car project from Gulmarg (G-1) to Kongdori (G-2) to Afarwat (G-3) built in 2 phases with ₹25 crore, earned ₹108 crore in 2022 alone. Tickets at ₹1,700 per person are too expensive. Must be reduced to ₹800 for J&K residents. ₹400 for all students.
0:36

Gulmarg Gondola: Cable car project from Gulmarg (G-1) to Kongdori (G-2) to Afarwat (G-3) built in 2 phases with ₹25 crore, earned ₹108 crore in 2022 alone. Tickets at ₹1,700 per person are too expensive. Must be reduced to ₹800 for J&K residents. ₹400 for all students.

Ahmed Ali Fayyaz

22,050 görüntüleme • 3 yıl önce

Build a humanoid robot yourself! 🪚 OpenArm is an open-source humanoid robot. It comes with full CAD, control code, firmware, and simulation tools, everything needed to build, modify, and operate it. The arms are designed to be compliant and backdrivable. Teleoperation is supported, with force feedback and real-time gravity compensation so operators can guide the arm naturally. Super important, on the simulation side, OpenArm works with platforms like MuJoCo and Isaac Sim, letting developers test policies in virtual environments before running on hardware. Assemble it yourself from a kit or get it prebuilt, the goal is accessibility for research labs, small teams, and enthusiasts. The project is run by Enactic in Tokyo, Japan, and aims to lower the barrier for experimenting with dexterous manipulation. Let's put robotics in mainstream! 🔥🔥 P.S. I put the project page in the comments!
1:07

Build a humanoid robot yourself! 🪚 OpenArm is an open-source humanoid robot. It comes with full CAD, control code, firmware, and simulation tools, everything needed to build, modify, and operate it. The arms are designed to be compliant and backdrivable. Teleoperation is supported, with force feedback and real-time gravity compensation so operators can guide the arm naturally. Super important, on the simulation side, OpenArm works with platforms like MuJoCo and Isaac Sim, letting developers test policies in virtual environments before running on hardware. Assemble it yourself from a kit or get it prebuilt, the goal is accessibility for research labs, small teams, and enthusiasts. The project is run by Enactic in Tokyo, Japan, and aims to lower the barrier for experimenting with dexterous manipulation. Let's put robotics in mainstream! 🔥🔥 P.S. I put the project page in the comments!

Lukas Ziegler

233,153 görüntüleme • 11 ay önce

Not a preplanned motion sequence. A robot deciding mid-jump what to do next. [📍 paper + demo] Researchers just showed a humanoid doing real parkour using only onboard perception. No motion script, no fixed obstacle layout. The system is called Perceptive Humanoid Parkour (PHP). Instead of memorizing a path, the robot reads depth from its cameras and continuously chooses actions. Step, vault, climb, or roll depending on what geometry appears in front of it. To make that possible, they combine three ideas: First, they stitch together human motion clips into long movement references so the robot learns fluid transitions instead of isolated tricks. Second, they train tracking policies with reinforcement learning so contacts land at the right time and the robot keeps balance during dynamic moves. Finally, everything is distilled into one perception policy that runs directly from depth input to action selection. The result on a Unitree G1: about 3 m/s vaults wall climbs up to 1.25 m nearly one minute continuous obstacle traversal adapting when obstacles move What matters is not the tricks. It is the shift in capability. Earlier humanoids executed motions. This one navigates situations. Once robots react to geometry instead of replaying trajectories, environments stop needing to be predictable. Warehouses, homes, and outdoors suddenly become the same problem. Thanks for sharing, Zhen Wu! Paper + demo: ——— Weekly robotics and AI insights. Subscribe free:
0:29

Not a preplanned motion sequence. A robot deciding mid-jump what to do next. [📍 paper + demo] Researchers just showed a humanoid doing real parkour using only onboard perception. No motion script, no fixed obstacle layout. The system is called Perceptive Humanoid Parkour (PHP). Instead of memorizing a path, the robot reads depth from its cameras and continuously chooses actions. Step, vault, climb, or roll depending on what geometry appears in front of it. To make that possible, they combine three ideas: First, they stitch together human motion clips into long movement references so the robot learns fluid transitions instead of isolated tricks. Second, they train tracking policies with reinforcement learning so contacts land at the right time and the robot keeps balance during dynamic moves. Finally, everything is distilled into one perception policy that runs directly from depth input to action selection. The result on a Unitree G1: about 3 m/s vaults wall climbs up to 1.25 m nearly one minute continuous obstacle traversal adapting when obstacles move What matters is not the tricks. It is the shift in capability. Earlier humanoids executed motions. This one navigates situations. Once robots react to geometry instead of replaying trajectories, environments stop needing to be predictable. Warehouses, homes, and outdoors suddenly become the same problem. Thanks for sharing, Zhen Wu! Paper + demo: ——— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

22,127 görüntüleme • 7 ay önce

Everything you love about generative models — now powered by real physics! Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications. Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: The Genesis physics engine and simulation platform is fully open source at We'll gradually roll out access to our generative framework in the near future. Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism. We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications. Open Source Code: Project webpage: Documentation: 1/n
3:26

Everything you love about generative models — now powered by real physics! Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications. Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: The Genesis physics engine and simulation platform is fully open source at We'll gradually roll out access to our generative framework in the near future. Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism. We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications. Open Source Code: Project webpage: Documentation: 1/n

Zhou Xian

3,822,365 görüntüleme • 1 yıl önce

Build a humanoid robot yourself! 🪚 OpenArm is an open-source humanoid robot. It comes with full CAD, control code, firmware, and simulation tools, everything needed to build, modify, and operate it. The arms are designed to be compliant and backdrivable. Teleoperation is supported, with force feedback and real-time gravity compensation so operators can guide the arm naturally. Super important, on the simulation side, OpenArm works with platforms like MuJoCo and Isaac Sim, letting developers test policies in virtual environments before running on hardware. Assemble it yourself from a kit or get it prebuilt, the goal is accessibility for research labs, small teams, and enthusiasts. The project is run by Enactic in Tokyo, Japan, and aims to lower the barrier for experimenting with dexterous manipulation. Let's put robotics in mainstream! 🔥🔥 🔗 Find the link here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
1:07

Build a humanoid robot yourself! 🪚 OpenArm is an open-source humanoid robot. It comes with full CAD, control code, firmware, and simulation tools, everything needed to build, modify, and operate it. The arms are designed to be compliant and backdrivable. Teleoperation is supported, with force feedback and real-time gravity compensation so operators can guide the arm naturally. Super important, on the simulation side, OpenArm works with platforms like MuJoCo and Isaac Sim, letting developers test policies in virtual environments before running on hardware. Assemble it yourself from a kit or get it prebuilt, the goal is accessibility for research labs, small teams, and enthusiasts. The project is run by Enactic in Tokyo, Japan, and aims to lower the barrier for experimenting with dexterous manipulation. Let's put robotics in mainstream! 🔥🔥 🔗 Find the link here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

69,078 görüntüleme • 18 gün önce

🎤 BIGBANG — The Kings Who Shaped K-Pop! 👑 From G-Dragon’s trend-setting genius 🎨, Taeyang’s soulful vocals 🎶, Daesung’s bright energy ☀️, T.O.P’s deep charisma 🎬, to Seungri’s stage charisma and unforgettable performances 🔥 — BIGBANG’s legacy continues to echo through every generation of K-pop fans! 🌍✨ 💬 Which BIGBANG era still gives you chills — Fantastic Baby, Haru Haru, or Still Life? Drop your answer below 👇 and tag your VIP bestie who’s been with them since day one! 💛 🌐 📸 allkpop 📘 #BIGBANG #VIP #KpopLegends #Allkpop #KpopTrend #YGEntertainment #FantasticBaby #HaruHaru #StillLife #KpopKings #KpopFandom #BIGBANGForever #AllkpopUpdate #Kpop2025
0:35

🎤 BIGBANG — The Kings Who Shaped K-Pop! 👑 From G-Dragon’s trend-setting genius 🎨, Taeyang’s soulful vocals 🎶, Daesung’s bright energy ☀️, T.O.P’s deep charisma 🎬, to Seungri’s stage charisma and unforgettable performances 🔥 — BIGBANG’s legacy continues to echo through every generation of K-pop fans! 🌍✨ 💬 Which BIGBANG era still gives you chills — Fantastic Baby, Haru Haru, or Still Life? Drop your answer below 👇 and tag your VIP bestie who’s been with them since day one! 💛 🌐 📸 allkpop 📘 #BIGBANG #VIP #KpopLegends #Allkpop #KpopTrend #YGEntertainment #FantasticBaby #HaruHaru #StillLife #KpopKings #KpopFandom #BIGBANGForever #AllkpopUpdate #Kpop2025

allkpop

177,763 görüntüleme • 10 ay önce

THIS IS SERIOUSLY😭😭 his fave youtuber ej hyung and HIS TAEKWONDO TEACHER WISHING JUNGWON HAPPY BIRHTDAY I TEARED UP WHEN TEACHER STARTED TO TALK😭😭 👤: It’s your Master here! I never imagined I’d be sending you a birthday video like this. You’ve been learning Taekwondo from me since you were little and seeing you grow up so well and active on TV and the internet makes me so happy and proud. You’ve already achieved so much but I hope you continue to be loved and recognized by even more fans. As someone who trained in Taekwondo, I hope you also become a warm-hearted artist who looks after those around you who might be in need. Always take care of your health, I wish you great success, and keep fighting! Yang Jungwon, Fighting! Happy Birthday! Oh, and I’m running the gym in Sadang-dong again, so come by and see me when you have time!"
1:19

THIS IS SERIOUSLY😭😭 his fave youtuber ej hyung and HIS TAEKWONDO TEACHER WISHING JUNGWON HAPPY BIRHTDAY I TEARED UP WHEN TEACHER STARTED TO TALK😭😭 👤: It’s your Master here! I never imagined I’d be sending you a birthday video like this. You’ve been learning Taekwondo from me since you were little and seeing you grow up so well and active on TV and the internet makes me so happy and proud. You’ve already achieved so much but I hope you continue to be loved and recognized by even more fans. As someone who trained in Taekwondo, I hope you also become a warm-hearted artist who looks after those around you who might be in need. Always take care of your health, I wish you great success, and keep fighting! Yang Jungwon, Fighting! Happy Birthday! Oh, and I’m running the gym in Sadang-dong again, so come by and see me when you have time!"

🦊♡🐰ྀི | fan acc

64,218 görüntüleme • 7 ay önce

We are excited to share our #CORL2024 paper on learning quadrotor obstacle avoidance from the visual stream of a single #eventcamera! Trained entirely in simulation! We demonstrate obstacle avoidance both in the dark and in a forest up to 5m/s. PDF: Video: Project page: Event cameras are sensors that output per-pixel-level intensity changes at microsecond latency resolution; they feature nearly zero motion blur and high dynamic range but produce a very large volume of events under significant ego-motion and further lack a high-fidelity continuous-time sensor model in simulation, making direct #sim2real transfer not possible. By leveraging depth prediction as a pretext task, we pre-train a reactive obstacle avoidance policy with “approximated” simulated events and then fine-tune the perception component with limited events-and-depth real-world data. This technique bridges the sim2real gap for #eventcameras! As at the current state, there is no continuous-time sensor model for event cameras, we hope that this work can finally spur future research leveraging simulation for training event-vision-based policies to create faster, agile robots! Kudos to Anish Bhattacharya, @marcocannic, Vijay Kumar Nikolai Matni UZH Science University of Zurich UZH Space Hub UZH IfI European Research Council (ERC) GRASP Laboratory Penn Engineering
1:25

We are excited to share our #CORL2024 paper on learning quadrotor obstacle avoidance from the visual stream of a single #eventcamera! Trained entirely in simulation! We demonstrate obstacle avoidance both in the dark and in a forest up to 5m/s. PDF: Video: Project page: Event cameras are sensors that output per-pixel-level intensity changes at microsecond latency resolution; they feature nearly zero motion blur and high dynamic range but produce a very large volume of events under significant ego-motion and further lack a high-fidelity continuous-time sensor model in simulation, making direct #sim2real transfer not possible. By leveraging depth prediction as a pretext task, we pre-train a reactive obstacle avoidance policy with “approximated” simulated events and then fine-tune the perception component with limited events-and-depth real-world data. This technique bridges the sim2real gap for #eventcameras! As at the current state, there is no continuous-time sensor model for event cameras, we hope that this work can finally spur future research leveraging simulation for training event-vision-based policies to create faster, agile robots! Kudos to Anish Bhattacharya, @marcocannic, Vijay Kumar Nikolai Matni UZH Science University of Zurich UZH Space Hub UZH IfI European Research Council (ERC) GRASP Laboratory Penn Engineering

Davide Scaramuzza

17,219 görüntüleme • 1 yıl önce

Humanoids finally move like humans… and can do more than copy. [Details + demos in thread 👇] A new framework, BeyondMimic, shows how to learn naturalistic whole-body control from human motion. But then goes further by composing those skills into versatile, zero-shot policies. Two key pieces make it work: •Motion tracking pipeline: robustly turns human mocap into dynamic robot motions (jump spins, sprinting, cartwheels) on real hardware •Guided diffusion policy: distills motion primitives and enables zero-shot downstream tasks with simple cost functions The result: ✅ State-of-the-art motion quality from mocap tracking ✅ Versatile task execution; waypoint navigation, joystick teleop, obstacle avoidance ✅ Stable sim-to-real transfer for diverse humanoid skills Thank you for sharing, Qiayuan Liao! 📍 Paper: Video:
0:42

Humanoids finally move like humans… and can do more than copy. [Details + demos in thread 👇] A new framework, BeyondMimic, shows how to learn naturalistic whole-body control from human motion. But then goes further by composing those skills into versatile, zero-shot policies. Two key pieces make it work: •Motion tracking pipeline: robustly turns human mocap into dynamic robot motions (jump spins, sprinting, cartwheels) on real hardware •Guided diffusion policy: distills motion primitives and enables zero-shot downstream tasks with simple cost functions The result: ✅ State-of-the-art motion quality from mocap tracking ✅ Versatile task execution; waypoint navigation, joystick teleop, obstacle avoidance ✅ Stable sim-to-real transfer for diverse humanoid skills Thank you for sharing, Qiayuan Liao! 📍 Paper: Video:

Ilir Aliu - eu/acc

72,522 görüntüleme • 1 yıl önce

"For those who come after" (TEASER) We traveled to the coast of Étretat to bring the characters of Clair Obscur: Expedition 33 into our reality. This project is a tribute to the artistry of the game and a labor of love from everyone involved. Clair Obscur: Expedition 33 ❤️🇫🇷🤍 Sandfall Interactive - CO:E33 out now! Full cinematic experience is coming soon. 🕯️ Maelle: fenixfatalist Verso: Me :3 Video: MingVideography (IG) Legendary 🎵 by Lorien Testard #ClairObscur #France #Expedition33
0:33

"For those who come after" (TEASER) We traveled to the coast of Étretat to bring the characters of Clair Obscur: Expedition 33 into our reality. This project is a tribute to the artistry of the game and a labor of love from everyone involved. Clair Obscur: Expedition 33 ❤️🇫🇷🤍 Sandfall Interactive - CO:E33 out now! Full cinematic experience is coming soon. 🕯️ Maelle: fenixfatalist Verso: Me :3 Video: MingVideography (IG) Legendary 🎵 by Lorien Testard #ClairObscur #France #Expedition33

T A R Y N

36,962 görüntüleme • 6 ay önce

This is wild: watch this guy go around in the Matrix Awakens (!) game explaining to AI NPCs they’re in a simulation. Remember, these are not scripted. They're not human voices. And this is not "some day", this is in Unity/Unreal now. Video games are about to be filled with AI NPCs indistinguishable from humans. Also trippy: watch AI NPCs go around trying to convince other NPCs they're in a simulation. Soon, there will be levels upon levels of simulated AI NPC worlds - and most of these AI NPCs ("fake people") will probably think they're in base reality, like most humans do. From Replica Studios: "NPCs controlled by AI will dynamically respond to the player's voice in real time. Characters will change their dialogue, emotional tone and body gestures in response to how the player addresses them." “You're not really real.” “That's… hard to process. Does this mean I don't exist?” “You exist. Sort of.” “What does that mean?? Am I real or not?” “It means that after I close this game, you don't exist.” “Oh… so I'm only here for as long as the game is running. That's… difficult to comprehend.”
1:59

This is wild: watch this guy go around in the Matrix Awakens (!) game explaining to AI NPCs they’re in a simulation. Remember, these are not scripted. They're not human voices. And this is not "some day", this is in Unity/Unreal now. Video games are about to be filled with AI NPCs indistinguishable from humans. Also trippy: watch AI NPCs go around trying to convince other NPCs they're in a simulation. Soon, there will be levels upon levels of simulated AI NPC worlds - and most of these AI NPCs ("fake people") will probably think they're in base reality, like most humans do. From Replica Studios: "NPCs controlled by AI will dynamically respond to the player's voice in real time. Characters will change their dialogue, emotional tone and body gestures in response to how the player addresses them." “You're not really real.” “That's… hard to process. Does this mean I don't exist?” “You exist. Sort of.” “What does that mean?? Am I real or not?” “It means that after I close this game, you don't exist.” “Oh… so I'm only here for as long as the game is running. That's… difficult to comprehend.”

AI Notkilleveryoneism Memes ⏸️

2,150,736 görüntüleme • 3 yıl önce

Louis Vuitton opens ‘Hotel Bangkok’ pop-up, only stop in Southeast Asia . Louis Vuitton has unveiled its Louis Vuitton Hotel Bangkok pop-up at Baan Trok Tua Ngok on Santiphap Road, Pom Prap Sattru Phai district, running from February 11 to March 15. . Bangkok is the only city in Southeast Asia selected for the project, following earlier stops in Shanghai, and ahead of New York and Seoul. . Across four floors, the pop-up spotlights five iconic Monogram bag shapes—Keepall, Speedy, Alma, Neverfull and Noé—tracing the maison’s travel heritage from past to present. . Highlights include a flag-lined façade (Floor 1), a Keepall story display with personalisation services and a Speedy P9 safe room (Floor 2), the Neverfull Gym and Noé Bar (Floor 3), plus the Speedy Room 1930 and an Alma-inspired terrace with a Paris feel (Floor 4). . Booking is available via the official site: . #LouisVuitton #Bangkok #NationVibes #LVMonogram130 #BamBam #뱀뱀 BamBam
0:59

Louis Vuitton opens ‘Hotel Bangkok’ pop-up, only stop in Southeast Asia . Louis Vuitton has unveiled its Louis Vuitton Hotel Bangkok pop-up at Baan Trok Tua Ngok on Santiphap Road, Pom Prap Sattru Phai district, running from February 11 to March 15. . Bangkok is the only city in Southeast Asia selected for the project, following earlier stops in Shanghai, and ahead of New York and Seoul. . Across four floors, the pop-up spotlights five iconic Monogram bag shapes—Keepall, Speedy, Alma, Neverfull and Noé—tracing the maison’s travel heritage from past to present. . Highlights include a flag-lined façade (Floor 1), a Keepall story display with personalisation services and a Speedy P9 safe room (Floor 2), the Neverfull Gym and Noé Bar (Floor 3), plus the Speedy Room 1930 and an Alma-inspired terrace with a Paris feel (Floor 4). . Booking is available via the official site: . #LouisVuitton #Bangkok #NationVibes #LVMonogram130 #BamBam #뱀뱀 BamBam

Thenationthailand

42,389 görüntüleme • 7 ay önce

Let's reverse engineer this demo. You need 3 things: (1) robust hardware and motor designs that treat simulation as first-class citizen; (2) a human motion capture ("mocap") dataset, such as those for film and gaming characters; (3) massively parallel RL training in GPU-accelerated simulation. Last October, our team trained a 1.5M parameter foundation model called HOVER for such agile motor control. It follows this recipe, roughly speaking (details in thread): (1) Simulation used to be an after-thought. Now, it has to be part of the hardware design process. If your robot doesn't simulate well, you can kiss RL goodbye. Hardware-simulation co-design is a very interesting emergent topic that only becomes meaningful with today's compute capability. (2) Human mocap dataset to produce natural-looking walking and running gaits. That's one huge advantage of using humanoid robot - you get to imitate from tons of human motions that were originally captured for movies or AAA games. At least 3 ways to use the data: - For initialization: pre-train the neural net to imitate human, and then finetune it into the robot form factor with physics turned on; - For reward function: penalize any deviations from the target pose; - For representation learning: treat the human poses as a "motion prior" to constrain the space of robot behaviors. (3) Shove the above into Isaac sim, add a lot of randomization, pump it through PPO, throw in a bunch of GPUs, and then watch Netflix till loss converges. If you have an urge to comment this is CGI, let me save you a few keystrokes — many academic labs now own the G1 robot in the flesh. See our team's HOVER work in the thread: 🧵
1:04

Let's reverse engineer this demo. You need 3 things: (1) robust hardware and motor designs that treat simulation as first-class citizen; (2) a human motion capture ("mocap") dataset, such as those for film and gaming characters; (3) massively parallel RL training in GPU-accelerated simulation. Last October, our team trained a 1.5M parameter foundation model called HOVER for such agile motor control. It follows this recipe, roughly speaking (details in thread): (1) Simulation used to be an after-thought. Now, it has to be part of the hardware design process. If your robot doesn't simulate well, you can kiss RL goodbye. Hardware-simulation co-design is a very interesting emergent topic that only becomes meaningful with today's compute capability. (2) Human mocap dataset to produce natural-looking walking and running gaits. That's one huge advantage of using humanoid robot - you get to imitate from tons of human motions that were originally captured for movies or AAA games. At least 3 ways to use the data: - For initialization: pre-train the neural net to imitate human, and then finetune it into the robot form factor with physics turned on; - For reward function: penalize any deviations from the target pose; - For representation learning: treat the human poses as a "motion prior" to constrain the space of robot behaviors. (3) Shove the above into Isaac sim, add a lot of randomization, pump it through PPO, throw in a bunch of GPUs, and then watch Netflix till loss converges. If you have an urge to comment this is CGI, let me save you a few keystrokes — many academic labs now own the G1 robot in the flesh. See our team's HOVER work in the thread: 🧵

Jim Fan

216,139 görüntüleme • 1 yıl önce

Exciting updates on Project GR00T! We discover a systematic way to scale up robot data, tackling the most painful pain point in robotics. The idea is simple: human collects demonstration on a real robot, and we multiply that data 1000x or more in simulation. Let’s break it down: 1. We use Apple Vision Pro (yes!!) to give the human operator first person control of the humanoid. Vision Pro parses human hand pose and retargets the motion to the robot hand, all in real time. From the human’s point of view, they are immersed in another body like the Avatar. Teleoperation is slow and time-consuming, but we can afford to collect a small amount of data. 2. We use RoboCasa, a generative simulation framework, to multiply the demonstration data by varying the visual appearance and layout of the environment. In Jensen’s keynote video below, the humanoid is now placing the cup in hundreds of kitchens with a huge diversity of textures, furniture, and object placement. We only have 1 physical kitchen at the GEAR Lab in NVIDIA HQ, but we can conjure up infinite ones in simulation. 3. Finally, we apply MimicGen, a technique to multiply the above data even more by varying the *motion* of the robot. MimicGen generates vast number of new action trajectories based on the original human data, and filters out failed ones (e.g. those that drop the cup) to form a much larger dataset. To sum up, given 1 human trajectory with Vision Pro -> RoboCasa produces N (varying visuals) -> MimicGen further augments to NxM (varying motions). This is the way to trade compute for expensive human data by GPU-accelerated simulation. A while ago, I mentioned that teleoperation is fundamentally not scalable, because we are always limited by 24 hrs/robot/day in the world of atoms. Our new GR00T synthetic data pipeline breaks this barrier in the world of bits. Scaling has been so much fun for LLMs, and it's finally our turn to have fun in robotics! We are building tools to enable everyone in the ecosystem to scale up with us. Links in thread:
1:21

Exciting updates on Project GR00T! We discover a systematic way to scale up robot data, tackling the most painful pain point in robotics. The idea is simple: human collects demonstration on a real robot, and we multiply that data 1000x or more in simulation. Let’s break it down: 1. We use Apple Vision Pro (yes!!) to give the human operator first person control of the humanoid. Vision Pro parses human hand pose and retargets the motion to the robot hand, all in real time. From the human’s point of view, they are immersed in another body like the Avatar. Teleoperation is slow and time-consuming, but we can afford to collect a small amount of data. 2. We use RoboCasa, a generative simulation framework, to multiply the demonstration data by varying the visual appearance and layout of the environment. In Jensen’s keynote video below, the humanoid is now placing the cup in hundreds of kitchens with a huge diversity of textures, furniture, and object placement. We only have 1 physical kitchen at the GEAR Lab in NVIDIA HQ, but we can conjure up infinite ones in simulation. 3. Finally, we apply MimicGen, a technique to multiply the above data even more by varying the *motion* of the robot. MimicGen generates vast number of new action trajectories based on the original human data, and filters out failed ones (e.g. those that drop the cup) to form a much larger dataset. To sum up, given 1 human trajectory with Vision Pro -> RoboCasa produces N (varying visuals) -> MimicGen further augments to NxM (varying motions). This is the way to trade compute for expensive human data by GPU-accelerated simulation. A while ago, I mentioned that teleoperation is fundamentally not scalable, because we are always limited by 24 hrs/robot/day in the world of atoms. Our new GR00T synthetic data pipeline breaks this barrier in the world of bits. Scaling has been so much fun for LLMs, and it's finally our turn to have fun in robotics! We are building tools to enable everyone in the ecosystem to scale up with us. Links in thread:

Jim Fan

364,670 görüntüleme • 2 yıl önce

🚱 “Uncle Vova, please give us water” — children from occupied Donbas beg Putin for the most basic necessity Children from the Russian-occupied Donbas recorded a video message to Vladimir Putin, asking for running water in their homes. 💬 “Uncle Vova, we want to run and laugh, not wait for trucks to bring us water… Please give us a miracle — water from the tap,” pleads a girl from Makiivka, where there has been no water for a month. The video was published by the organization Obereg Life, which claims it has officially sent the message to Putin. In Donetsk, stable water supply has been gone since 2022. Until recently, water was provided every other day — now local authorities are planning to reduce it to once every three days. Even when water does arrive, it’s muddy, full of sediment, and doesn’t reach the upper floors. Just a reminder: before the “Russian world” came, Donetsk hosted the UEFA Euro 2012 and even a Rihanna concert.
2:25

🚱 “Uncle Vova, please give us water” — children from occupied Donbas beg Putin for the most basic necessity Children from the Russian-occupied Donbas recorded a video message to Vladimir Putin, asking for running water in their homes. 💬 “Uncle Vova, we want to run and laugh, not wait for trucks to bring us water… Please give us a miracle — water from the tap,” pleads a girl from Makiivka, where there has been no water for a month. The video was published by the organization Obereg Life, which claims it has officially sent the message to Putin. In Donetsk, stable water supply has been gone since 2022. Until recently, water was provided every other day — now local authorities are planning to reduce it to once every three days. Even when water does arrive, it’s muddy, full of sediment, and doesn’t reach the upper floors. Just a reminder: before the “Russian world” came, Donetsk hosted the UEFA Euro 2012 and even a Rihanna concert.

NEXTA

53,257 görüntüleme • 1 yıl önce

🚨Shocking but true: If you can't handle a marriage, don't get into it The viral video of a husband from Moradabad exposes a dark reality. Reportedly, when he went to bring his wife back, he caught her with another man in a hotel. Instead of legal exit routes, modern relationships are plagued by toxic manipulation, extreme conspiracies like drum plotting , and desertion. Running away without a legal divorce is a blatant violation of the Hindu Marriage Act. If you don't love them, say NO beforehand. Don't ruin lives later. Note: Content based on viral unverified allegations.
0:35

🚨Shocking but true: If you can't handle a marriage, don't get into it The viral video of a husband from Moradabad exposes a dark reality. Reportedly, when he went to bring his wife back, he caught her with another man in a hotel. Instead of legal exit routes, modern relationships are plagued by toxic manipulation, extreme conspiracies like drum plotting , and desertion. Running away without a legal divorce is a blatant violation of the Hindu Marriage Act. If you don't love them, say NO beforehand. Don't ruin lives later. Note: Content based on viral unverified allegations.

Ramesh Speaks

1,563,663 görüntüleme • 3 ay önce

High school requires girls to disclose their menstrual cycle—or lose a grade. Parents must send a note in writing to gym teacher with "suspected length of their cycle." If female student does not participate—it will result in "loss of points toward final grade." Students who bring in the required note from their parents will then be required to complete "dry land" exercises—to make up for the missed swimming. "For sure against privacy for the young ladies that are going to class," said a parent. "And they shouldn't have a grade reduction." Lakeview High School says they plan to "engage parents in a conversation" about the new policy. The school is located in St. Clair Shores, Michigan. Posting first full length and credited video here to help raise awareness.
1:47

High school requires girls to disclose their menstrual cycle—or lose a grade. Parents must send a note in writing to gym teacher with "suspected length of their cycle." If female student does not participate—it will result in "loss of points toward final grade." Students who bring in the required note from their parents will then be required to complete "dry land" exercises—to make up for the missed swimming. "For sure against privacy for the young ladies that are going to class," said a parent. "And they shouldn't have a grade reduction." Lakeview High School says they plan to "engage parents in a conversation" about the new policy. The school is located in St. Clair Shores, Michigan. Posting first full length and credited video here to help raise awareness.

LongTime🤓FirstTime👨‍💻

701,446 görüntüleme • 13 gün önce