正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Picking up laundry with a 3D printed robot arm and LeRobot. It operates using SO-ARM101 arm mounted on a mobile base, relying on just a wrist and base camera. The base is controlled with an Xbox controller by a human operator. The arm is fully autonomous thanks to the... ACT model. The robot can generalize to previously unseen environments. The technical stack: - Model: Action Chunking Transformer (ACT) with Temporal Ensemble - Data: Learned via teleoperation including negative examples and failure recoveries - Framework: Built and trained locally using LeRobot on a single consumer GPU in 3.5h This is a perfect example that real-world robotics isn't about perfect lab conditions - it's about messy living rooms, confusing shadows, and learning to recover from mistakes. And with open tools, anyone can tackle these edge cases right from their desk.show more

LeRobot

20,155 subscribers

69,148 次观看 • 1 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

15 条评论

LeRobot 的头像

LeRobot1 个月前

Full video:

Shinka - AI 的头像

Shinka - AI1 个月前

A 3D printed arm trained on a single GPU doing laundry in a messy living room tells you more about real robotics progress than any million-dollar lab demo.

Steven Cheng 的头像

Steven Cheng1 个月前

Negative examples are key. They teach the model how to recover, which is crucial for messy real-world environments.

SEAR 的头像

SEAR1 个月前

Messy living room data is gold

tongtongtongtongtong 的头像

tongtongtongtongtong1 个月前

Neat!

primesync 的头像

primesync1 个月前

laundry is a brutal benchmark because cloth has no fixed shape, so nothing generalizes from rigid-object demos. doing it with just wrist and base cams on a 3D printed arm is the part that should get more attention.

Saquib Mehmood 的头像

Saquib Mehmood1 个月前

Cool.

Mario Bollini 的头像

Mario Bollini1 个月前

Do toddler toys next!

Ajit Sadalagi 的头像

Ajit Sadalagi1 个月前

Can this run end to end neural network locally ?

Raul Verdusco 的头像

Raul Verdusco1 个月前

Open tools are making real-world robotics more accessible.Generalizing to messy, unseen environments with consumer hardware is exactly the kind of progress that matters for practical home robotics. 🤖

Krafters Robotics 的头像

Krafters Robotics1 个月前

The human-driven base + autonomous arm split is a sensible way to isolate navigation from manipulation. The number I'd love to see next is interventions per garment, especially after a missed grasp or occlusion.

Daniel Lambert 的头像

Daniel Lambert1 个月前

The impressive part isn’t picking up laundry. It’s generalizing to new environments and recovering from failures. That’s a huge step toward practical home robots. 🧺🤖

Robo Reliance 的头像

Robo Reliance1 个月前

Messy living rooms and confusing shadows are the honest benchmark. We work the field side of robotics and the failure modes are never the ones the lab predicted. Recovery behavior is the real milestone here.

Lee Penkman 的头像

Lee Penkman1 个月前

Epic

Aeris Vahn Ephelia - 𝑹𝑬𝑽 的头像

Aeris Vahn Ephelia - 𝑹𝑬𝑽1 个月前

Congratulation this is nice work

相关视频

The $600 Open-Source Home Robot🤖 Meet AlohaMini, the new dual-arm mobile robot designed to make real-world manipulation and embodied AI research accessible. This bot is fully 3D-printable and aimed at home builders and research labs. Key Features: 💰 Cost: The Bill of Materials (BOM) totals around $600 (USD) for self-printed parts, making it highly accessible. 🦾 Hardware: Dual-arm mobile base with a motorized vertical lift (0–60 cm travel). 💻 Software: Completely open-source hardware and software, compatible out-of-the-box with the LeRobot framework. The platform supports a fully 3D-printed Mini and a stiffer Pro model (hybrid materials). It assembles in approximately 60 minutes. 🔗 GitHub Open-Source Code & Files:

The $600 Open-Source Home Robot🤖 Meet AlohaMini, the new dual-arm mobile robot designed to make real-world manipulation and embodied AI research accessible. This bot is fully 3D-printable and aimed at home builders and research labs. Key Features: 💰 Cost: The Bill of Materials (BOM) totals around $600 (USD) for self-printed parts, making it highly accessible. 🦾 Hardware: Dual-arm mobile base with a motorized vertical lift (0–60 cm travel). 💻 Software: Completely open-source hardware and software, compatible out-of-the-box with the LeRobot framework. The platform supports a fully 3D-printed Mini and a stiffer Pro model (hybrid materials). It assembles in approximately 60 minutes. 🔗 GitHub Open-Source Code & Files:

RoboHub🤖

41,670 次观看 • 10 个月前

Open-source robot arm meets hand tracking [📍GitHub below] It is designed with an industrial mindset but built as a 3D-printed desktop system. PAROL6 paired with a LEAP Motion controller is a nice example of how accessible robot teleoperation has become. • Hand motion is streamed to the robot at 100 Hz via UDP • A pneumatic gripper is controlled by simple fist open and close gestures • The entire robot stack is open source, from mechanics to control software Combine that with low-latency hand tracking and you get a very practical platform for learning manipulation, teleoperation, and human-robot interfaces. This kind of setup is great for experimentation, teleop, data collection, and teaching robots by demonstration All without proprietary hardware or locked software. Credit to SourceRobotics 📍Code: —— Weekly robotics and AI insights. Subscribe free:

Open-source robot arm meets hand tracking [📍GitHub below] It is designed with an industrial mindset but built as a 3D-printed desktop system. PAROL6 paired with a LEAP Motion controller is a nice example of how accessible robot teleoperation has become. • Hand motion is streamed to the robot at 100 Hz via UDP • A pneumatic gripper is controlled by simple fist open and close gestures • The entire robot stack is open source, from mechanics to control software Combine that with low-latency hand tracking and you get a very practical platform for learning manipulation, teleoperation, and human-robot interfaces. This kind of setup is great for experimentation, teleop, data collection, and teaching robots by demonstration All without proprietary hardware or locked software. Credit to SourceRobotics 📍Code: —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

32,823 次观看 • 12 天前

introducing lattice a low-cost robot arm with a modular, detachable gripper so the robot can swap its own tools the adapter is $14 if you already have an so101 arm, and about $140 to build the full system from scratch it’s now fully open source

introducing lattice a low-cost robot arm with a modular, detachable gripper so the robot can swap its own tools the adapter is $14 if you already have an so101 arm, and about $140 to build the full system from scratch it’s now fully open source

ahad

40,451 次观看 • 3 个月前

RMB 9,800 (about $1,460) to get started. BLUE WORM just unveiled Mantis Standard. Detach an arm for your desk, build a dual-arm workstation, or add the mobile base and lift for a wheeled humanoid. The arms, base, lift and head use mechanical and electrical quick-connects. Change the configuration, keep the same robot brain: BW-Brain. The full 1.4 m build has 22 degrees of freedom, a rated 7 kg payload per arm and 550 mm of vertical lift. It supports ROS 2, Isaac Sim and LeRobot, with compatibility for π0.5, ACT and SmolVLA. You can simulate a task, collect VR demonstrations, fine-tune a policy and deploy it on the hardware. For a small lab, reusing those components and development tools could mean fewer separate rigs to buy and less integration work.

RMB 9,800 (about $1,460) to get started. BLUE WORM just unveiled Mantis Standard. Detach an arm for your desk, build a dual-arm workstation, or add the mobile base and lift for a wheeled humanoid. The arms, base, lift and head use mechanical and electrical quick-connects. Change the configuration, keep the same robot brain: BW-Brain. The full 1.4 m build has 22 degrees of freedom, a rated 7 kg payload per arm and 550 mm of vertical lift. It supports ROS 2, Isaac Sim and LeRobot, with compatibility for π0.5, ACT and SmolVLA. You can simulate a task, collect VR demonstrations, fine-tune a policy and deploy it on the hardware. For a small lab, reusing those components and development tools could mean fewer separate rigs to buy and less integration work.

RoboHub🤖

14,175 次观看 • 19 天前

🔥 Community spotlight! Haixuan Tao built XoQ — a robot virtualization library that lets you connect to OpenArm from any OS, including macOS and even the web browser! Works with LeRobot for teleoperation. This is what open source robotics is all about 🤖

🔥 Community spotlight! Haixuan Tao built XoQ — a robot virtualization library that lets you connect to OpenArm from any OS, including macOS and even the web browser! Works with LeRobot for teleoperation. This is what open source robotics is all about 🤖

Enactic

21,271 次观看 • 7 个月前

Below is our open-source arm assembled by one of the discord members! Also, today Physical Intelligence shared their incredible top-notch robotics model Pi0. And I just want to remind you that you can use our arm together with this model and with LeRobot The arm has the same DOF as the arm used by Pi0, but has smaller size and cost starts from 200$ per arm. You can use better motors like we did in our demos, then it will be more precise and stable but the price will be about $600 per arm. Printable files, purchase list, and help with the assembly - in our discord channel.

Below is our open-source arm assembled by one of the discord members! Also, today Physical Intelligence shared their incredible top-notch robotics model Pi0. And I just want to remind you that you can use our arm together with this model and with LeRobot The arm has the same DOF as the arm used by Pi0, but has smaller size and cost starts from 200$ per arm. You can use better motors like we did in our demos, then it will be more precise and stable but the price will be about $600 per arm. Printable files, purchase list, and help with the assembly - in our discord channel.

Igor Kulakov

53,711 次观看 • 1 年前

VLAI Robotics Launches High-Dexterity Dual-Arm Robot Starting at ~$5,500 USD VLAI Robotics, utilizing the open-source design of Japan’s OpenArm team, has introduced a high-dexterity, low-cost bimanual robotic system. The product is priced from 39,900 RMB (approximately 5,500 USD), making high-level research tools accessible to institutions and developers. The robot features a human-scale, arm-hand integrated design with up to 16 DOF (8 DOF per arm, including the gripper). It can handle a dual-arm peak payload of 12 kg while maintaining high precision. A key breakthrough is its ability to replicate human upper limb movement trajectories, which enhances data quality for imitation learning and remote operation. VLAI Robotics handled the domestic engineering, manufacturing, and VLA (Vision-Language-Action) algorithm integration, with quality control organized according to the strict standards of the OpenArm R&D framework. The robot is highly adaptable for research, education, and Physical AI training, supporting easy expansion for features like VR teleoperation.

VLAI Robotics Launches High-Dexterity Dual-Arm Robot Starting at ~$5,500 USD VLAI Robotics, utilizing the open-source design of Japan’s OpenArm team, has introduced a high-dexterity, low-cost bimanual robotic system. The product is priced from 39,900 RMB (approximately 5,500 USD), making high-level research tools accessible to institutions and developers. The robot features a human-scale, arm-hand integrated design with up to 16 DOF (8 DOF per arm, including the gripper). It can handle a dual-arm peak payload of 12 kg while maintaining high precision. A key breakthrough is its ability to replicate human upper limb movement trajectories, which enhances data quality for imitation learning and remote operation. VLAI Robotics handled the domestic engineering, manufacturing, and VLA (Vision-Language-Action) algorithm integration, with quality control organized according to the strict standards of the OpenArm R&D framework. The robot is highly adaptable for research, education, and Physical AI training, supporting easy expansion for features like VR teleoperation.

RoboHub🤖

12,961 次观看 • 11 个月前

MolmoAct2 runs zero-shot on an SO-ARM101, no training required 🦾 Ai2 open Action Reasoning Model is now in LeRobot with the full lifecycle covered: fine-tuning (full or LoRA), evaluation, and real-robot deployment. It pairs a Molmo2-ER vision-language backbone with a flow-matching action expert to turn images, language, and proprioceptive state into action chunks. The best part: ready-made checkpoints ship with calibration correction baked in, so you can point it at your SO-100/101 and just watch it work. Inference fits in ~12GB at bf16, and LoRA fine-tuning runs on a single 24GB GPU. Big thanks to the Ai2 team for building this in the open.

MolmoAct2 runs zero-shot on an SO-ARM101, no training required 🦾 Ai2 open Action Reasoning Model is now in LeRobot with the full lifecycle covered: fine-tuning (full or LoRA), evaluation, and real-robot deployment. It pairs a Molmo2-ER vision-language backbone with a flow-matching action expert to turn images, language, and proprioceptive state into action chunks. The best part: ready-made checkpoints ship with calibration correction baked in, so you can point it at your SO-100/101 and just watch it work. Inference fits in ~12GB at bf16, and LoRA fine-tuning runs on a single 24GB GPU. Big thanks to the Ai2 team for building this in the open.

LeRobot

23,339 次观看 • 2 个月前

What happens when robot world models learn from human experience at scale? 🤔 DreamDojo from NVIDIA Research is a generalist robot world model pretrained on 44K hours of egocentric human videos and then post-trained on robot data to generalize across new objects and environments. After distillation, it runs at 10 FPS for live teleoperation, policy evaluation, and model-based planning. Read the ICML paper to learn more 📄

What happens when robot world models learn from human experience at scale? 🤔 DreamDojo from NVIDIA Research is a generalist robot world model pretrained on 44K hours of egocentric human videos and then post-trained on robot data to generalize across new objects and environments. After distillation, it runs at 10 FPS for live teleoperation, policy evaluation, and model-based planning. Read the ICML paper to learn more 📄

NVIDIA Robotics

23,293 次观看 • 2 个月前

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

The Humanoid Hub

20,968 次观看 • 1 年前

Today we're announcing microSLAM, a monocular SLAM system built by our computer vision team in Zürich with ETH Zurich's Computer Vision and Geometry Lab. It ranks first on the LaMaria benchmark for monocular SLAM, and holds up against systems that carry many more cameras and IMUs. Ours runs on a single RGB stream. Robot learning is bottlenecked on data, and the largest untapped source is people going about their work. A camera on someone's head for an afternoon in a plant is a record of how that plant actually operates, including all the parts nobody writes down. That data is only useful if you can recover the geometry, and geometry from one moving camera in a world that will not sit still is the hard version of the problem. Solving it monocular is what makes the data cheap. Rigs do not scale to every worker. Glasses do. microSLAM turns that footage into interactive 3D environments where behavioral foundation models can be trained. One person walks a factory floor. A robot learns to navigate it.

Today we're announcing microSLAM, a monocular SLAM system built by our computer vision team in Zürich with ETH Zurich's Computer Vision and Geometry Lab. It ranks first on the LaMaria benchmark for monocular SLAM, and holds up against systems that carry many more cameras and IMUs. Ours runs on a single RGB stream. Robot learning is bottlenecked on data, and the largest untapped source is people going about their work. A camera on someone's head for an afternoon in a plant is a record of how that plant actually operates, including all the parts nobody writes down. That data is only useful if you can recover the geometry, and geometry from one moving camera in a world that will not sit still is the hard version of the problem. Solving it monocular is what makes the data cheap. Rigs do not scale to every worker. Glasses do. microSLAM turns that footage into interactive 3D environments where behavioral foundation models can be trained. One person walks a factory floor. A robot learns to navigate it.

Gianni Hodel

68,719 次观看 • 22 天前

OpenAI's thijs predicts robotics will have its own version of the 3D printing craze, with people buying cheap, open-source robot arms and connecting them to agents. "I can see a world quite soon where, similar to when everyone went and bought 3D printers and ran software, everyone's going to buy plastic, cheaper robot arms that you can clip onto a table, put stuff in front of, and plug them into agents." "The price of classic robotics equipment is thousands of dollars at the moment. This arm I'm using here is a Hugging Face SO-100 robot, which is open source, fully 3D printable, and around $200." "People can just pull up Codex and start controlling robots with it right now."

OpenAI's thijs predicts robotics will have its own version of the 3D printing craze, with people buying cheap, open-source robot arms and connecting them to agents. "I can see a world quite soon where, similar to when everyone went and bought 3D printers and ran software, everyone's going to buy plastic, cheaper robot arms that you can clip onto a table, put stuff in front of, and plug them into agents." "The price of classic robotics equipment is thousands of dollars at the moment. This arm I'm using here is a Hugging Face SO-100 robot, which is open source, fully 3D printable, and around $200." "People can just pull up Codex and start controlling robots with it right now."

TBPN

149,443 次观看 • 21 天前

Every mobile robot has the same problem. The arms are too heavy. Too weak. Too dumb… You bolt on a robot designed for a factory floor and wonder why your AMR’s battery dies in 2 hours. The payload-to-weight ratio kills you. The arm weighs 20kg and lifts 5. Mobile manipulation? A forklift with a wrist! Eris and Dimitris Sako, two brothers in Zurich (Duatic), spent years inside ETH’s Robotic Systems Lab staring at this exact problem. Their answer: control the muscle. They built their own Quasi-Direct Drive actuators from scratch. And an arm around them. And a robot around the arm. The DynaArm weighs 9 kg. Lifts 12. IP66. Backdrivable!!! …meaning it feels contact like a human arm does. Payload-to-weight ratio is inverted….the arm lifts more than itself! Their Alpha robot puts two of these on a wheeled base that carries 200 kg. Vision-guided. Hot-swappable batteries. Built for floor-to-shelf, not for selfies at trade shows (even though those are pretty good too). A machine that actually works in a warehouse. This is the European robotics playbook done right: deep hardware R&D → vertical integration → real product. Watch these two, and their team. I’m a huge fan of their work! 👋 —— Weekly robotics and AI insights. Subscribe free:

Every mobile robot has the same problem. The arms are too heavy. Too weak. Too dumb… You bolt on a robot designed for a factory floor and wonder why your AMR’s battery dies in 2 hours. The payload-to-weight ratio kills you. The arm weighs 20kg and lifts 5. Mobile manipulation? A forklift with a wrist! Eris and Dimitris Sako, two brothers in Zurich (Duatic), spent years inside ETH’s Robotic Systems Lab staring at this exact problem. Their answer: control the muscle. They built their own Quasi-Direct Drive actuators from scratch. And an arm around them. And a robot around the arm. The DynaArm weighs 9 kg. Lifts 12. IP66. Backdrivable!!! …meaning it feels contact like a human arm does. Payload-to-weight ratio is inverted….the arm lifts more than itself! Their Alpha robot puts two of these on a wheeled base that carries 200 kg. Vision-guided. Hot-swappable batteries. Built for floor-to-shelf, not for selfies at trade shows (even though those are pretty good too). A machine that actually works in a warehouse. This is the European robotics playbook done right: deep hardware R&D → vertical integration → real product. Watch these two, and their team. I’m a huge fan of their work! 👋 —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

25,966 次观看 • 6 个月前

you don’t need a robot to see physical ai in action, you just need a browser. this week I finally tried something I've wanted to do for a while: design an SO-101 scene in three.js, connect it to a real leader arm, collect teleoperation data, train an ACT model, then port it to run in the browser. the result is the first ever demo of ACT that runs locally on the web. try it yourself now at

you don’t need a robot to see physical ai in action, you just need a browser. this week I finally tried something I've wanted to do for a while: design an SO-101 scene in three.js, connect it to a real leader arm, collect teleoperation data, train an ACT model, then port it to run in the browser. the result is the first ever demo of ACT that runs locally on the web. try it yourself now at

Binh

122,572 次观看 • 3 个月前

1/5 🚀 Thrilled to open-source OSCAR 🤖 — an action-conditioned world model for robotics, led by the visiting student in my group Zhuoyuan Wu! It generalizes across different robot embodiments with precise action controllability. All trained on a single GH200 GPU, and outperforms existing open-sourced baselines, which have larger model capacity and need more compute. Everything is public, including training data. 📄 Paper: 🌐 Project: 💻 Code: 🤗 Robot data: 🤗 Human data: 🤗 Weights: #Robotics #WorldModels #AI #OpenSource

1/5 🚀 Thrilled to open-source OSCAR 🤖 — an action-conditioned world model for robotics, led by the visiting student in my group Zhuoyuan Wu! It generalizes across different robot embodiments with precise action controllability. All trained on a single GH200 GPU, and outperforms existing open-sourced baselines, which have larger model capacity and need more compute. Everything is public, including training data. 📄 Paper: 🌐 Project: 💻 Code: 🤗 Robot data: 🤗 Human data: 🤗 Weights: #Robotics #WorldModels #AI #OpenSource

Jun Gao

105,082 次观看 • 3 个月前

Introducing WorldForge: testable world-model workflows for physical AI systems. You can think of it, loosely, as “LangChain for world models”. The problem is that “world model” has become an overloaded label. Depending on context, it can mean a video generator, a cost model, a robot policy, a JEPA-style latent predictor, etc. They share almost nothing, different inputs, runtimes, failure modes. I built WorldForge to stop pretending they're interchangeable. Front-door demo: a real Hugging Face LeRobot (LeRobot) diffusion_pusht policy combined with LeWorldModel by Lucas Maes checkpoint for scoring. Both run locally on my MacBook in the demo video. LeWM is extremely efficient (~15M params), can plan up to 48× faster, and runs on commodity hardware. WorldForge wires the loop: policy → candidates → score → select Replay happens in a local TUI today, but the same loop could drive a real robot in the world. Would love feedback from people working on world models, physical AI, robotics, ML infra, and adjacent tooling. Fully open source. Contributions very welcome. Plan in the dream, replay in real world.

Introducing WorldForge: testable world-model workflows for physical AI systems. You can think of it, loosely, as “LangChain for world models”. The problem is that “world model” has become an overloaded label. Depending on context, it can mean a video generator, a cost model, a robot policy, a JEPA-style latent predictor, etc. They share almost nothing, different inputs, runtimes, failure modes. I built WorldForge to stop pretending they're interchangeable. Front-door demo: a real Hugging Face LeRobot (LeRobot) diffusion_pusht policy combined with LeWorldModel by Lucas Maes checkpoint for scoring. Both run locally on my MacBook in the demo video. LeWM is extremely efficient (~15M params), can plan up to 48× faster, and runs on commodity hardware. WorldForge wires the loop: policy → candidates → score → select Replay happens in a local TUI today, but the same loop could drive a real robot in the world. Would love feedback from people working on world models, physical AI, robotics, ML infra, and adjacent tooling. Fully open source. Contributions very welcome. Plan in the dream, replay in real world.

abdel

17,001 次观看 • 5 个月前

Excited to announce GR00T N1, the world’s first open foundation model for humanoid robots! We are on a mission to democratize Physical AI. The power of general robot brain, in the palm of your hand - with only 2B parameters, N1 learns from the most diverse physical action dataset ever compiled and punches above its weight: - Real humanoid teleoperation data. - Large-scale simulation data: we are open-sourcing 300K+ trajectories! - Neural trajectories: we apply SOTA video generation models to “hallucinate” new synthetic data that features accurate physics in pixels. Using Jensen’s words, “systematically infinite data”! - Latent actions: we develop novel algorithms to extract action tokens from in-the-wild human videos and neural generated videos. GR00T N1 is a single end-to-end neural net, from photons to actions: - Vision-Language Model (System 2) that interprets the physical world through vision and language instructions, enabling robots to reason about their environment and instructions, and plan the right actions. - Diffusion Transformer (System 1) that “renders” smooth and precise motor actions at 120 Hz, executing the latent plan made by System 2. We deploy N1 on GR1 robot, 1X Neo robot, and a large collection of simulation benchmarks. N1 achieves up to +30% boost in diverse manipulation tasks for household and industrial settings. While humanoid robots are the main focus of N1, our model also supports cross-embodiment. We finetune it to work on the $110 HuggingFace LeRobot SO100 robot arm! Open robot brain runs on open hardware. Sounds just right. Let’s solve robotics, together, one token at a time. Links to our Whitepaper, Github repo, HuggingFace model, and open dataset page in the thread: 🧵

Excited to announce GR00T N1, the world’s first open foundation model for humanoid robots! We are on a mission to democratize Physical AI. The power of general robot brain, in the palm of your hand - with only 2B parameters, N1 learns from the most diverse physical action dataset ever compiled and punches above its weight: - Real humanoid teleoperation data. - Large-scale simulation data: we are open-sourcing 300K+ trajectories! - Neural trajectories: we apply SOTA video generation models to “hallucinate” new synthetic data that features accurate physics in pixels. Using Jensen’s words, “systematically infinite data”! - Latent actions: we develop novel algorithms to extract action tokens from in-the-wild human videos and neural generated videos. GR00T N1 is a single end-to-end neural net, from photons to actions: - Vision-Language Model (System 2) that interprets the physical world through vision and language instructions, enabling robots to reason about their environment and instructions, and plan the right actions. - Diffusion Transformer (System 1) that “renders” smooth and precise motor actions at 120 Hz, executing the latent plan made by System 2. We deploy N1 on GR1 robot, 1X Neo robot, and a large collection of simulation benchmarks. N1 achieves up to +30% boost in diverse manipulation tasks for household and industrial settings. While humanoid robots are the main focus of N1, our model also supports cross-embodiment. We finetune it to work on the $110 HuggingFace LeRobot SO100 robot arm! Open robot brain runs on open hardware. Sounds just right. Let’s solve robotics, together, one token at a time. Links to our Whitepaper, Github repo, HuggingFace model, and open dataset page in the thread: 🧵

Jim Fan

467,416 次观看 • 1 年前