LeRobot
@LeRobotHF • 20,677 subscribers
~ Lowering the barrier to entry for robotics ~ Crafted with care by @HuggingFace 🤗 Join our discord: https://t.co/Sx2jdT0jeF
Shorts
Videos
VLA-JEPA just dropped in LeRobot 🤖 What makes this model special is that it does not just learn what action to take from a given observation, it also leverages a JEPA world model to learn action-relevant dynamics. During training, the VLA leverages V-JEPA2 by conditioning its predictor. This clever trick adds a world modeling objective to the training, which also allows pretraining on human videos. At inference, the world model is dropped entirely, keeping only a standard VLA architecture: Qwen backbone and action head. The demo here was only fine-tuned on 13 examples, showing great pretraining capability and running in real time on NVIDIA Robotics DGX Spark! VLA-JEPA is the first world model to be ported to LeRobot, and I feel like it won't be the last 🚀 Thomas Wolf clem 🤗
LeRobot321,209 просмотров • 4 месяцев назад
Releasing the Unfolding Robotics blog! Time to unfold robotics: we trained a robot to fold clothes using 8 bimanual setups, 100+ hours of demonstrations, and 5k+ GPU hours. Flashy robot demos are everywhere. But you rarely see the real story: the data, the failures, the engineering. We’re sharing everything: code, data, and details in the blog →
LeRobot283,166 просмотров • 6 месяцев назад
We took part in the research preview of MHS from Anthropic Here is Claude Code running a real SO-ARM101. Nothing in this was trained - no policy, no teleoperation, no demonstrations. The agent measured the workspace itself and wrote the motion. The calibration is the interesting part. No checkerboard, no camera intrinsics: the arm is its own ruler. Torque drops, a human rests the closed gripper on 16 dots the software draws in the camera view, and the robot reads back where they are. 4.1 mm position accuracy, 3.0 mm placement. Best run so far: 12 bricks placed with all four colour groups formed. A full hands-off run, start to finish, is what's next. Research preview today, open source coming soon.
LeRobot57,736 просмотров • 1 месяц назад
Picking up laundry with a 3D printed robot arm and LeRobot. It operates using SO-ARM101 arm mounted on a mobile base, relying on just a wrist and base camera. The base is controlled with an Xbox controller by a human operator. The arm is fully autonomous thanks to the ACT model. The robot can generalize to previously unseen environments. The technical stack: - Model: Action Chunking Transformer (ACT) with Temporal Ensemble - Data: Learned via teleoperation including negative examples and failure recoveries - Framework: Built and trained locally using LeRobot on a single consumer GPU in 3.5h This is a perfect example that real-world robotics isn't about perfect lab conditions - it's about messy living rooms, confusing shadows, and learning to recover from mistakes. And with open tools, anyone can tackle these edge cases right from their desk.
LeRobot69,293 просмотров • 2 месяцев назад
100,000 hours of human hands doing real work. LeRobot format, fully annotated and open. EgoSuite-Open100K from Lightwheel X Hugging Face: 15,000+ tasks across 15,000+ real scenes, hand + body pose and wrist cams. Recorded across 7 environment categories and 128 scene types, from kitchens and bedrooms to warehouse floors and assembly lines. It's open for academic research and commercial training, and streams straight from the Hub. Always great to see a dataset at this scale released openly on the Hub. More info:
LeRobot45,647 просмотров • 1 месяц назад
LeRobot v0.6.0 is officially here: Imagine, Evaluate, Improve! 🤖🚀 We are closing the robot learning loop with massive upgrades for the open-source robotics community. From policies that imagine the future to a much leaner installation, here is what is new: - 🌍 World Models: VLA-JEPA, LingBot-VA, and FastWAM help your policies anticipate the future. - 👀 VLA Expansion: Welcome GR00T 1.7, MolmoAct2, EO-1, Multitask DiT, and EVO1. - 🏅 Reward Models API: Track success seamlessly with Robometer and TOPReward. - 🎯 Unified Evaluation: 6 new simulation benchmarks, all accessible via the lerobot-eval CLI. - ☁️ And more: lerobot-rollout CLI for DAgger corrections, HF Jobs cloud training, up to 2x faster data loading, GUI - LeLab, many docs improvements Ready to build the future of robotics? Dive into the full release notes here: clem 🤗 Thomas Wolf
LeRobot60,152 просмотров • 3 месяцев назад
This is so cool! One of our community members built a Space on Hugging Face with step by step build instructions for SO-ARM101. It includes nice 3D visualizations of each step with detailed description. Here is a link to the space: Join our Discord and share what you built:
LeRobot19,876 просмотров • 1 месяц назад
🎉 1,000 datasets have now been recorded with LeLab! Our goal: make the journey from connecting a robot → recording demonstrations → training → real-world testing as accessible as possible 🤗 Thank you to everyone building and experimenting with us ❤️ What we’ve learned + what’s next for LeLab 👇
LeRobot13,969 просмотров • 25 дней назад
Meet VLA-JEPA, the first world model policy in LeRobot 🌍 Most VLAs just learn observation → action. VLA-JEPA also learns action-relevant dynamics: during training, a JEPA world model has to predict upcoming frames in latent space from the model's own actions. Then, at inference, the world model disappears entirely, leaving a standard, fast VLA: Qwen backbone, action head, nothing extra. In our demo it was fine-tuned on just 13 examples and ran in real time on an NVIDIA DGX Spark. On a consumer RTX 3080, it hits 10Hz using under 6GB of VRAM. World-model supervision during training, zero extra cost at inference. That's the whole trick.
LeRobot42,389 просмотров • 3 месяцев назад
Train AI robots without writing a single line of code. 🤖 We just launched LeLab, the official graphical user interface for LeRobot built by Nicolas Rabault. It completely removes the command line from the robot learning workflow, taking you from raw hardware to autonomous movement visually. If you've ever wanted to get into AI robotics but were held back by complex terminal setups, this is for you. - Zero-Terminal Setup: Smart calibration with automatic USB port detection. - Easy Data Collection: Teleoperate your robot and record a dataset. - One-Click GPU Training: Don't have a massive local GPU? Scale your training instantly with Hugging Face Jobs right inside the app. Just plug in your SO-ARM101 and start teaching your robot. We put together a complete, step-by-step video guide showing exactly how to get started and train your first policy. Docs: GitHub:
LeRobot50,857 просмотров • 4 месяцев назад
$28 3D printed teleoperator device. And of course it's open source! The design features clean cable routing, same resting position as the SO-ARM101 and optional rubber bands that improve teleoperation experience (great idea). Used with MuJoCo simulator in the video below but can also be used with the follower arm. Awesome contribution from our community member! Check out the project here:
LeRobot22,811 просмотров • 1 месяц назад
Is it really worth it joining a robotics hackathon? One year ago, thousands of people around the world spent a single weekend building AI robots using the open-source LeRobot stack. Many had never built a robot before. We sat down with the winners to ask what they made and where they are now. For many of them, that weekend changed everything, turning the intimidating field of AI robotics into an accessible hobby or even a full-time career. Here are a few key takeaways from their year-long journey: - You don't need a PhD to start. For many participants, this was their very first robot - One weekend can launch a career. Winning secured internships and full-time jobs - Community lowers the barrier to entry - Learning by doing is exponentially faster. Robotics still requires getting your hands dirty to truly progress Watch the full video to see their projects and hear their advice for taking the first step. clem 🤗, Thomas Wolf
LeRobot40,509 просмотров • 3 месяцев назад
Your robot can now see in depth 👀📏 LeRobot v0.6.0 adds real end-to-end depth support. Plug in an Intel RealSense, set use_depth: true, and depth maps join your RGB observations automatically: captured in millimeters, compressed into compact 12-bit depth video alongside your normal camera streams, and decoded back to physical units at training time. Depth renders live while you record and in the dataset viewer, and it works across SO-100/101, Koch, OpenArm, reBot, Unitree G1 and more. You also get full control over how everything gets encoded, RGB or depth, codec, quality, presets, all of it. More on how it works:
LeRobot31,119 просмотров • 3 месяцев назад
This is the full video of the hardest version of the task: t-shirt folding from unstructured initial states. This setting really requires at least some strategy, since the robot first has to spread the shirt before it can complete the fold. Full details on data collection strategies in the blog below. 👇
LeRobot53,991 просмотров • 6 месяцев назад
MolmoAct2 runs zero-shot on an SO-ARM101, no training required 🦾 Ai2 open Action Reasoning Model is now in LeRobot with the full lifecycle covered: fine-tuning (full or LoRA), evaluation, and real-robot deployment. It pairs a Molmo2-ER vision-language backbone with a flow-matching action expert to turn images, language, and proprioceptive state into action chunks. The best part: ready-made checkpoints ship with calibration correction baked in, so you can point it at your SO-100/101 and just watch it work. Inference fits in ~12GB at bf16, and LoRA fine-tuning runs on a single 24GB GPU. Big thanks to the Ai2 team for building this in the open.
LeRobot23,339 просмотров • 3 месяцев назад
How do you know your robot actually succeeded? In v0.6.0, you ask a VLM 🤖 LeRobot ships a new unified reward models API with two zero-shot additions. Robometer is a general-purpose reward model trained on 1M+ trajectories across 21 robot embodiments: point it at any LeRobot dataset and it scores task progress and success straight from video and a language instruction, no task-specific training needed. TOPReward goes even further: no reward weights at all. It wraps an off-the-shelf VLM and reads the log-probability of the token "True" given the video and the instruction. Both come with labeling scripts that write per-frame progress curves into your dataset, ready for reward-aware behavior cloning. More on how they work:
LeRobot22,592 просмотров • 3 месяцев назад
🤖Adding new RL algorithms to LeRobot just got much easier. Demo: HIL-SERL training with a SAC-based RL algorithm on an SO-100 for a hole-in-hand peg-in-hole task. Sparse reward, only 30 offline demos mixed with live robot experience, and ~1 hour of online training with human interventions only when the policy fails. The bottom graph tracks intervention rate: high at the start, steadily dropping as the policy improves. The refactor separates algorithm logic from training infrastructure: • RLAlgorithm owns learning logic • RLTrainer handles orchestration • DataMixer combines rollouts, demos, interventions, and future data sources Adding an RL algorithm now looks much closer to adding a policy: one algorithm file, one config, one registry entry. SAC is first. RLT, RECAP, ConRFT, QC-FQL, DSRL, and VLA RL fine-tuning next! Thomas Wolf clem 🤗
LeRobot31,237 просмотров • 5 месяцев назад