LeRobot
@LeRobotHF • 19,122 subscribers
~ Lowering the barrier to entry for robotics ~ Crafted with care by @HuggingFace 🤗 Join our discord: https://t.co/Sx2jdT0jeF
Shorts
Videos
We took part in the research preview of MHS from Anthropic Here is Claude Code running a real SO-ARM101. Nothing in this was trained - no policy, no teleoperation, no demonstrations. The agent measured the workspace itself and wrote the motion. The calibration is the interesting part. No checkerboard, no camera intrinsics: the arm is its own ruler. Torque drops, a human rests the closed gripper on 16 dots the software draws in the camera view, and the robot reads back where they are. 4.1 mm position accuracy, 3.0 mm placement. Best run so far: 12 bricks placed with all four colour groups formed. A full hands-off run, start to finish, is what's next. Research preview today, open source coming soon.
LeRobot54,902 görüntüleme • 5 gün önce
VLA-JEPA just dropped in LeRobot 🤖 What makes this model special is that it does not just learn what action to take from a given observation, it also leverages a JEPA world model to learn action-relevant dynamics. During training, the VLA leverages V-JEPA2 by conditioning its predictor. This clever trick adds a world modeling objective to the training, which also allows pretraining on human videos. At inference, the world model is dropped entirely, keeping only a standard VLA architecture: Qwen backbone and action head. The demo here was only fine-tuned on 13 examples, showing great pretraining capability and running in real time on NVIDIA Robotics DGX Spark! VLA-JEPA is the first world model to be ported to LeRobot, and I feel like it won't be the last 🚀 Thomas Wolf clem 🤗
LeRobot320,328 görüntüleme • 2 ay önce
100,000 hours of human hands doing real work. LeRobot format, fully annotated and open. EgoSuite-Open100K from Lightwheel X Hugging Face: 15,000+ tasks across 15,000+ real scenes, hand + body pose and wrist cams. Recorded across 7 environment categories and 128 scene types, from kitchens and bedrooms to warehouse floors and assembly lines. It's open for academic research and commercial training, and streams straight from the Hub. Always great to see a dataset at this scale released openly on the Hub. More info:
LeRobot44,047 görüntüleme • 12 gün önce
Picking up laundry with a 3D printed robot arm and LeRobot. It operates using SO-ARM101 arm mounted on a mobile base, relying on just a wrist and base camera. The base is controlled with an Xbox controller by a human operator. The arm is fully autonomous thanks to the ACT model. The robot can generalize to previously unseen environments. The technical stack: - Model: Action Chunking Transformer (ACT) with Temporal Ensemble - Data: Learned via teleoperation including negative examples and failure recoveries - Framework: Built and trained locally using LeRobot on a single consumer GPU in 3.5h This is a perfect example that real-world robotics isn't about perfect lab conditions - it's about messy living rooms, confusing shadows, and learning to recover from mistakes. And with open tools, anyone can tackle these edge cases right from their desk.
LeRobot67,896 görüntüleme • 26 gün önce
Releasing the Unfolding Robotics blog! Time to unfold robotics: we trained a robot to fold clothes using 8 bimanual setups, 100+ hours of demonstrations, and 5k+ GPU hours. Flashy robot demos are everywhere. But you rarely see the real story: the data, the failures, the engineering. We’re sharing everything: code, data, and details in the blog →
LeRobot282,162 görüntüleme • 4 ay önce
LeRobot v0.6.0 is officially here: Imagine, Evaluate, Improve! 🤖🚀 We are closing the robot learning loop with massive upgrades for the open-source robotics community. From policies that imagine the future to a much leaner installation, here is what is new: - 🌍 World Models: VLA-JEPA, LingBot-VA, and FastWAM help your policies anticipate the future. - 👀 VLA Expansion: Welcome GR00T 1.7, MolmoAct2, EO-1, Multitask DiT, and EVO1. - 🏅 Reward Models API: Track success seamlessly with Robometer and TOPReward. - 🎯 Unified Evaluation: 6 new simulation benchmarks, all accessible via the lerobot-eval CLI. - ☁️ And more: lerobot-rollout CLI for DAgger corrections, HF Jobs cloud training, up to 2x faster data loading, GUI - LeLab, many docs improvements Ready to build the future of robotics? Dive into the full release notes here: clem 🤗 Thomas Wolf
LeRobot55,284 görüntüleme • 1 ay önce
Meet VLA-JEPA, the first world model policy in LeRobot 🌍 Most VLAs just learn observation → action. VLA-JEPA also learns action-relevant dynamics: during training, a JEPA world model has to predict upcoming frames in latent space from the model's own actions. Then, at inference, the world model disappears entirely, leaving a standard, fast VLA: Qwen backbone, action head, nothing extra. In our demo it was fine-tuned on just 13 examples and ran in real time on an NVIDIA DGX Spark. On a consumer RTX 3080, it hits 10Hz using under 6GB of VRAM. World-model supervision during training, zero extra cost at inference. That's the whole trick.
LeRobot42,389 görüntüleme • 1 ay önce
Your robot can now see in depth 👀📏 LeRobot v0.6.0 adds real end-to-end depth support. Plug in an Intel RealSense, set use_depth: true, and depth maps join your RGB observations automatically: captured in millimeters, compressed into compact 12-bit depth video alongside your normal camera streams, and decoded back to physical units at training time. Depth renders live while you record and in the dataset viewer, and it works across SO-100/101, Koch, OpenArm, reBot, Unitree G1 and more. You also get full control over how everything gets encoded, RGB or depth, codec, quality, presets, all of it. More on how it works:
LeRobot30,730 görüntüleme • 1 ay önce
Train AI robots without writing a single line of code. 🤖 We just launched LeLab, the official graphical user interface for LeRobot built by Nicolas Rabault. It completely removes the command line from the robot learning workflow, taking you from raw hardware to autonomous movement visually. If you've ever wanted to get into AI robotics but were held back by complex terminal setups, this is for you. - Zero-Terminal Setup: Smart calibration with automatic USB port detection. - Easy Data Collection: Teleoperate your robot and record a dataset. - One-Click GPU Training: Don't have a massive local GPU? Scale your training instantly with Hugging Face Jobs right inside the app. Just plug in your SO-ARM101 and start teaching your robot. We put together a complete, step-by-step video guide showing exactly how to get started and train your first policy. Docs: GitHub:
LeRobot50,515 görüntüleme • 3 ay önce
Is it really worth it joining a robotics hackathon? One year ago, thousands of people around the world spent a single weekend building AI robots using the open-source LeRobot stack. Many had never built a robot before. We sat down with the winners to ask what they made and where they are now. For many of them, that weekend changed everything, turning the intimidating field of AI robotics into an accessible hobby or even a full-time career. Here are a few key takeaways from their year-long journey: - You don't need a PhD to start. For many participants, this was their very first robot - One weekend can launch a career. Winning secured internships and full-time jobs - Community lowers the barrier to entry - Learning by doing is exponentially faster. Robotics still requires getting your hands dirty to truly progress Watch the full video to see their projects and hear their advice for taking the first step. clem 🤗, Thomas Wolf
LeRobot40,300 görüntüleme • 2 ay önce
This is the full video of the hardest version of the task: t-shirt folding from unstructured initial states. This setting really requires at least some strategy, since the robot first has to spread the shirt before it can complete the fold. Full details on data collection strategies in the blog below. 👇
LeRobot53,936 görüntüleme • 4 ay önce
How do you know your robot actually succeeded? In v0.6.0, you ask a VLM 🤖 LeRobot ships a new unified reward models API with two zero-shot additions. Robometer is a general-purpose reward model trained on 1M+ trajectories across 21 robot embodiments: point it at any LeRobot dataset and it scores task progress and success straight from video and a language instruction, no task-specific training needed. TOPReward goes even further: no reward weights at all. It wraps an off-the-shelf VLM and reads the log-probability of the token "True" given the video and the instruction. Both come with labeling scripts that write per-frame progress curves into your dataset, ready for reward-aware behavior cloning. More on how they work:
LeRobot22,358 görüntüleme • 1 ay önce
MolmoAct2 runs zero-shot on an SO-ARM101, no training required 🦾 Ai2 open Action Reasoning Model is now in LeRobot with the full lifecycle covered: fine-tuning (full or LoRA), evaluation, and real-robot deployment. It pairs a Molmo2-ER vision-language backbone with a flow-matching action expert to turn images, language, and proprioceptive state into action chunks. The best part: ready-made checkpoints ship with calibration correction baked in, so you can point it at your SO-100/101 and just watch it work. Inference fits in ~12GB at bf16, and LoRA fine-tuning runs on a single 24GB GPU. Big thanks to the Ai2 team for building this in the open.
LeRobot22,200 görüntüleme • 1 ay önce
![GR00T 1.7 just landed in LeRobot 🟢 We upgraded our NVIDIA Robotics GR00T integration to 1.7, the newest open generation of NVIDIA's cross-embodiment foundation model. It swaps the previous VLM for Cosmos-Reason2-2B feeding a flow-matching action head, and we parity-tested it against NVIDIA's own Isaac GR00T implementation: same inputs, same outputs. Flash-attention is now optional too, so pip install 'lerobot[groot]' just works, and you can load NVIDIA's published checkpoints directly. Isaac Teleop is now in the ecosystem too, letting you drive an SO-101 with a VR controller over CloudXR/OpenXR.](https://image.24vids.com/tw-2075188216433615085/amplify_video_thumb/2075160927603720192/img/LwyLjK9Qa_-qj8Bn.jpg)
GR00T 1.7 just landed in LeRobot 🟢 We upgraded our NVIDIA Robotics GR00T integration to 1.7, the newest open generation of NVIDIA's cross-embodiment foundation model. It swaps the previous VLM for Cosmos-Reason2-2B feeding a flow-matching action head, and we parity-tested it against NVIDIA's own Isaac GR00T implementation: same inputs, same outputs. Flash-attention is now optional too, so pip install 'lerobot[groot]' just works, and you can load NVIDIA's published checkpoints directly. Isaac Teleop is now in the ecosystem too, letting you drive an SO-101 with a VR controller over CloudXR/OpenXR.
LeRobot19,572 görüntüleme • 1 ay önce
🤖Adding new RL algorithms to LeRobot just got much easier. Demo: HIL-SERL training with a SAC-based RL algorithm on an SO-100 for a hole-in-hand peg-in-hole task. Sparse reward, only 30 offline demos mixed with live robot experience, and ~1 hour of online training with human interventions only when the policy fails. The bottom graph tracks intervention rate: high at the start, steadily dropping as the policy improves. The refactor separates algorithm logic from training infrastructure: • RLAlgorithm owns learning logic • RLTrainer handles orchestration • DataMixer combines rollouts, demos, interventions, and future data sources Adding an RL algorithm now looks much closer to adding a policy: one algorithm file, one config, one registry entry. SAC is first. RLT, RECAP, ConRFT, QC-FQL, DSRL, and VLA RL fine-tuning next! Thomas Wolf clem 🤗
LeRobot30,745 görüntüleme • 3 ay önce
Most robot datasets store one sentence per episode: the task. Real problems need more: a plan, a memory, room to answer questions and change course. We just shipped full language support in the LeRobot dataset format. Now lerobot-annotate points a vision-language model at each episode and writes the subtasks, plans, memory, and grounded Q&A itself - roughly $1.20 for 100 episodes. From there, recipes turn the annotated rows into policy-agnostic training messages, so the same dataset can teach planning, memory, tool calls, or low-level control, no dataset redesign required. Read how it all works on the blog:
LeRobot12,854 görüntüleme • 1 ay önce
Our folding project showed what is possible with open-source. Reproducing it shouldn't take a PhD. LeRobot now ships with AGENT_GUIDE.md → so now your AI agent knows the insights, how to set up an SO-101, record good data, and pick a policy for your GPU. Just ask!
LeRobot30,930 görüntüleme • 4 ay önce
🚀 We just shipped a big upgrade to our imitation-learning-in-simulation playground in LeRobot, built together with Lightwheel ! You can now teleoperate robots in sim (keyboard or real robot) and collect training demos instantly. This makes it possible to run real IL research on harder, more realistic manipulation tasks, even if you don’t have hardware. New tasks: 🟠 pick orange to the plate 🧺 fold cloth (yes!) 📦 pick 2 “e” toys to the box 🔴 lift red cube Through our partnership with Lightwheel, LeIsaac was integrated into EnvHub on day one, a best-practice integration that strengthens both ecosystems and pushes simulation-first robotics forward. 👉 Load a task via EnvHub, start teleop-ing, record your demo, upload the data on the hub and you’re ready to train or test.
LeRobot44,647 görüntüleme • 9 ay önce
🎯 One CLI to evaluate every VLA benchmark Evaluating a robot policy used to mean wrangling six different repos, six different setups. Not anymore. LeRobot 0.6.0 ships 6 new simulation benchmarks - all runnable through the same lerobot-eval CLI, each with its own docs page, Docker image, and SmolVLA baseline tested in CI: 🧩 LIBERO-plus - ~10,000 perturbed variants across 7 axes, tells you exactly when a policy breaks 🤲 RoboTwin 2.0 - 50 bimanual tasks on SAPIEN + 100k+ ready-to-train trajectories on the Hub 🍳 RoboCasa365 - 365 kitchen tasks across 2,500 procedurally generated kitchens 🧠 RoboCerebra - long-horizon episodes chaining 3–6 sub-goals 🔁 RoboMME - a memory exam: count, track, imitate 📚 VLABench - knowledge & reasoning, from physics to brewing coffee end to end Add LIBERO, Meta-World, and NVIDIA IsaacLab-Arena, and that's 9 benchmark families under one roof - with parallel eval now up to 2x faster. See it in action below, and dive into the full release:
LeRobot11,307 görüntüleme • 1 ay önce
