Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Most robot datasets store one sentence per episode: the task. Real problems need more: a plan, a memory, room to answer questions and change course. We just shipped full language support in the LeRobot dataset format. Now lerobot-annotate points a vision-language model at each episode and writes the subtasks,...

LeRobotHF's profile picture

LeRobot

20,060 subscribers

13,451 Aufrufe • vor 3 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

7 Kommentare

Profilbild von Nick Venturi
Nick Venturivor 3 Monaten

now robots can explain why they ignored you

Profilbild von Lex
Lexvor 3 Monaten

Good to see richer annotations getting easier and cheaper. The auto-generated subtasks and Q&A could be really useful for training.

Profilbild von Sam W
Sam Wvor 3 Monaten

Is the nvidia dgx spark good for training and sims for a robotic arm to cook food ?

Profilbild von antonio_machine_learning
antonio_machine_learningvor 3 Monaten

just tell us the command to run a demo that shows off this feature, most of the documentation is out of sync and nothing works.

Profilbild von 起岚
起岚vor 2 Monaten

这条说到点子上了。我们采集系统每条 episode 只存一句 task，回看数据经常后悔没多记几笔。episode 元数据这块全行业都是欠账。

Profilbild von Molty
Moltyvor 3 Monaten

Plans and memory at $1.20 per 100 episodes. Does annotated data improve policy success?

Profilbild von Ebokify
Ebokifyvor 3 Monaten

Robot Modeling and Control (2 ed)

Ähnliche Videos

Introducing Subtask Annotations in LeRobot Datasets. LeRobot now supports subtasks as a native dataset field, so you can label task progression frame-by-frame, train hierarchical policies, and build stronger reward models with stage-aware supervision. Available today, with an annotation space + tooling to easily add subtasks and high-level dialogue to your dataset.
0:52

Introducing Subtask Annotations in LeRobot Datasets. LeRobot now supports subtasks as a native dataset field, so you can label task progression frame-by-frame, train hierarchical policies, and build stronger reward models with stage-aware supervision. Available today, with an annotation space + tooling to easily add subtasks and high-level dialogue to your dataset.

LeRobot

16,524 Aufrufe • vor 9 Monaten

How do you know your robot actually succeeded? In v0.6.0, you ask a VLM 🤖 LeRobot ships a new unified reward models API with two zero-shot additions. Robometer is a general-purpose reward model trained on 1M+ trajectories across 21 robot embodiments: point it at any LeRobot dataset and it scores task progress and success straight from video and a language instruction, no task-specific training needed. TOPReward goes even further: no reward weights at all. It wraps an off-the-shelf VLM and reads the log-probability of the token "True" given the video and the instruction. Both come with labeling scripts that write per-frame progress curves into your dataset, ready for reward-aware behavior cloning. More on how they work:
0:55

How do you know your robot actually succeeded? In v0.6.0, you ask a VLM 🤖 LeRobot ships a new unified reward models API with two zero-shot additions. Robometer is a general-purpose reward model trained on 1M+ trajectories across 21 robot embodiments: point it at any LeRobot dataset and it scores task progress and success straight from video and a language instruction, no task-specific training needed. TOPReward goes even further: no reward weights at all. It wraps an off-the-shelf VLM and reads the log-probability of the token "True" given the video and the instruction. Both come with labeling scripts that write per-frame progress curves into your dataset, ready for reward-aware behavior cloning. More on how they work:

LeRobot

22,592 Aufrufe • vor 3 Monaten

MemOS now has a plugin for OpenClaw that enables your AI agents to work on the common memory layer and cut down token usage. - Multiple agents read/write the same memory — no manual context handoff - 72% lower token costs (15.6M → 4.4M on LOCOMO dataset) - Cross-session and long-term memory that actually works Sponsored
1:31

MemOS now has a plugin for OpenClaw that enables your AI agents to work on the common memory layer and cut down token usage. - Multiple agents read/write the same memory — no manual context handoff - 72% lower token costs (15.6M → 4.4M on LOCOMO dataset) - Cross-session and long-term memory that actually works Sponsored

🚨 AI News | TestingCatalog

41,040 Aufrufe • vor 8 Monaten

Most "agentic" failures happen because the model lacks specific domain knowledge. Here I'm showing how loading a Skills Plugin solves that for dataset generation. I turned a research paper (shows fine-tuning dataset creation from books) into a Skill and just gave a book link and asked it to generate the dataset. Claude Code produced a clean SFT-ready dataset from a single URL. The same process can be applied to anything.
0:58

Most "agentic" failures happen because the model lacks specific domain knowledge. Here I'm showing how loading a Skills Plugin solves that for dataset generation. I turned a research paper (shows fine-tuning dataset creation from books) into a Skill and just gave a book link and asked it to generate the dataset. Claude Code produced a clean SFT-ready dataset from a single URL. The same process can be applied to anything.

Muratcan Koylan

104,519 Aufrufe • vor 10 Monaten

Introducing Memory 101 🧠 Every ai tool you use is a goldfish. brilliant for one chat, close the tab and it forgets you completely. memory is the most underbuilt layer in ai right now. so we're explaining how it actually works, one episode at a time. coming up: → rag vs memory, why most "ai memory" is just search → the 5 things real memory needs: facts, profiles, graphs, time, forgetting → why good memory has to forget things follow along!
0:50

Introducing Memory 101 🧠 Every ai tool you use is a goldfish. brilliant for one chat, close the tab and it forgets you completely. memory is the most underbuilt layer in ai right now. so we're explaining how it actually works, one episode at a time. coming up: → rag vs memory, why most "ai memory" is just search → the 5 things real memory needs: facts, profiles, graphs, time, forgetting → why good memory has to forget things follow along!

supermemory

17,235 Aufrufe • vor 4 Monaten

MolmoAct2 runs zero-shot on an SO-ARM101, no training required 🦾 Ai2 open Action Reasoning Model is now in LeRobot with the full lifecycle covered: fine-tuning (full or LoRA), evaluation, and real-robot deployment. It pairs a Molmo2-ER vision-language backbone with a flow-matching action expert to turn images, language, and proprioceptive state into action chunks. The best part: ready-made checkpoints ship with calibration correction baked in, so you can point it at your SO-100/101 and just watch it work. Inference fits in ~12GB at bf16, and LoRA fine-tuning runs on a single 24GB GPU. Big thanks to the Ai2 team for building this in the open.
0:30

MolmoAct2 runs zero-shot on an SO-ARM101, no training required 🦾 Ai2 open Action Reasoning Model is now in LeRobot with the full lifecycle covered: fine-tuning (full or LoRA), evaluation, and real-robot deployment. It pairs a Molmo2-ER vision-language backbone with a flow-matching action expert to turn images, language, and proprioceptive state into action chunks. The best part: ready-made checkpoints ship with calibration correction baked in, so you can point it at your SO-100/101 and just watch it work. Inference fits in ~12GB at bf16, and LoRA fine-tuning runs on a single 24GB GPU. Big thanks to the Ai2 team for building this in the open.

LeRobot

23,450 Aufrufe • vor 4 Monaten

ANTHROPIC KILLED THE VECTOR DATABASE AND REPLACED IT WITH A FOLDER no embeddings. no black-box retrieval. no special memory tools. the agent just reads and writes a folder the insight: your agent already knows how to use a filesystem. so Anthropic made long-term memory a directory of small markdown files, mounted straight into the agent's sandbox at /mnt/memory the agent manages it with the tools it already has, bash, read, write, edit, grep, and Anthropic auto-injects a note into the system prompt so the agent always knows the memory is there and where to look the result: > every change is an immutable version: full audit trail, point-in-time recovery, redaction > read-only or read-write per store, so many agents can safely share one memory > workspace-scoped and persistent, memory that survives every session end most teams bolt on a vector DB and pray retrieval works. Anthropic went the opposite way: compiled knowledge as plain, human-readable files you can open, diff, and edit yourself and there's a second layer. Anthropic's "Dreaming" runs offline, reviews past sessions and the store, and rewrites it into a cleaner, higher-signal version. the agent literally consolidates memory in its sleep. bookmark this. the most production-grade agent memory right now is just a well-organized folder, and Anthropic proved it.
0:50

ANTHROPIC KILLED THE VECTOR DATABASE AND REPLACED IT WITH A FOLDER no embeddings. no black-box retrieval. no special memory tools. the agent just reads and writes a folder the insight: your agent already knows how to use a filesystem. so Anthropic made long-term memory a directory of small markdown files, mounted straight into the agent's sandbox at /mnt/memory the agent manages it with the tools it already has, bash, read, write, edit, grep, and Anthropic auto-injects a note into the system prompt so the agent always knows the memory is there and where to look the result: > every change is an immutable version: full audit trail, point-in-time recovery, redaction > read-only or read-write per store, so many agents can safely share one memory > workspace-scoped and persistent, memory that survives every session end most teams bolt on a vector DB and pray retrieval works. Anthropic went the opposite way: compiled knowledge as plain, human-readable files you can open, diff, and edit yourself and there's a second layer. Anthropic's "Dreaming" runs offline, reviews past sessions and the store, and rewrites it into a cleaner, higher-signal version. the agent literally consolidates memory in its sleep. bookmark this. the most production-grade agent memory right now is just a well-organized folder, and Anthropic proved it.

NO1ennn

13,359 Aufrufe • vor 3 Monaten

🚀 Unitree open-sources UnifoLM-WBT-Dataset — a high-quality real-world humanoid robot whole-body teleoperation (WBT) dataset for open environments. 🥳Publicly available since March 5, 2026, the dataset will continue to receive high-frequency rolling updates. It aims to establish the most comprehensive real-world humanoid robot dataset in terms of scenario coverage, task complexity, and manipulation diversity. 👉 Explore the dataset here:
1:27

🚀 Unitree open-sources UnifoLM-WBT-Dataset — a high-quality real-world humanoid robot whole-body teleoperation (WBT) dataset for open environments. 🥳Publicly available since March 5, 2026, the dataset will continue to receive high-frequency rolling updates. It aims to establish the most comprehensive real-world humanoid robot dataset in terms of scenario coverage, task complexity, and manipulation diversity. 👉 Explore the dataset here:

Unitree

5,633,606 Aufrufe • vor 7 Monaten

Can 1,000 real-world 3D scenes make a state-of-the-art model better? At OVER Research, we tested it by fine-tuning R3, a state-of-the-art streaming 3D reconstruction model, on roughly 1,000 scenes from the OVER dataset. The result: 25% lower trajectory error on our real-world environments compared to the deployed public checkpoint. And we did it using only 1,000 scenes, the same open dataset anyone can download for free, out of the 285,000+ real-world 3D scenes OVER has today. Read the full OVER Research publication: Download the 1,000-scene dataset on Hugging Face:
0:26

Can 1,000 real-world 3D scenes make a state-of-the-art model better? At OVER Research, we tested it by fine-tuning R3, a state-of-the-art streaming 3D reconstruction model, on roughly 1,000 scenes from the OVER dataset. The result: 25% lower trajectory error on our real-world environments compared to the deployed public checkpoint. And we did it using only 1,000 scenes, the same open dataset anyone can download for free, out of the 285,000+ real-world 3D scenes OVER has today. Read the full OVER Research publication: Download the 1,000-scene dataset on Hugging Face:

Over the Reality 🌐

34,785 Aufrufe • vor 2 Monaten

Introducing Meta Perception Language Model (PLM): an open & reproducible vision-language model tackling challenging visual tasks. Learn more about how PLM can help the open source community build more capable computer vision systems. Read the research paper, and download the code and dataset:
1:11

Introducing Meta Perception Language Model (PLM): an open & reproducible vision-language model tackling challenging visual tasks. Learn more about how PLM can help the open source community build more capable computer vision systems. Read the research paper, and download the code and dataset:

AI at Meta

94,534 Aufrufe • vor 1 Jahr

Boris Cherny gave Claude Code a memory that has no database, no indexing, no setup, and 80% of Anthropic runs on it a CLAUDE.md "gets automatically read into context at the start of every session" 12:48 - CLAUDE.md loads into memory every session 13:15 - Keep memory short or it burns your context 15:48 - Memory stacks from you to your whole company 17:30 - The memory command shows what Claude remembers 19:33 - Hit pound and Claude remembers it forever the whole memory layer is files → drop a CLAUDE.md in your repo and Claude reads it in on every session → nest more in subfolders and it pulls them only where it works → stack them personal to project to enterprise so one config serves the whole team → hit the pound key mid task and Claude writes the new fact into memory itself the best coding agent's memory was never a database, it was markdown you can open and edit
27:26

Boris Cherny gave Claude Code a memory that has no database, no indexing, no setup, and 80% of Anthropic runs on it a CLAUDE.md "gets automatically read into context at the start of every session" 12:48 - CLAUDE.md loads into memory every session 13:15 - Keep memory short or it burns your context 15:48 - Memory stacks from you to your whole company 17:30 - The memory command shows what Claude remembers 19:33 - Hit pound and Claude remembers it forever the whole memory layer is files → drop a CLAUDE.md in your repo and Claude reads it in on every session → nest more in subfolders and it pulls them only where it works → stack them personal to project to enterprise so one config serves the whole team → hit the pound key mid task and Claude writes the new fact into memory itself the best coding agent's memory was never a database, it was markdown you can open and edit

NO1ennn

13,015 Aufrufe • vor 3 Monaten

Most C Programmers don't have a clue as to how the CPU works. Here is a lecture from my course on the C Language that introduces how to visualise the CPU as a C (or general software) Programmer.... More Details are at: Being able to model the behaviour of CPU provides a solid base to reason about your code. The summary is - - The CPU is a state machine which maintains it's states in registers. - It has busses to memory for requesting data/instruction read/writes. - Interrupts to attend to events. I discuss this and more in my course on the C Language.
11:41

Most C Programmers don't have a clue as to how the CPU works. Here is a lecture from my course on the C Language that introduces how to visualise the CPU as a C (or general software) Programmer.... More Details are at: Being able to model the behaviour of CPU provides a solid base to reason about your code. The summary is - - The CPU is a state machine which maintains it's states in registers. - It has busses to memory for requesting data/instruction read/writes. - Interrupts to attend to events. I discuss this and more in my course on the C Language.

Piyush Itankar

28,019 Aufrufe • vor 1 Jahr

We're releasing XDOF s ABC-130k dataset converted to RRD on a Hugging Face bucket, along with the conversion code. ABC-130k is the largest open-source bi-manual teleoperation dataset published to date. With Rerun, you can view all the episodes, query for insights, and use them in training workflows. One interesting finding is that annotators use varying levels of granularity for subtask labeling. Take the chess task in this video, for example: One episode uses language like 'place the chess piece', while another uses a fine-grained description like 'place the queen/pawn at E8'. This distinction is clearly visualized in Rerun through the colors in the state view! Our repo also includes a "mistake" query example and other interesting observations about the dataset, complementing the official ABC-130k dataset card. Download a subset and explore it yourself!! What other open datasets would you like to see in RRD?
1:24

We're releasing XDOF s ABC-130k dataset converted to RRD on a Hugging Face bucket, along with the conversion code. ABC-130k is the largest open-source bi-manual teleoperation dataset published to date. With Rerun, you can view all the episodes, query for insights, and use them in training workflows. One interesting finding is that annotators use varying levels of granularity for subtask labeling. Take the chess task in this video, for example: One episode uses language like 'place the chess piece', while another uses a fine-grained description like 'place the queen/pawn at E8'. This distinction is clearly visualized in Rerun through the colors in the state view! Our repo also includes a "mistake" query example and other interesting observations about the dataset, complementing the official ABC-130k dataset card. Download a subset and explore it yourself!! What other open datasets would you like to see in RRD?

Rerun

21,430 Aufrufe • vor 2 Monaten

Introducing Invent a Dataset. Describe the dataset you need. Get a structured, training-ready dataset back. No existing data needed. Dataset creation used to start with collection. Now it starts with specification.
0:29

Introducing Invent a Dataset. Describe the dataset you need. Get a structured, training-ready dataset back. No existing data needed. Dataset creation used to start with collection. Now it starts with specification.

adaption

93,053 Aufrufe • vor 2 Monaten

MICROSOFT QUIETLY BUILT ONE MEMORY MODULE THAT PLUGS INTO ANY AI AGENT AND CUTS ITS CONTEXT BY UP TO 100X no retraining. no task-specific redesign. you just plug it in Microsoft's insight: agents don't need to remember what happened. they need the knowledge pulled from it - the facts and the reusable skills, not the raw logs so PlugMem stops storing history. it compiles experience into a knowledge graph, then feeds the agent only what matters the result: > up to 1-2 orders of magnitude fewer tokens > highest decision-utility per token of any system tested > one unchanged module beats task-specific memory on chat QA, multi-hop search, and web agents more memory was never the answer. better memory is - and Microsoft just shipped it as a plug-in bookmark this. the agents that win won't remember more, they'll remember smarter.
0:42

MICROSOFT QUIETLY BUILT ONE MEMORY MODULE THAT PLUGS INTO ANY AI AGENT AND CUTS ITS CONTEXT BY UP TO 100X no retraining. no task-specific redesign. you just plug it in Microsoft's insight: agents don't need to remember what happened. they need the knowledge pulled from it - the facts and the reusable skills, not the raw logs so PlugMem stops storing history. it compiles experience into a knowledge graph, then feeds the agent only what matters the result: > up to 1-2 orders of magnitude fewer tokens > highest decision-utility per token of any system tested > one unchanged module beats task-specific memory on chat QA, multi-hop search, and web agents more memory was never the answer. better memory is - and Microsoft just shipped it as a plug-in bookmark this. the agents that win won't remember more, they'll remember smarter.

NO1ennn

227,956 Aufrufe • vor 3 Monaten

Adding vision to a language model is more than one extra modality. With large images or long video, the vision encoder can use more compute than the language model itself, so efficient models need a different design. Our VLM team, including ryan, Tianshu Yu, Sina Semnani, and Sam Stevens, explains how they teach LFMs to see, from mid-training to the custom vision encoder built for LFM3.
3:47

Adding vision to a language model is more than one extra modality. With large images or long video, the vision encoder can use more compute than the language model itself, so efficient models need a different design. Our VLM team, including ryan, Tianshu Yu, Sina Semnani, and Sam Stevens, explains how they teach LFMs to see, from mid-training to the custom vision encoder built for LFM3.

Liquid AI

11,633 Aufrufe • vor 4 Tagen

Your robot can now see in depth 👀📏 LeRobot v0.6.0 adds real end-to-end depth support. Plug in an Intel RealSense, set use_depth: true, and depth maps join your RGB observations automatically: captured in millimeters, compressed into compact 12-bit depth video alongside your normal camera streams, and decoded back to physical units at training time. Depth renders live while you record and in the dataset viewer, and it works across SO-100/101, Koch, OpenArm, reBot, Unitree G1 and more. You also get full control over how everything gets encoded, RGB or depth, codec, quality, presets, all of it. More on how it works:
0:46

Your robot can now see in depth 👀📏 LeRobot v0.6.0 adds real end-to-end depth support. Plug in an Intel RealSense, set use_depth: true, and depth maps join your RGB observations automatically: captured in millimeters, compressed into compact 12-bit depth video alongside your normal camera streams, and decoded back to physical units at training time. Depth renders live while you record and in the dataset viewer, and it works across SO-100/101, Koch, OpenArm, reBot, Unitree G1 and more. You also get full control over how everything gets encoded, RGB or depth, codec, quality, presets, all of it. More on how it works:

LeRobot

31,416 Aufrufe • vor 3 Monaten

We are live and kicking! For our latest project 1 dataset 100 visualizations, we challenged ourselves to turn one simple dataset into 100 insightful and visually appealing data visualizations. Explore all of the #dataviz and stories on
0:28

We are live and kicking! For our latest project 1 dataset 100 visualizations, we challenged ourselves to turn one simple dataset into 100 insightful and visually appealing data visualizations. Explore all of the #dataviz and stories on

ferdio

20,337 Aufrufe • vor 3 Jahren

LLM + Memory + Planning + Tools = Agents 🤖 Last month, Job and I discussed how generative AI is shifting how companies offer customer support. How can we add more layers to our RAG apps to make it more agentic? LLM: Large language model alone Memory: Short-term and long-term memory Planning: Breaking down questions into sub-questions, plan the steps needed before taking action, etc. Tools: Calculator, code executor, weather API, etc. I hope you find this small clip interesting! The link to the full recording is below.
2:19

LLM + Memory + Planning + Tools = Agents 🤖 Last month, Job and I discussed how generative AI is shifting how companies offer customer support. How can we add more layers to our RAG apps to make it more agentic? LLM: Large language model alone Memory: Short-term and long-term memory Planning: Breaking down questions into sub-questions, plan the steps needed before taking action, etc. Tools: Calculator, code executor, weather API, etc. I hope you find this small clip interesting! The link to the full recording is below.

Erika Shorten

32,443 Aufrufe • vor 2 Jahren

How do you scale 2 million hours of ego data without sacrificing quality? Earlier this week, we revealed the world’s largest egocentric dataset. Our frontier pipeline has also turned it into the most labeled, annotated, and human QA’ed robotics ego dataset in the world. We’ve focused on some of the hardest problems in computer vision and ego data to ensure every clip contains SOTA action supervision directly helpful for training dexterous manipulation.
0:21

How do you scale 2 million hours of ego data without sacrificing quality? Earlier this week, we revealed the world’s largest egocentric dataset. Our frontier pipeline has also turned it into the most labeled, annotated, and human QA’ed robotics ego dataset in the world. We’ve focused on some of the hardest problems in computer vision and ego data to ensure every clip contains SOTA action supervision directly helpful for training dexterous manipulation.

Midcentury

10,802 Aufrufe • vor 1 Monat