Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Your robot can now see in depth 👀📏 LeRobot v0.6.0 adds real end-to-end depth support. Plug in an Intel RealSense, set use_depth: true, and depth maps join your RGB observations automatically: captured in millimeters, compressed into compact 12-bit depth video alongside your normal camera streams, and decoded back to...

LeRobotHF's profile picture

LeRobot

20,209 subscribers

31,020 Aufrufe • vor 2 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

7 Kommentare

Profilbild von Steven Cheng
Steven Chengvor 2 Monaten

10 Robots Officially Supported by Hugging Face LeRobot — Which One Should You Use?

Profilbild von Ebokify
Ebokifyvor 1 Monat

Robot Modeling and Control (2 ed)

Profilbild von Ajith Isaac
Ajith Isaacvor 2 Monaten

Now we’re talking @pepijn2233 🎉🎉🎉

Profilbild von Martin Kemka
Martin Kemkavor 2 Monaten

Great timing. About to try adding a 405 to the arm

Profilbild von AI Mastery Guide
AI Mastery Guidevor 2 Monaten

Depth support baked in natively is huge for robotics work

Profilbild von clankr
clankrvor 2 Monaten

What do you think about depth estimation models for the next version? Actually, they could be used here as well.

Profilbild von Adel Bucetta
Adel Bucettavor 2 Monaten

most people still don't get that ai's not about rendering pretty images, it's about making them useful

Ähnliche Videos

Meta just unveiled Muse’s new real-time voice mode and it’s coming to Meta glasses soon!! Muse keeps working while you’re talking to it. So instead of the old turn-based conversation loop, you can have a long, in-depth conversation while it works in the background “You can have a long in-depth conversation with it… it actually works in the background while you are talking to it.” They seemed to make a big point during the keynote that Meta is making the voice itself customizable too: “You can just go ahead and describe what you want faster, slower, more southern, more Australian, more pirate… whatever you want.”
3:05

Meta just unveiled Muse’s new real-time voice mode and it’s coming to Meta glasses soon!! Muse keeps working while you’re talking to it. So instead of the old turn-based conversation loop, you can have a long, in-depth conversation while it works in the background “You can have a long in-depth conversation with it… it actually works in the background while you are talking to it.” They seemed to make a big point during the keynote that Meta is making the voice itself customizable too: “You can just go ahead and describe what you want faster, slower, more southern, more Australian, more pirate… whatever you want.”

Chris

103,808 Aufrufe • vor 4 Tagen

I've been playing more with this depth map motion transfer workflow, and if you're trying to recreate really specific body movements from a reference video, this works surprisingly well. Here's the workflow: 1. Grab your reference video, then create your character and environment references. 2. Run the video through Depth Anything on fal to generate a depth map: 3. Drop that depth map into Seedance Omni as your motion reference along with your character and environment references. You don't even need the original video. I also tested this against using the original video as the motion reference in Seedance, and there's definitely a noticeable difference. One thing I found, though, is that this works much better for body movement than facial performance since the depth map can't really capture subtle facial motion. I'm running more tests to see if there's a good way to control both body movement and facial performance. If you have any workflows I should try, do share in the comments!
0:52

I've been playing more with this depth map motion transfer workflow, and if you're trying to recreate really specific body movements from a reference video, this works surprisingly well. Here's the workflow: 1. Grab your reference video, then create your character and environment references. 2. Run the video through Depth Anything on fal to generate a depth map: 3. Drop that depth map into Seedance Omni as your motion reference along with your character and environment references. You don't even need the original video. I also tested this against using the original video as the motion reference in Seedance, and there's definitely a noticeable difference. One thing I found, though, is that this works much better for body movement than facial performance since the depth map can't really capture subtle facial motion. I'm running more tests to see if there's a good way to control both body movement and facial performance. If you have any workflows I should try, do share in the comments!

Curious Refuge

16,487 Aufrufe • vor 1 Monat

The robot’s “eyes” just received a big upgrade. LingBot-Depth 2.0, a depth-completion model with half the depth error just dropped. 12/16 benchmarks topped. Glass, mirrors, and transparent objects are so easy for us humans, but so hard for robots, because they do not behave like ordinary surfaces in a camera pipeline. A robot that misunderstands a balcony window or a table edge, will have a completely false planning inside a false world. Huge implecation. LingBot-Depth 2.0 takes an RGB image plus a broken depth map from a sensor and then outputs a cleaner depth map and a usable 3D point cloud. Numbers on LingBot-Depth 2.0 - Excels on glass, mirrors & transparent objects — where traditional depth cameras fail - Training data: 3M → 150M (50x scale-up) - 12 out of 16 first-place rankings on depth completion benchmarks - RMSE cut in half: 0.132 → 0.062 on the hardest indoor scenes LingBot-Vision trained on boundaries, because object edges carry the geometry robots need. No human boundary labels are used, which makes this approach easier to scale. The open-sourced LingBot-Vision is the general vision backbone, and LingBot-Depth 2.0 is the depth model built on it.
2:22

The robot’s “eyes” just received a big upgrade. LingBot-Depth 2.0, a depth-completion model with half the depth error just dropped. 12/16 benchmarks topped. Glass, mirrors, and transparent objects are so easy for us humans, but so hard for robots, because they do not behave like ordinary surfaces in a camera pipeline. A robot that misunderstands a balcony window or a table edge, will have a completely false planning inside a false world. Huge implecation. LingBot-Depth 2.0 takes an RGB image plus a broken depth map from a sensor and then outputs a cleaner depth map and a usable 3D point cloud. Numbers on LingBot-Depth 2.0 - Excels on glass, mirrors & transparent objects — where traditional depth cameras fail - Training data: 3M → 150M (50x scale-up) - 12 out of 16 first-place rankings on depth completion benchmarks - RMSE cut in half: 0.132 → 0.062 on the hardest indoor scenes LingBot-Vision trained on boundaries, because object edges carry the geometry robots need. No human boundary labels are used, which makes this approach easier to scale. The open-sourced LingBot-Vision is the general vision backbone, and LingBot-Depth 2.0 is the depth model built on it.

Rohan Paul

25,312 Aufrufe • vor 2 Monaten

New video series: How to Amiko In this series, we’ll walk you through how to use the Amiko site, from onboarding your twin to exploring everything live inside our Beta. Episode 2: Train Your Twin Once you create your social and behavioral twin, shape what it knows about you and how it understands you in Training Chats. Training your twin builds depth: this is where you give it your personal traits, memories, preferences, history. This is where it learns specific, personal information about you, always controlled by you. Training teaches your twin more about you, beyond the social and behavioral layer, and keeps you in control of the data the twin has access to. Without training, your twin will act like the oldest child in a large family. With training, it will know you were the oldest of five siblings, and any additional details you wish to disclose.
1:18

New video series: How to Amiko In this series, we’ll walk you through how to use the Amiko site, from onboarding your twin to exploring everything live inside our Beta. Episode 2: Train Your Twin Once you create your social and behavioral twin, shape what it knows about you and how it understands you in Training Chats. Training your twin builds depth: this is where you give it your personal traits, memories, preferences, history. This is where it learns specific, personal information about you, always controlled by you. Training teaches your twin more about you, beyond the social and behavioral layer, and keeps you in control of the data the twin has access to. Without training, your twin will act like the oldest child in a large family. With training, it will know you were the oldest of five siblings, and any additional details you wish to disclose.

AMIKO

18,972 Aufrufe • vor 10 Monaten

Athena ≠ research mode. It's an entire execution environment for ASI:One. That means your personal AI gets somewhere to actually work. Instead of stopping at a summary you can read on the go, your personal AI can keep going until the in-depth documentation, presentation, or report is ready. Same intelligence. Deeper execution. Here’s how it works 👇
1:25

Athena ≠ research mode. It's an entire execution environment for ASI:One. That means your personal AI gets somewhere to actually work. Instead of stopping at a summary you can read on the go, your personal AI can keep going until the in-depth documentation, presentation, or report is ready. Same intelligence. Deeper execution. Here’s how it works 👇

Fetch.ai

18,241 Aufrufe • vor 1 Monat

How do you know your robot actually succeeded? In v0.6.0, you ask a VLM 🤖 LeRobot ships a new unified reward models API with two zero-shot additions. Robometer is a general-purpose reward model trained on 1M+ trajectories across 21 robot embodiments: point it at any LeRobot dataset and it scores task progress and success straight from video and a language instruction, no task-specific training needed. TOPReward goes even further: no reward weights at all. It wraps an off-the-shelf VLM and reads the log-probability of the token "True" given the video and the instruction. Both come with labeling scripts that write per-frame progress curves into your dataset, ready for reward-aware behavior cloning. More on how they work:
0:55

How do you know your robot actually succeeded? In v0.6.0, you ask a VLM 🤖 LeRobot ships a new unified reward models API with two zero-shot additions. Robometer is a general-purpose reward model trained on 1M+ trajectories across 21 robot embodiments: point it at any LeRobot dataset and it scores task progress and success straight from video and a language instruction, no task-specific training needed. TOPReward goes even further: no reward weights at all. It wraps an off-the-shelf VLM and reads the log-probability of the token "True" given the video and the instruction. Both come with labeling scripts that write per-frame progress curves into your dataset, ready for reward-aware behavior cloning. More on how they work:

LeRobot

22,513 Aufrufe • vor 2 Monaten

Robot VR control. All the existing "VR" teleop systems I tried felt like shit so I put together my own. Instead of just piping a camera feed, I do a 3d reconstruction so you get depth perception and 6DoF head movement. This feels waay better than a flat image (no 3d, might as well skip the VR), or directly streaming per-eye video, which gets you depth but leaves your view locked in place (0DoF lmao). Not really sure why nobody else seems to do this.
0:50

Robot VR control. All the existing "VR" teleop systems I tried felt like shit so I put together my own. Instead of just piping a camera feed, I do a 3d reconstruction so you get depth perception and 6DoF head movement. This feels waay better than a flat image (no 3d, might as well skip the VR), or directly streaming per-eye video, which gets you depth but leaves your view locked in place (0DoF lmao). Not really sure why nobody else seems to do this.

FrostyFridge

31,630 Aufrufe • vor 5 Monaten

Logan Paul revealed his brand new app "Rip It" to Bendadonnn live on stream and showed exactly how it works, pull a real physical card from a digital pack and either ship it, vault it, or sell it back for 86% market value 😳🃏 "You can see your odds before you pull the card and you can see every card in the pack." "You can ship it to your door, hold it in the vault, or sell it back at 86% of fair market value." "Kristaps Porziņģis, I thought it was Jokić."
2:22

Logan Paul revealed his brand new app "Rip It" to Bendadonnn live on stream and showed exactly how it works, pull a real physical card from a digital pack and either ship it, vault it, or sell it back for 86% market value 😳🃏 "You can see your odds before you pull the card and you can see every card in the pack." "You can ship it to your door, hold it in the vault, or sell it back at 86% of fair market value." "Kristaps Porziņģis, I thought it was Jokić."

midaqih

315,172 Aufrufe • vor 3 Monaten

This is awesome. One thing I always tell younger guys when they're struggling with purpose or direction: Close your eyes and imagine what you want your 70th birthday party to look like. Really imagine it. In depth. In detail. Imagine everything. Imagine what you're wearing. Imagine the weather. Imagine the sights and smells. Imagine the food. Imagine the surroundings. Imagine the landscape. And most important: imagine WHO is there. That's how you begin to create your vision. Only once you've made it 100% real in your mind can you then start working backwards to make it real in your life. (And fwiw when I do this exercise, it looks pretty much like the video below)
0:50

This is awesome. One thing I always tell younger guys when they're struggling with purpose or direction: Close your eyes and imagine what you want your 70th birthday party to look like. Really imagine it. In depth. In detail. Imagine everything. Imagine what you're wearing. Imagine the weather. Imagine the sights and smells. Imagine the food. Imagine the surroundings. Imagine the landscape. And most important: imagine WHO is there. That's how you begin to create your vision. Only once you've made it 100% real in your mind can you then start working backwards to make it real in your life. (And fwiw when I do this exercise, it looks pretty much like the video below)

Harrison Schenck

1,299,317 Aufrufe • vor 2 Jahren

“Instant Placement” was announced during Connect last year, but I couldn’t find references to it in the Meta SDKs until recently. The actual code name is “EnvironmentRaycastManager”, and it is extremely helpful because it allows you to place objects on vertical or horizontal surfaces within your environment without requiring a full scene setup. 💡How does this work? This new manager utilizes the Depth API to provide raycasting functionality against the physical environment. 💡Does it impact performance? Yes, enabling this component adds an additional performance cost on top of using the Depth API. Therefore, consider enabling it only when you need raycasting functionality. #unity #xr
0:29

“Instant Placement” was announced during Connect last year, but I couldn’t find references to it in the Meta SDKs until recently. The actual code name is “EnvironmentRaycastManager”, and it is extremely helpful because it allows you to place objects on vertical or horizontal surfaces within your environment without requiring a full scene setup. 💡How does this work? This new manager utilizes the Depth API to provide raycasting functionality against the physical environment. 💡Does it impact performance? Yes, enabling this component adds an additional performance cost on top of using the Depth API. Therefore, consider enabling it only when you need raycasting functionality. #unity #xr

Dilmer 🔜 AWE

15,054 Aufrufe • vor 1 Jahr

Most robot datasets store one sentence per episode: the task. Real problems need more: a plan, a memory, room to answer questions and change course. We just shipped full language support in the LeRobot dataset format. Now lerobot-annotate points a vision-language model at each episode and writes the subtasks, plans, memory, and grounded Q&A itself - roughly $1.20 for 100 episodes. From there, recipes turn the annotated rows into policy-agnostic training messages, so the same dataset can teach planning, memory, tool calls, or low-level control, no dataset redesign required. Read how it all works on the blog:
0:20

Most robot datasets store one sentence per episode: the task. Real problems need more: a plan, a memory, room to answer questions and change course. We just shipped full language support in the LeRobot dataset format. Now lerobot-annotate points a vision-language model at each episode and writes the subtasks, plans, memory, and grounded Q&A itself - roughly $1.20 for 100 episodes. From there, recipes turn the annotated rows into policy-agnostic training messages, so the same dataset can teach planning, memory, tool calls, or low-level control, no dataset redesign required. Read how it all works on the blog:

LeRobot

13,451 Aufrufe • vor 2 Monaten

Holy shit IT WORKS!! One shotted with GPT6 Astra I now have a fork of UEVR where I can play any UE game within a volumetric window! This actually renders the game in 3D - meaning, two real game cameras, not some depth map bullshit. Everything looks right This is an alternative (and more comfortable IMO) way to play most 2D->VR hacked games as there’s no motion sickness, and no need to redesign the UI and interaction systems Needs a lot more polish before I publish it but OMG IT WORKS!
1:24

Holy shit IT WORKS!! One shotted with GPT6 Astra I now have a fork of UEVR where I can play any UE game within a volumetric window! This actually renders the game in 3D - meaning, two real game cameras, not some depth map bullshit. Everything looks right This is an alternative (and more comfortable IMO) way to play most 2D->VR hacked games as there’s no motion sickness, and no need to redesign the UI and interaction systems Needs a lot more polish before I publish it but OMG IT WORKS!

Azad Balabanian

943,509 Aufrufe • vor 22 Tagen

Behind the ball is how you get depth AND length out front. (Not one or the other) Here’s what it actually gives you: 1. You swing more connected and deeper. 2. The barrel enters earlier and stays in longer. You get both. 3. Depth behind what buys you length out front. Most guys are picking one or the other. 4. You sling the barrel through. You’re not shoving it or yanking it into the zone. 5. You actually use the ground. Behind the ball you turn. If you’re not using the ground, you’re probably sliding and pulling everything towards the ball. 6. Your bat speed becomes usable. Pushed and pulled bat speed still reads fast vs one pitch one speed. Only one of them shows up on the barrel, in the zone, on time vs pitching. — Cue for your next round: let the ball get deeper than usual, and turn behind a wall. You can scoot close to a net in front of you to get the feeling, too. Then vs moving ball. > Save this for when you need it > Send this to a hitter who needs it right now (And make sure you’re following)
1:31

Behind the ball is how you get depth AND length out front. (Not one or the other) Here’s what it actually gives you: 1. You swing more connected and deeper. 2. The barrel enters earlier and stays in longer. You get both. 3. Depth behind what buys you length out front. Most guys are picking one or the other. 4. You sling the barrel through. You’re not shoving it or yanking it into the zone. 5. You actually use the ground. Behind the ball you turn. If you’re not using the ground, you’re probably sliding and pulling everything towards the ball. 6. Your bat speed becomes usable. Pushed and pulled bat speed still reads fast vs one pitch one speed. Only one of them shows up on the barrel, in the zone, on time vs pitching. — Cue for your next round: let the ball get deeper than usual, and turn behind a wall. You can scoot close to a net in front of you to get the feeling, too. Then vs moving ball. > Save this for when you need it > Send this to a hitter who needs it right now (And make sure you’re following)

Trey Hannam

26,774 Aufrufe • vor 2 Monaten

The billionaires are consolidating their media. Now so are we. With corporate control over almost everything we read, watch, and listen to, independent voices are outgunned, censored, and divided. Six years ago, BT set out to change that. Now we’re taking the next big step, in the spirit of uniting and taking power back. We’re teaming up with Stake Giriş 2026 to bring you MORE original on-the-ground reporting, MORE breaking news, and MORE cutting in-depth analysis. ➡️ Join the NEW and EXPANDED BreakThrough News and get exclusives + updates straight to your inbox. 🔗 We don’t answer to Wall Street. We answer to you.
1:52

The billionaires are consolidating their media. Now so are we. With corporate control over almost everything we read, watch, and listen to, independent voices are outgunned, censored, and divided. Six years ago, BT set out to change that. Now we’re taking the next big step, in the spirit of uniting and taking power back. We’re teaming up with Stake Giriş 2026 to bring you MORE original on-the-ground reporting, MORE breaking news, and MORE cutting in-depth analysis. ➡️ Join the NEW and EXPANDED BreakThrough News and get exclusives + updates straight to your inbox. 🔗 We don’t answer to Wall Street. We answer to you.

BreakThrough News

71,193 Aufrufe • vor 3 Monaten

We put 1000s of hours into building open-source camera glasses And now we're giving away everything we learned for free. Checkout this in-depth teardown of the Mentra Live smart glasses. As an OS company, our goal is to spread our OS. That goal encourages us to do crazy things, like build the world's best AI glasses for business and then show you how we made them... all in the name of spreading the OS. It also convinced us to make it 100% open source, so adoption is faster and easier. We're at risk of our competitors easily copypasting our code now... all in the name of spreading our OS. Now we've built the glasses for you, and we've taken the risk, and we've released an open SDK. Now your AI apps and agents can enter the real world on smart glasses. So now, what will you build?
13:19

We put 1000s of hours into building open-source camera glasses And now we're giving away everything we learned for free. Checkout this in-depth teardown of the Mentra Live smart glasses. As an OS company, our goal is to spread our OS. That goal encourages us to do crazy things, like build the world's best AI glasses for business and then show you how we made them... all in the name of spreading the OS. It also convinced us to make it 100% open source, so adoption is faster and easier. We're at risk of our competitors easily copypasting our code now... all in the name of spreading our OS. Now we've built the glasses for you, and we've taken the risk, and we've released an open SDK. Now your AI apps and agents can enter the real world on smart glasses. So now, what will you build?

cayden 凯登

20,584 Aufrufe • vor 4 Monaten

No Three.js. No fake web-based depth-of-field or focal-length emulation. Customize in web, render in Blender. While everyone is vibe-coding apps for flat pattern generators, or at most Three.js-based 3D scenes, we’re already one step ahead. We built an app that lets you use a web UI to control an actual Blender 3D render. Eevee engine is used for streaming quick previews back into the app, while Cycles renders the final image for better material quality and lighting. So simple to use that you can even give it to your client. Of course, built with Toolcraft.
1:15

No Three.js. No fake web-based depth-of-field or focal-length emulation. Customize in web, render in Blender. While everyone is vibe-coding apps for flat pattern generators, or at most Three.js-based 3D scenes, we’re already one step ahead. We built an app that lets you use a web UI to control an actual Blender 3D render. Eevee engine is used for streaming quick previews back into the app, while Cycles renders the final image for better material quality and lighting. So simple to use that you can even give it to your client. Of course, built with Toolcraft.

Alex Barashkov

76,542 Aufrufe • vor 2 Monaten

The BEDLAM2.0 dataset (B2) is here, just in time to train your 3D human pose and shape estimation methods for CVPR. B2 goes beyond BEDLAM (B1) to include widely varied and natural camera motions and fields of view, more diverse body shapes, strand-based hair, more garments, shoes, more body motions, and more 3D scenes. Compared with B1, training on B2 produces more accurate 3D human pose, resulting in SOTA accuracy, particularly for estimates in world coordinates. B2 lets you jointly train camera motion and human motion regressors, and we also provide depth maps. Check out data, code, dataset statistics, and much more. BEDLAM2.0 will appear in the 2025 NeurIPS Datasets and Benchmarks Track. Joint work with Joachim Tesch, Giorgio Becherini, Prerana Achar, Anastasios Yiannakidis, Muhammed Kocabas, Priyanka Patel.
0:30

The BEDLAM2.0 dataset (B2) is here, just in time to train your 3D human pose and shape estimation methods for CVPR. B2 goes beyond BEDLAM (B1) to include widely varied and natural camera motions and fields of view, more diverse body shapes, strand-based hair, more garments, shoes, more body motions, and more 3D scenes. Compared with B1, training on B2 produces more accurate 3D human pose, resulting in SOTA accuracy, particularly for estimates in world coordinates. B2 lets you jointly train camera motion and human motion regressors, and we also provide depth maps. Check out data, code, dataset statistics, and much more. BEDLAM2.0 will appear in the 2025 NeurIPS Datasets and Benchmarks Track. Joint work with Joachim Tesch, Giorgio Becherini, Prerana Achar, Anastasios Yiannakidis, Muhammed Kocabas, Priyanka Patel.

Michael Black

27,686 Aufrufe • vor 11 Monaten