Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Can robots learn without training❓ [𝗜𝘁'𝘀 𝗼𝗽𝗲𝗻 𝘀𝗼𝘂𝗿𝗰𝗲𝗱 ⬇ ] Teaching robots to do complex tasks WITHOUT spending hours training them. Sounds cool, right? That's exactly what DIAL-MPC does! The first training-free method for whole-body torque control using full-order dynamics: ✅ Instantly checks if a robot's moves are right... show more

Ilir Aliu

55,119 subscribers

71,540 görüntüleme • 1 yıl önce •via X (Twitter)

Oyun Eğitim Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

🚀Thrilled to share what we’ve been building at TRI over the past several months: our first Large Behavior Models (LBMs) are here! I’m proud to have been a core contributor to the multi-task policy learning and post-training efforts. At TRI, we’ve been researching how LBMs can help robots learn faster, better, and more efficiently. The key takeaways: ✅ We built an evaluation pipeline to benchmark LBM performance with real 𝐬𝐭𝐚𝐭𝐢𝐬𝐭𝐢𝐜𝐚𝐥 𝐜𝐨𝐧𝐟𝐢𝐝𝐞𝐧𝐜𝐞 ✅ Pre-training on hundreds of tasks makes models more robust—plus, we can teach new, complex tasks with 80% 𝐥𝐞𝐬𝐬 𝐝𝐚𝐭𝐚 ✅ The bigger and more diverse the pre-training, the better the results Check out our overview video, webpage and paper for more details: ✨ 🌎 📄 We hope this work helps move the field of robotics forward!

🚀Thrilled to share what we’ve been building at TRI over the past several months: our first Large Behavior Models (LBMs) are here! I’m proud to have been a core contributor to the multi-task policy learning and post-training efforts. At TRI, we’ve been researching how LBMs can help robots learn faster, better, and more efficiently. The key takeaways: ✅ We built an evaluation pipeline to benchmark LBM performance with real 𝐬𝐭𝐚𝐭𝐢𝐬𝐭𝐢𝐜𝐚𝐥 𝐜𝐨𝐧𝐟𝐢𝐝𝐞𝐧𝐜𝐞 ✅ Pre-training on hundreds of tasks makes models more robust—plus, we can teach new, complex tasks with 80% 𝐥𝐞𝐬𝐬 𝐝𝐚𝐭𝐚 ✅ The bigger and more diverse the pre-training, the better the results Check out our overview video, webpage and paper for more details: ✨ 🌎 📄 We hope this work helps move the field of robotics forward!

Zubair Irshad

20,518 görüntüleme • 1 yıl önce

If you’ve been ignoring Axis Robotics because it looks like another random points farm, read this. In the last 48 hours, Unitree said robotics is approaching its "ChatGPT moment", while the chairman of ACE Robotics believes it could happen by the end of 2027. But for robots to reach that level, they need a crazy amount of training data. That’s exactly what Axis is building. You control simulated robots directly from your browser, complete simple tasks and earn points. Those movements also help create data that can be used to train real robots. And this isn’t some tiny experiment anymore: - $12M raised - 123K+ contributors - 3M+ robot trajectories - Community data already used to train a real robot So yeah, we’re basically farming a potential airdrop while teaching our future robot servants how to work 😂 If you haven’t started yet, it’s completely free. You only need a tiny amount of gas on Base to sign your completed tasks. ✅ Start farming Axis points: Important: Sign every completed task from the History page, otherwise you won’t receive the points.

If you’ve been ignoring Axis Robotics because it looks like another random points farm, read this. In the last 48 hours, Unitree said robotics is approaching its "ChatGPT moment", while the chairman of ACE Robotics believes it could happen by the end of 2027. But for robots to reach that level, they need a crazy amount of training data. That’s exactly what Axis is building. You control simulated robots directly from your browser, complete simple tasks and earn points. Those movements also help create data that can be used to train real robots. And this isn’t some tiny experiment anymore: - $12M raised - 123K+ contributors - 3M+ robot trajectories - Community data already used to train a real robot So yeah, we’re basically farming a potential airdrop while teaching our future robot servants how to work 😂 If you haven’t started yet, it’s completely free. You only need a tiny amount of gas on Base to sign your completed tasks. ✅ Start farming Axis points: Important: Sign every completed task from the History page, otherwise you won’t receive the points.

Pranjal Bora 🧭

31,455 görüntüleme • 1 ay önce

Robotics keeps hitting the same wall. Single task RL works, but... it does not scale to hundreds of tasks or new embodiments. This new paper looks like a real step toward fixing that. The team introduces MMBench, a benchmark with 200 tasks across many domains and robots, and Newt, a language conditioned world model trained online across all 200 tasks at once. The simple idea behind Newt: The model learns from demos to get the right priors It trains across many tasks through online interaction It uses language to ground the goal It adapts fast when a new task shows up What stood out to me: ✅ One model trained on 200 tasks at the same time ✅ Language conditioned control for both states and RGB ✅ Better data efficiency than strong baselines ✅ Strong open loop control ✅ Fast adaptation to new tasks and embodiments ✅ Full release of 200 checkpoints, 4000 demos, code, and benchmark This is a good push toward general control instead of one model per task. If you want the full paper: Project page: —- Weekly robotics and AI insights. Subscribe free:

Robotics keeps hitting the same wall. Single task RL works, but... it does not scale to hundreds of tasks or new embodiments. This new paper looks like a real step toward fixing that. The team introduces MMBench, a benchmark with 200 tasks across many domains and robots, and Newt, a language conditioned world model trained online across all 200 tasks at once. The simple idea behind Newt: The model learns from demos to get the right priors It trains across many tasks through online interaction It uses language to ground the goal It adapts fast when a new task shows up What stood out to me: ✅ One model trained on 200 tasks at the same time ✅ Language conditioned control for both states and RGB ✅ Better data efficiency than strong baselines ✅ Strong open loop control ✅ Fast adaptation to new tasks and embodiments ✅ Full release of 200 checkpoints, 4000 demos, code, and benchmark This is a good push toward general control instead of one model per task. If you want the full paper: Project page: —- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

70,090 görüntüleme • 10 ay önce

The next breakthrough for robot VLMs isn’t a bigger model 🧠 it’s memory. RAL accepted ✅ We show that a robot can ground a VLM using its own real-world experience, not extra training. EXPTEACH/PRAGMABOT = self-generated memory + closed-loop reflection: Fail → diagnose → replan → succeed → store experience Next time → retrieve the right experience → succeed in one shot Results on real robot tasks: 36% → 84% with reflection (STM) 22% → 80% single-trial success with long-term memory (LTM+RAG) And yes, we saw tool use emerge without explicitly asking for it. If you’re fighting brittle “VLM plans” on hardware, this is for you. #AI #Robotics ANYbotics

The next breakthrough for robot VLMs isn’t a bigger model 🧠 it’s memory. RAL accepted ✅ We show that a robot can ground a VLM using its own real-world experience, not extra training. EXPTEACH/PRAGMABOT = self-generated memory + closed-loop reflection: Fail → diagnose → replan → succeed → store experience Next time → retrieve the right experience → succeed in one shot Results on real robot tasks: 36% → 84% with reflection (STM) 22% → 80% single-trial success with long-term memory (LTM+RAG) And yes, we saw tool use emerge without explicitly asking for it. If you’re fighting brittle “VLM plans” on hardware, this is for you. #AI #Robotics ANYbotics

Haitham Bou Ammar

12,206 görüntüleme • 8 ay önce

30 minutes of video. Robot learns the task. Open-source, end-to-end. An open-source framework for training robot policies from only 30 minutes of human egocentric videos captured via Meta Aria glasses: Achieving zero-shot transfer to robots without any robot data collection. The method relies on Interaction-Centric Tokens that encode hand-object spatial relationships invariant to embodiment and viewpoint, supplemented by auxiliary objectives like object motion prediction and latent consistency to extract richer supervision signals from the same data. HumanEgo demonstrates strong cross-embodiment, cross-environment performance on bimanual tasks, outperforming baselines like ACT and teleop data while being trainable on a single RTX 4090 GPU. Thanks for sharing, Zhi (Leo) Wang. 📌 Website: Paper: Code: Video: ——- Weekly robotics and AI insights. Subscribe free:

30 minutes of video. Robot learns the task. Open-source, end-to-end. An open-source framework for training robot policies from only 30 minutes of human egocentric videos captured via Meta Aria glasses: Achieving zero-shot transfer to robots without any robot data collection. The method relies on Interaction-Centric Tokens that encode hand-object spatial relationships invariant to embodiment and viewpoint, supplemented by auxiliary objectives like object motion prediction and latent consistency to extract richer supervision signals from the same data. HumanEgo demonstrates strong cross-embodiment, cross-environment performance on bimanual tasks, outperforming baselines like ACT and teleop data while being trainable on a single RTX 4090 GPU. Thanks for sharing, Zhi (Leo) Wang. 📌 Website: Paper: Code: Video: ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

17,077 görüntüleme • 4 ay önce

Xiaomi Humanoid Robot New Factory Training Since Q1, Xiaomi’s humanoid robots have been training on dual-side operations at the self-tapping screw workstation in its auto factory, achieving a 98% success rate. Today, Xiaomi shared that the CyberOne Gen2 autonomously performed two repetitive tasks in the final assembly logistics area: sorting center console side panels and folding material boxes --both reaching 90% success rates. They deployed both bipedal and wheeled models, working together with material transport robots as part of the broader intelligent automation system. I noticed the wheeled CyberOne was clearly more efficient and stable on the panel sorting task. In the first video, you can also see some noticeable jitter when the bipedal robot stops moving (though this should be relatively easy to fix). Xiaomi also candidly pointed out a current limitation in dexterity: when folding the second side of the material box, the robot still needs to adjust its orientation first to face the buckle, whereas skilled workers can pull the ring directly by feel without any micro-adjustments. Real-world scenarios are indeed the best benchmark.

Xiaomi Humanoid Robot New Factory Training Since Q1, Xiaomi’s humanoid robots have been training on dual-side operations at the self-tapping screw workstation in its auto factory, achieving a 98% success rate. Today, Xiaomi shared that the CyberOne Gen2 autonomously performed two repetitive tasks in the final assembly logistics area: sorting center console side panels and folding material boxes --both reaching 90% success rates. They deployed both bipedal and wheeled models, working together with material transport robots as part of the broader intelligent automation system. I noticed the wheeled CyberOne was clearly more efficient and stable on the panel sorting task. In the first video, you can also see some noticeable jitter when the bipedal robot stops moving (though this should be relatively easy to fix). Xiaomi also candidly pointed out a current limitation in dexterity: when folding the second side of the material box, the robot still needs to adjust its orientation first to face the buckle, whereas skilled workers can pull the ring directly by feel without any micro-adjustments. Real-world scenarios are indeed the best benchmark.

CyberRobo

27,619 görüntüleme • 2 ay önce

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.

Rohan Paul

37,121 görüntüleme • 7 ay önce

MAKING AXIS TASKS FEEL EASY 👀🤖 I made a quick video showing how to complete an Axis Robotics task step by step. The task gives you a clear goal, steps, controls and camera views to guide you. In this one, the goal is simple, Open the trash bin, pick up the onion, and place it inside. 🧅 Just follow the instructions, control the robot arm and complete the task. What I like is that even beginners can understand what they need to do. But the interesting part is what happens with these interactions after we complete them. Each completed task can contribute valuable robot training data, helping Physical Ai systems learn how to handle different objects and environments. Small tasks from the community can become useful data for teaching robots how to interact with the real world. That’s what makes Axis interesting to me.

MAKING AXIS TASKS FEEL EASY 👀🤖 I made a quick video showing how to complete an Axis Robotics task step by step. The task gives you a clear goal, steps, controls and camera views to guide you. In this one, the goal is simple, Open the trash bin, pick up the onion, and place it inside. 🧅 Just follow the instructions, control the robot arm and complete the task. What I like is that even beginners can understand what they need to do. But the interesting part is what happens with these interactions after we complete them. Each completed task can contribute valuable robot training data, helping Physical Ai systems learn how to handle different objects and environments. Small tasks from the community can become useful data for teaching robots how to interact with the real world. That’s what makes Axis interesting to me.

SufianXFN

14,419 görüntüleme • 1 ay önce

First fully ML-framework-free 3D Gaussian Splatting implementation in LichtFeld Studio. I’ve completed the migration of the full training pipeline to a custom CUDA-based tensor library. No PyTorch, no LibTorch, no autograd. Every gradient is implemented by hand, either through CUDA kernels or minimal abstractions on top. This makes it the first full training setup for 3D Gaussian Splatting with zero dependencies on existing ML frameworks. It’s not just about independence, it's about control! We now manage every byte of GPU memory, which opens the door to tighter optimization and finer performance tuning. The framework footprint is minimal, without pulling in gigabytes of ML runtime code that was never designed for real-time or graphics-driven applications. A few modules, such as the metrics and 3DGUT interfaces, are still being ported, and some operations are temporarily naïve, so performance is not yet on par with master. But this refactor lays the groundwork for: - A fully self-contained binary - Fine-grained memory optimization - Easier experimentation without the weight of an ML stack We’re getting close.

First fully ML-framework-free 3D Gaussian Splatting implementation in LichtFeld Studio. I’ve completed the migration of the full training pipeline to a custom CUDA-based tensor library. No PyTorch, no LibTorch, no autograd. Every gradient is implemented by hand, either through CUDA kernels or minimal abstractions on top. This makes it the first full training setup for 3D Gaussian Splatting with zero dependencies on existing ML frameworks. It’s not just about independence, it's about control! We now manage every byte of GPU memory, which opens the door to tighter optimization and finer performance tuning. The framework footprint is minimal, without pulling in gigabytes of ML runtime code that was never designed for real-time or graphics-driven applications. A few modules, such as the metrics and 3DGUT interfaces, are still being ported, and some operations are temporarily naïve, so performance is not yet on par with master. But this refactor lays the groundwork for: - A fully self-contained binary - Fine-grained memory optimization - Easier experimentation without the weight of an ML stack We’re getting close.

MrNeRF

50,571 görüntüleme • 11 ay önce

Imagine you go to a store and you want to buy candy. The shopkeeper knows you're a real kid because they can see you standing right there. Now imagine you send a robot to buy candy for you. The shopkeeper looks at the robot and thinks: wait, who sent this? Is this robot allowed to buy candy? What if someone else's robot pretends to be yours and steals your candy money? That's basically what's happening with AI right now. Companies like Visa let people buy things all over the world. But now, smart computer robots (AI agents) want to buy things too. Shop around, compare prices, even pay for stuff. Visa looked at this and said: nope, not yet. Because they have no way to check if the robot is real, who it belongs to, or if it's allowed to spend that money. The problem is that all the rules we have for checking identity - showing your ID, scanning your face, typing your password - only work for humans. Robots can't do any of that. Worse, bad robots can actually copy and fake human identities really well. So Evin McMullen evin, Billions Network co-founder and CEO, says we need a new kind of ID system. One where you can prove something is true without showing all your private stuff. Like proving you're tall enough for a ride without telling anyone your exact height. That's called zero-knowledge proof. And for the robots specifically, we need something called KYA - Know Your Agent. It's like giving every robot its own ID card that says: this is who I am, this is what I'm allowed to do, and this is the human responsible for me. Until we build that, the robot economy can't really get going. Here is Evin’s Thought Leader article at Silicon Valleys Journal

Imagine you go to a store and you want to buy candy. The shopkeeper knows you're a real kid because they can see you standing right there. Now imagine you send a robot to buy candy for you. The shopkeeper looks at the robot and thinks: wait, who sent this? Is this robot allowed to buy candy? What if someone else's robot pretends to be yours and steals your candy money? That's basically what's happening with AI right now. Companies like Visa let people buy things all over the world. But now, smart computer robots (AI agents) want to buy things too. Shop around, compare prices, even pay for stuff. Visa looked at this and said: nope, not yet. Because they have no way to check if the robot is real, who it belongs to, or if it's allowed to spend that money. The problem is that all the rules we have for checking identity - showing your ID, scanning your face, typing your password - only work for humans. Robots can't do any of that. Worse, bad robots can actually copy and fake human identities really well. So Evin McMullen evin, Billions Network co-founder and CEO, says we need a new kind of ID system. One where you can prove something is true without showing all your private stuff. Like proving you're tall enough for a ride without telling anyone your exact height. That's called zero-knowledge proof. And for the robots specifically, we need something called KYA - Know Your Agent. It's like giving every robot its own ID card that says: this is who I am, this is what I'm allowed to do, and this is the human responsible for me. Until we build that, the robot economy can't really get going. Here is Evin’s Thought Leader article at Silicon Valleys Journal

Billions Network

21,775 görüntüleme • 8 ay önce

A simple MPC controller for path tracking implemented in python. Few resources actually teach it without assuming a PhD first. This open-source project does exactly that: An iterative MPC tracking controller (built with CVXPY and MuJoCo) that shows how to make a robot (or autonomous vehicle) follow a path and avoid obstacles in real time. → Autonomous logistics, robot navigation, vehicle control; there's a path-following algorithm behind all of it. → Most implementations are black boxes. This one shows you what's actually happening. → It uses convex optimization (no non-linear solver), and still handles real vehicle physics through iterative linearization. What makes it practical is that you can run it with the mushr car model in MuJoCo, or headless without any physics simulation if you just want to understand the logic. If you're making the jump from classical control to real-time optimisation, this is a very good starting point. 📌 Repository: ——- Weekly robotics and AI insights. Subscribe free:

A simple MPC controller for path tracking implemented in python. Few resources actually teach it without assuming a PhD first. This open-source project does exactly that: An iterative MPC tracking controller (built with CVXPY and MuJoCo) that shows how to make a robot (or autonomous vehicle) follow a path and avoid obstacles in real time. → Autonomous logistics, robot navigation, vehicle control; there's a path-following algorithm behind all of it. → Most implementations are black boxes. This one shows you what's actually happening. → It uses convex optimization (no non-linear solver), and still handles real vehicle physics through iterative linearization. What makes it practical is that you can run it with the mushr car model in MuJoCo, or headless without any physics simulation if you just want to understand the logic. If you're making the jump from classical control to real-time optimisation, this is a very good starting point. 📌 Repository: ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

16,064 görüntüleme • 28 gün önce

My guy is definitely going through something 😭 Seriously though, jokes aside, who is actually working on their training regularly? Are you lifting weights, training your cardio, training at the shooting range, working through your preparedness gear make sure you know how to use everything? As a husband and father, being prepared is part of the responsibility of protecting your family. You cannot control when a storm hits, the power goes out, roads become blocked, or help is delayed. What you can control is whether you are capable when your family looks to you. Training regularly builds more than muscle. It builds discipline, endurance, confidence, and the ability to function when you are tired, stressed, or afraid. A father should be able to carry his children, help an injured family member, work for hours if necessary, and stay calm when everyone else is looking for direction. Preparedness means having food, water, medical supplies, backup power, tools, and a plan before they are needed. It also means knowing how to use them. Gear without knowledge is just equipment sitting on a shelf. The goal is not paranoia or living in fear. It is becoming difficult to helplessly overwhelm. Your family should know that when something goes wrong, Dad has prepared, Dad has trained, and Dad is ready to take care of the people entrusted to him. What are you going to start working on?

My guy is definitely going through something 😭 Seriously though, jokes aside, who is actually working on their training regularly? Are you lifting weights, training your cardio, training at the shooting range, working through your preparedness gear make sure you know how to use everything? As a husband and father, being prepared is part of the responsibility of protecting your family. You cannot control when a storm hits, the power goes out, roads become blocked, or help is delayed. What you can control is whether you are capable when your family looks to you. Training regularly builds more than muscle. It builds discipline, endurance, confidence, and the ability to function when you are tired, stressed, or afraid. A father should be able to carry his children, help an injured family member, work for hours if necessary, and stay calm when everyone else is looking for direction. Preparedness means having food, water, medical supplies, backup power, tools, and a plan before they are needed. It also means knowing how to use them. Gear without knowledge is just equipment sitting on a shelf. The goal is not paranoia or living in fear. It is becoming difficult to helplessly overwhelm. Your family should know that when something goes wrong, Dad has prepared, Dad has trained, and Dad is ready to take care of the people entrusted to him. What are you going to start working on?

Giga Based Dad

13,238 görüntüleme • 16 gün önce

This is Marcus. Served in Afghanistan. Three deployments. Night terrors every single night for eight years. Wakes up drenched in sweat. Sometimes screaming. His VA counselor recommended a trained service dog. Wait list was four years. Cost was $25,000. Marcus couldn’t wait that long. He adopted Rocky from a rescue last October. Four-year-old German Shepherd. Rejected from police training for being “too gentle with people.” First night Rocky came home, Marcus had a nightmare around 2 AM. Rocky jumped on the bed without any training. Laid his entire body across Marcus’s chest. 92 pounds of pressure. Marcus woke up. Didn’t scream this time. Just cried quietly. Rocky stayed right there until Marcus fell back asleep. Now Rocky does this automatically every single night. Marcus showed the night vision footage to his counselor. She said: “That’s exactly the deep pressure therapy we teach service dogs. Rocky’s doing it instinctively. That’s a $25,000 behavior he was born knowing.” Rocky failed police training. But he’s saving Marcus’s life every night. Sometimes the ones who don’t make the cut are exactly who we need.

This is Marcus. Served in Afghanistan. Three deployments. Night terrors every single night for eight years. Wakes up drenched in sweat. Sometimes screaming. His VA counselor recommended a trained service dog. Wait list was four years. Cost was $25,000. Marcus couldn’t wait that long. He adopted Rocky from a rescue last October. Four-year-old German Shepherd. Rejected from police training for being “too gentle with people.” First night Rocky came home, Marcus had a nightmare around 2 AM. Rocky jumped on the bed without any training. Laid his entire body across Marcus’s chest. 92 pounds of pressure. Marcus woke up. Didn’t scream this time. Just cried quietly. Rocky stayed right there until Marcus fell back asleep. Now Rocky does this automatically every single night. Marcus showed the night vision footage to his counselor. She said: “That’s exactly the deep pressure therapy we teach service dogs. Rocky’s doing it instinctively. That’s a $25,000 behavior he was born knowing.” Rocky failed police training. But he’s saving Marcus’s life every night. Sometimes the ones who don’t make the cut are exactly who we need.

Gabriele Corno

1,022,260 görüntüleme • 2 ay önce

The term "continual learning" has become overloaded if you see it as an ML problem. One classic thread is about memorization: regularization-based continual learning methods, such as EWC, MAS, and SI, estimate which parameters mattered for previous tasks and resist changing them too much. One modern thread is about adaptation: test-time training and inference-time learning methods, such as TTT, adapt part of the model on the incoming test stream before making predictions. These are sometimes discussed as separate threads. But in modern scalable architectures, I think they are better seen as complementary constraints: a model that learns quickly at test time also benefits from a mechanism for deciding what not to forget. In our #ECCV2026 paper, we study this in large-scale 4D reconstruction: how to build fast spatial memory that can adapt over long observation streams while reducing collapse and forgetting. Instead of using fully plastic test-time updates, we stabilize fast-weight adaptation with an elastic prior that balances adaptation and memory. Key ideas: - Elastic Test-Time Training: Fisher-weighted consolidation for fast-weight updates - EMA anchor weights that provide a moving reference for stability - Chunk-by-chunk inference for long 3D/4D observation streams We show that this scales across large 3D/4D pretraining settings, including both LRM-style and LVSM-style models, and improves reconstruction across benchmarks including Stereo4D, NVIDIA, and DL3DV-140. We release model checkpoints across different design choices: resolution, post-training curriculum, and whether the model uses an explicit 4DGS intermediate representation. - Homepage: - Paper: - Code: - Models: This work is co-led with Xueyang Yu, contributed by Haoyu Zhen Yuncong Yang, and advised by Michigan SLED Lab Chuang Gan.

The term "continual learning" has become overloaded if you see it as an ML problem. One classic thread is about memorization: regularization-based continual learning methods, such as EWC, MAS, and SI, estimate which parameters mattered for previous tasks and resist changing them too much. One modern thread is about adaptation: test-time training and inference-time learning methods, such as TTT, adapt part of the model on the incoming test stream before making predictions. These are sometimes discussed as separate threads. But in modern scalable architectures, I think they are better seen as complementary constraints: a model that learns quickly at test time also benefits from a mechanism for deciding what not to forget. In our #ECCV2026 paper, we study this in large-scale 4D reconstruction: how to build fast spatial memory that can adapt over long observation streams while reducing collapse and forgetting. Instead of using fully plastic test-time updates, we stabilize fast-weight adaptation with an elastic prior that balances adaptation and memory. Key ideas: - Elastic Test-Time Training: Fisher-weighted consolidation for fast-weight updates - EMA anchor weights that provide a moving reference for stability - Chunk-by-chunk inference for long 3D/4D observation streams We show that this scales across large 3D/4D pretraining settings, including both LRM-style and LVSM-style models, and improves reconstruction across benchmarks including Stereo4D, NVIDIA, and DL3DV-140. We release model checkpoints across different design choices: resolution, post-training curriculum, and whether the model uses an explicit 4DGS intermediate representation. - Homepage: - Paper: - Code: - Models: This work is co-led with Xueyang Yu, contributed by Haoyu Zhen Yuncong Yang, and advised by Michigan SLED Lab Chuang Gan.

Martin Ziqiao Ma

34,012 görüntüleme • 3 ay önce

Massive update for AI Engineers! Training diffusion models just got a lot easier. dLLM is an open-source library that does for diffusion models what Hugging Face did for transformers. Here's why this matters: Traditional autoregressive models generate text left-to-right, one token at a time. Diffusion models work differently - they refine the entire sequence iteratively, giving you better control over generation quality and more flexible editing capabilities. The problem? Building and training these models required stitching together scattered tools and reimplementing research papers from scratch. Most of the tooling has been scattered and is hard to reproduce. dLLM changes this: It unifies everything you need to train, evaluate, and deploy diffusion language models: ↳ Scalable training with LoRA, DeepSpeed, and FSDP support ↳ Unified evaluation that abstracts away inference complexity ↳ Ready-to-use recipes for pretraining, finetuning, and evaluation The library includes implementations of models like LLaDA and Dream, plus training algorithms like Edit Flows that enable insertion, deletion, and substitution operations. The team just released ModernBERT-Chat models showing you can turn BERT into lightweight chatbots through masked instruction tuning. This is practical and worth exploring. The setup is straightforward. Run locally with Accelerate or scale to multi-node clusters with Slurm. If you're working with language models and want to explore diffusion-based approaches without rebuilding infrastructure, dLLM gives you a production-ready starting point. Link to the repo in the next tweet.

Massive update for AI Engineers! Training diffusion models just got a lot easier. dLLM is an open-source library that does for diffusion models what Hugging Face did for transformers. Here's why this matters: Traditional autoregressive models generate text left-to-right, one token at a time. Diffusion models work differently - they refine the entire sequence iteratively, giving you better control over generation quality and more flexible editing capabilities. The problem? Building and training these models required stitching together scattered tools and reimplementing research papers from scratch. Most of the tooling has been scattered and is hard to reproduce. dLLM changes this: It unifies everything you need to train, evaluate, and deploy diffusion language models: ↳ Scalable training with LoRA, DeepSpeed, and FSDP support ↳ Unified evaluation that abstracts away inference complexity ↳ Ready-to-use recipes for pretraining, finetuning, and evaluation The library includes implementations of models like LLaDA and Dream, plus training algorithms like Edit Flows that enable insertion, deletion, and substitution operations. The team just released ModernBERT-Chat models showing you can turn BERT into lightweight chatbots through masked instruction tuning. This is practical and worth exploring. The setup is straightforward. Run locally with Accelerate or scale to multi-node clusters with Slurm. If you're working with language models and want to explore diffusion-based approaches without rebuilding infrastructure, dLLM gives you a production-ready starting point. Link to the repo in the next tweet.

Akshay 🚀

64,011 görüntüleme • 10 ay önce

$40 hours of human work. That’s what this humanoid could save every month! A construction company is already testing a Unitree G1 on a real job site, using the robot for site inspections, 360° imaging, data collection and progress monitoring. The robot starts at around $13,500, while the company says its deployment can save roughly 40 hours of work every month. That adds up to around 480 hours a year from a machine that costs a fraction of traditional industrial equipment. The interesting part is that this isn't about replacing an entire construction worker. It's about removing hundreds of repetitive hours from the workflow, including walking inspection routes, documenting progress and collecting information across the site. Humans can then spend their time on decisions and tasks that actually require them. This is where humanoid robots become economically interesting. Construction sites are already designed around human movement, so a robot with two arms, two legs and a human-sized body can potentially work in the same spaces without rebuilding the entire environment. Every additional task it learns turns those same hardware costs into more productive hours. And the economics get even more interesting as prices fall and production scales. A robot that saves 480 hours per year doesn't need to be perfect or replace a full-time employee to justify its existence. It just needs to reliably take over the boring, repetitive work that companies are already paying humans to do. 480 hours saved. Thousands of dollars in hardware. One construction site. This is how humanoids will enter the workforce, not by replacing everyone overnight, but by quietly taking over the hours nobody wants to spend.$

40 hours of human work. That’s what this humanoid could save every month! A construction company is already testing a Unitree G1 on a real job site, using the robot for site inspections, 360° imaging, data collection and progress monitoring. The robot starts at around $13,500, while the company says its deployment can save roughly 40 hours of work every month. That adds up to around 480 hours a year from a machine that costs a fraction of traditional industrial equipment. The interesting part is that this isn't about replacing an entire construction worker. It's about removing hundreds of repetitive hours from the workflow, including walking inspection routes, documenting progress and collecting information across the site. Humans can then spend their time on decisions and tasks that actually require them. This is where humanoid robots become economically interesting. Construction sites are already designed around human movement, so a robot with two arms, two legs and a human-sized body can potentially work in the same spaces without rebuilding the entire environment. Every additional task it learns turns those same hardware costs into more productive hours. And the economics get even more interesting as prices fall and production scales. A robot that saves 480 hours per year doesn't need to be perfect or replace a full-time employee to justify its existence. It just needs to reliably take over the boring, repetitive work that companies are already paying humans to do. 480 hours saved. Thousands of dollars in hardware. One construction site. This is how humanoids will enter the workforce, not by replacing everyone overnight, but by quietly taking over the hours nobody wants to spend.

Future Memo

18,925 görüntüleme • 26 gün önce

𝗗𝗼𝗻'𝘁 𝗳𝗶𝗻𝗲-𝘁𝘂𝗻𝗲 𝗿𝗼𝗯𝗼𝘁 𝗳𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 𝗺𝗼𝗱𝗲𝗹𝘀. 𝗦𝘁𝗲𝗲𝗿 𝘁𝗵𝗲𝗺 𝘄𝗶𝘁𝗵 𝗵𝘂𝗺𝗮𝗻 𝗰𝗼𝗿𝗿𝗲𝗰𝘁𝗶𝗼𝗻𝘀 𝗶𝗻𝘀𝘁𝗲𝗮𝗱, 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝘁𝗵𝗲 𝗯𝗮𝘀𝗲 𝗽𝗼𝗹𝗶𝗰𝘆 Modern VLAs and world-action models can perform impressive manipulation skills, but adapting them reliably to new robots and tasks remains challenging. A natural solution is DAgger-style online imitation learning: deploy the robot, collect human corrections, and update the policy. Yet foundation models are fragile in the low-data regime, fine-tuning on a handful of interventions can improve one behavior while degrading others. Online post-training or reinforcement learning can require costly data collection and exploration, making real-world learning expensive and potentially unsafe. In our new paper, 𝗙𝗹𝗼𝘄𝗗𝗔𝗴𝗴𝗲𝗿, we take a different approach: 𝗜𝗻𝘀𝘁𝗲𝗮𝗱 𝗼𝗳 𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝘁𝗵𝗲 𝗳𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 𝗺𝗼𝗱𝗲𝗹, 𝘄𝗲 𝗹𝗲𝗮𝗿𝗻 𝗵𝗼𝘄 𝘁𝗼 𝘀𝘁𝗲𝗲𝗿 𝗶𝘁 𝗳𝗿𝗼𝗺 𝗵𝘂𝗺𝗮𝗻 𝗰𝗼𝗿𝗿𝗲𝗰𝘁𝗶𝗼𝗻𝘀. The key idea is 𝗮𝗰𝘁𝗶𝗼𝗻 𝗶𝗻𝘃𝗲𝗿𝘀𝗶𝗼𝗻: we map human corrective actions back into the latent noise space of the frozen generative policy. These latent targets train a lightweight controller that adapts the robot while preserving the original model's capabilities. Across simulation and real robots, FlowDAgger: 📈 Learns from only 5–20 human intervention episodes 🏆 Outperforms supervised fine-tuning and latent-space reinforcement learning 🤖 Works across VLAs, diffusion policies, and world-action models ✔️ Provides reliable improvements without modifying the pretrained policy We believe this offers a practical path toward making robot foundation models improve during deployment, learning from the way humans naturally teach: through corrections. 📄 Paper: 🌐 Project: 💻 Code: This project was led by my amazing colleague Michael Murray with help from Daphne Chen, Simran Bagaria, Dean Fortier, Tess Hellebrekers, Harshavardhan Reddy Gajarla, Galen Mullins and Andrey Kolobov at Microsoft Research and Maya Cakmak at University of Washington

𝗗𝗼𝗻'𝘁 𝗳𝗶𝗻𝗲-𝘁𝘂𝗻𝗲 𝗿𝗼𝗯𝗼𝘁 𝗳𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 𝗺𝗼𝗱𝗲𝗹𝘀. 𝗦𝘁𝗲𝗲𝗿 𝘁𝗵𝗲𝗺 𝘄𝗶𝘁𝗵 𝗵𝘂𝗺𝗮𝗻 𝗰𝗼𝗿𝗿𝗲𝗰𝘁𝗶𝗼𝗻𝘀 𝗶𝗻𝘀𝘁𝗲𝗮𝗱, 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝘁𝗵𝗲 𝗯𝗮𝘀𝗲 𝗽𝗼𝗹𝗶𝗰𝘆 Modern VLAs and world-action models can perform impressive manipulation skills, but adapting them reliably to new robots and tasks remains challenging. A natural solution is DAgger-style online imitation learning: deploy the robot, collect human corrections, and update the policy. Yet foundation models are fragile in the low-data regime, fine-tuning on a handful of interventions can improve one behavior while degrading others. Online post-training or reinforcement learning can require costly data collection and exploration, making real-world learning expensive and potentially unsafe. In our new paper, 𝗙𝗹𝗼𝘄𝗗𝗔𝗴𝗴𝗲𝗿, we take a different approach: 𝗜𝗻𝘀𝘁𝗲𝗮𝗱 𝗼𝗳 𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝘁𝗵𝗲 𝗳𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 𝗺𝗼𝗱𝗲𝗹, 𝘄𝗲 𝗹𝗲𝗮𝗿𝗻 𝗵𝗼𝘄 𝘁𝗼 𝘀𝘁𝗲𝗲𝗿 𝗶𝘁 𝗳𝗿𝗼𝗺 𝗵𝘂𝗺𝗮𝗻 𝗰𝗼𝗿𝗿𝗲𝗰𝘁𝗶𝗼𝗻𝘀. The key idea is 𝗮𝗰𝘁𝗶𝗼𝗻 𝗶𝗻𝘃𝗲𝗿𝘀𝗶𝗼𝗻: we map human corrective actions back into the latent noise space of the frozen generative policy. These latent targets train a lightweight controller that adapts the robot while preserving the original model's capabilities. Across simulation and real robots, FlowDAgger: 📈 Learns from only 5–20 human intervention episodes 🏆 Outperforms supervised fine-tuning and latent-space reinforcement learning 🤖 Works across VLAs, diffusion policies, and world-action models ✔️ Provides reliable improvements without modifying the pretrained policy We believe this offers a practical path toward making robot foundation models improve during deployment, learning from the way humans naturally teach: through corrections. 📄 Paper: 🌐 Project: 💻 Code: This project was led by my amazing colleague Michael Murray with help from Daphne Chen, Simran Bagaria, Dean Fortier, Tess Hellebrekers, Harshavardhan Reddy Gajarla, Galen Mullins and Andrey Kolobov at Microsoft Research and Maya Cakmak at University of Washington

Oier Mees

13,469 görüntüleme • 2 ay önce