Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Here’s a pretty weird and surprising result - retrieval-augmented generation works unreasonably well for robot learning – but only when parameterized using difference vectors! We introduce Difference-Aware Retrieval Policies for Imitation Learning (DARP), a simple, semi-parametric RAG architecture for imitation learning that achieves gains of up to 200% over... show more

Abhishek Gupta

12,231 subscribers

21,426 views • 4 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Robora Sim: A PyBullet-Powered Environment for Learning Robotic Physical Intelligence We are currently building our Robora simulation environment setup for our sim based learning, leveraging PyBullet, an industry-standard physics engine widely used in AI-driven robotics research and development. The environment is optimized with GPU-accelerated learning algorithms, enabling high-speed imitation learning and reinforcement learning within a safe and controlled virtual setup before shipping out to real world. This simulation platform allows our models to learn, adapt, and generalize across different robot morphologies, terrain types and task objectives - all before deployment to the real world. At it's core, the system combines a VLA-powered high-level planner with low-level motion control algorithms, working cohesively to produce emergent, physically intelligent behaviors. This synergy between simulation, learning, and real-world transfer marks a major step forward in our pursuit of adaptive and intelligent robotic systems. Through advanced domain randomization and synthetic data generation, the Robora Simulation Environment ensures that policies trained in simulation transfer effectively to real-world robots, minimizing the sim-to-real gap. Moreover, users will be able to test and integrate their own hardware kits within selected simulation environments in the Robora Dapp, ensuring seamless compatibility and safer real-world implementation.

Robora Sim: A PyBullet-Powered Environment for Learning Robotic Physical Intelligence We are currently building our Robora simulation environment setup for our sim based learning, leveraging PyBullet, an industry-standard physics engine widely used in AI-driven robotics research and development. The environment is optimized with GPU-accelerated learning algorithms, enabling high-speed imitation learning and reinforcement learning within a safe and controlled virtual setup before shipping out to real world. This simulation platform allows our models to learn, adapt, and generalize across different robot morphologies, terrain types and task objectives - all before deployment to the real world. At it's core, the system combines a VLA-powered high-level planner with low-level motion control algorithms, working cohesively to produce emergent, physically intelligent behaviors. This synergy between simulation, learning, and real-world transfer marks a major step forward in our pursuit of adaptive and intelligent robotic systems. Through advanced domain randomization and synthetic data generation, the Robora Simulation Environment ensures that policies trained in simulation transfer effectively to real-world robots, minimizing the sim-to-real gap. Moreover, users will be able to test and integrate their own hardware kits within selected simulation environments in the Robora Dapp, ensuring seamless compatibility and safer real-world implementation.

Robora

23,540 views • 1 year ago

had a crazy idea today for the past two months i've been building a pretty massive app and truth be told, i often play catch-up on architecture, because features move so damn fast so today i asked claude (opus 5.5, high) "I have a weird task for you. I want you to look at the app and create, for lack of a better word, a canvas that I can zoom in and zoom out and move around to see how our architecture actually works." (full prompt in first comment) this came out pretty great and it truly visualized, with steps and connected graphs what everything does 10/10 would recommend

had a crazy idea today for the past two months i've been building a pretty massive app and truth be told, i often play catch-up on architecture, because features move so damn fast so today i asked claude (opus 5.5, high) "I have a weird task for you. I want you to look at the app and create, for lack of a better word, a canvas that I can zoom in and zoom out and move around to see how our architecture actually works." (full prompt in first comment) this came out pretty great and it truly visualized, with steps and connected graphs what everything does 10/10 would recommend

Robin Ebers

187,500 views • 12 days ago

𝗥𝗼𝗯𝗼𝘁𝘀 𝗱𝗼𝗻’𝘁 𝗻𝗲𝗲𝗱 𝗺𝗼𝗿𝗲 𝗱𝗲𝗺𝗼𝗻𝘀𝘁𝗿𝗮𝘁𝗶𝗼𝗻𝘀. 𝗧𝗵𝗲𝘆 𝗻𝗲𝗲𝗱 𝘁𝗼 𝗹𝗲𝗮𝗿𝗻 𝗳𝗿𝗼𝗺 𝗳𝗮𝗶𝗹𝘂𝗿𝗲 — 𝗮𝗳𝘁𝗲𝗿 𝘄𝗮𝘁𝗰𝗵𝗶𝗻𝗴 𝗵𝘂𝗺𝗮𝗻𝘀. Most robot learning systems assume failure is the end of learning. In our new work, we study whether robots can improve after deployment by learning from their own failures, without any human intervention, teleoperation, or corrective labels. The key idea is simple: human videos contain structure about how the world works. We use them to learn cross-embodiment representations of action, dynamics, and value, enabling a shared predictive space between human behavior and robot experience. This allows a new learning loop: 👉 pretrain on human videos 👉 deploy robot policy 👉 observe failures 👉 reinterpret failures using human priors 👉 improve autonomously We evaluate this across 7 real-world manipulation tasks, showing: 📈 40% → 81% success rate 🏆 Strong improvements over π0.6 RECAP and RISE ✔️ Zero human intervention during post-deployment improvement 🧬 Generalizes across robot embodiments and policy backbones A key finding is that explicit failure repair significantly outperforms failure reweighting, yielding substantially larger gains under identical data conditions (+25 pts vs +5 pts on the same π0.5 base policy). Overall, the results suggest a shift in how we think about robot learning: Human videos are not only for pretraining policies. They can provide the structure needed for continual self-improvement after deployment. 📄 Paper: 🌐 Project: I am grateful for working with the fantastic leads Hanzhi Chen and Anran Zhang, and our collaborators Simon Schaefer, Kejia Chen, Shi Chen, Daniel Cremers. Special thanks to Stefan Leutenegger for co-advising this project with me. ETH Zürich TU München Microsoft Check out Hanzhi's 🧵 for more details

𝗥𝗼𝗯𝗼𝘁𝘀 𝗱𝗼𝗻’𝘁 𝗻𝗲𝗲𝗱 𝗺𝗼𝗿𝗲 𝗱𝗲𝗺𝗼𝗻𝘀𝘁𝗿𝗮𝘁𝗶𝗼𝗻𝘀. 𝗧𝗵𝗲𝘆 𝗻𝗲𝗲𝗱 𝘁𝗼 𝗹𝗲𝗮𝗿𝗻 𝗳𝗿𝗼𝗺 𝗳𝗮𝗶𝗹𝘂𝗿𝗲 — 𝗮𝗳𝘁𝗲𝗿 𝘄𝗮𝘁𝗰𝗵𝗶𝗻𝗴 𝗵𝘂𝗺𝗮𝗻𝘀. Most robot learning systems assume failure is the end of learning. In our new work, we study whether robots can improve after deployment by learning from their own failures, without any human intervention, teleoperation, or corrective labels. The key idea is simple: human videos contain structure about how the world works. We use them to learn cross-embodiment representations of action, dynamics, and value, enabling a shared predictive space between human behavior and robot experience. This allows a new learning loop: 👉 pretrain on human videos 👉 deploy robot policy 👉 observe failures 👉 reinterpret failures using human priors 👉 improve autonomously We evaluate this across 7 real-world manipulation tasks, showing: 📈 40% → 81% success rate 🏆 Strong improvements over π0.6 RECAP and RISE ✔️ Zero human intervention during post-deployment improvement 🧬 Generalizes across robot embodiments and policy backbones A key finding is that explicit failure repair significantly outperforms failure reweighting, yielding substantially larger gains under identical data conditions (+25 pts vs +5 pts on the same π0.5 base policy). Overall, the results suggest a shift in how we think about robot learning: Human videos are not only for pretraining policies. They can provide the structure needed for continual self-improvement after deployment. 📄 Paper: 🌐 Project: I am grateful for working with the fantastic leads Hanzhi Chen and Anran Zhang, and our collaborators Simon Schaefer, Kejia Chen, Shi Chen, Daniel Cremers. Special thanks to Stefan Leutenegger for co-advising this project with me. ETH Zürich TU München Microsoft Check out Hanzhi's 🧵 for more details

Oier Mees

12,732 views • 3 months ago

Today we’re announcing our new Mentava Basics curriculum, aimed at kids who are still a bit too young for the standard Mentava curriculum. Here’s what people don’t understand about teaching a 2 or 3 year old to read: The necessary skills for reading don’t all develop at the same time. The ability to associate letters with sounds happens first, and at a pretty early age. If you think about it, learning “this funny shaped animal says moo” is pretty similar to learning “this funny shaped line says aaa.” However, the second necessary skill for reading is blending those letter sounds together, and kids often aren’t developmentally capable of doing that until at least 6-12 months later. Until today, our recommendation has been to wait until the child is developmentally ready to blend sounds. and then we just go full speed and teach them everything all at once, as fast as possible. But sometimes we have students who start a little younger. And then their parents are confused, because they see that their kids are having a ton of fun learning letter sounds super fast, but then are being gatekept from additional learning because they aren't yet developmentally ready to blend those sounds. That’s why we created the new Mentava Basics curriculum. Mentava Basics lets our youngest students focus on letter/sound pairings until they're developmentally ready to begin blending them. Mentava Basics takes the fun and delight of the core Mentava experience, but applies it to a curriculum that’s developmentally appropriate for even younger children. Mentava’s standard curriculum is still the fastest way to go from zero to reading, but with Mentava Basics now we can give kids a head start by helping them learn their letter sounds in advance. If your child is struggling with blending and you think it may just be a developmental readiness issue, you can use the grownup menu to switch into the Mentava Basics curriculum. We save your progress on both pathways, so you can switch back to our standard curriculum whenever you want.

Today we’re announcing our new Mentava Basics curriculum, aimed at kids who are still a bit too young for the standard Mentava curriculum. Here’s what people don’t understand about teaching a 2 or 3 year old to read: The necessary skills for reading don’t all develop at the same time. The ability to associate letters with sounds happens first, and at a pretty early age. If you think about it, learning “this funny shaped animal says moo” is pretty similar to learning “this funny shaped line says aaa.” However, the second necessary skill for reading is blending those letter sounds together, and kids often aren’t developmentally capable of doing that until at least 6-12 months later. Until today, our recommendation has been to wait until the child is developmentally ready to blend sounds. and then we just go full speed and teach them everything all at once, as fast as possible. But sometimes we have students who start a little younger. And then their parents are confused, because they see that their kids are having a ton of fun learning letter sounds super fast, but then are being gatekept from additional learning because they aren't yet developmentally ready to blend those sounds. That’s why we created the new Mentava Basics curriculum. Mentava Basics lets our youngest students focus on letter/sound pairings until they're developmentally ready to begin blending them. Mentava Basics takes the fun and delight of the core Mentava experience, but applies it to a curriculum that’s developmentally appropriate for even younger children. Mentava’s standard curriculum is still the fastest way to go from zero to reading, but with Mentava Basics now we can give kids a head start by helping them learn their letter sounds in advance. If your child is struggling with blending and you think it may just be a developmental readiness issue, you can use the grownup menu to switch into the Mentava Basics curriculum. We save your progress on both pathways, so you can switch back to our standard curriculum whenever you want.

Niels Hoven 🐮

10,540 views • 1 year ago

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.

Rohan Paul

37,121 views • 7 months ago

TESLA HALTED MODEL S AND MODEL X PRODUCTION TO BUILD AN ARMY OF OPTIMUS ROBOTS The Fremont assembly line was torn down in 46 days. In its place, Tesla is building a line for humanoid production, aiming for a million units a year A humanoid robot is a body shaped like a human. Physical AI is the intelligence that controls that body Walking and making coffee is often just imitation learning from a scripted routine. But once the environment shifts, the learned trick stops working Language models had the entire internet to train on. Robotics has nothing close to that scale of data, which is why one giant brain hasn't worked for anyone yet The industry is moving toward modularity instead - separate models for vision, movement, and planning, each improved on its own The real question is no longer whether a robot can move impressively. It's whether it can pull its sensors into one picture of the world and adapt to whatever wasn't scripted for it

TESLA HALTED MODEL S AND MODEL X PRODUCTION TO BUILD AN ARMY OF OPTIMUS ROBOTS The Fremont assembly line was torn down in 46 days. In its place, Tesla is building a line for humanoid production, aiming for a million units a year A humanoid robot is a body shaped like a human. Physical AI is the intelligence that controls that body Walking and making coffee is often just imitation learning from a scripted routine. But once the environment shifts, the learned trick stops working Language models had the entire internet to train on. Robotics has nothing close to that scale of data, which is why one giant brain hasn't worked for anyone yet The industry is moving toward modularity instead - separate models for vision, movement, and planning, each improved on its own The real question is no longer whether a robot can move impressively. It's whether it can pull its sensors into one picture of the world and adapt to whatever wasn't scripted for it

iamigorekk

22,205 views • 1 month ago

As a newly appointed 𝗔𝘀𝘀𝗶𝘀𝘁𝗮𝗻𝘁 𝗣𝗿𝗼𝗳𝗲𝘀𝘀𝗼𝗿 at Imperial College London, I'm thrilled to announce the 𝗦𝗮𝗳𝗲 𝗪𝗵𝗼𝗹𝗲-𝗯𝗼𝗱𝘆 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝘁 𝗥𝗼𝗯𝗼𝘁𝗶𝗰𝘀 𝗟𝗮𝗯 (𝗦𝗪𝗜𝗥𝗟) at 𝗜𝗺𝗽𝗲𝗿𝗶𝗮𝗹 𝗖𝗼𝗹𝗹𝗲𝗴𝗲 𝗟𝗼𝗻𝗱𝗼𝗻. 𝗦𝗮𝗳𝗲 𝗪𝗵𝗼𝗹𝗲-𝗯𝗼𝗱𝘆 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝘁 𝗥𝗼𝗯𝗼𝘁𝗶𝗰𝘀 𝗟𝗮𝗯 (𝗦𝗪𝗜𝗥𝗟) ( is a new research lab focused on the intersection of safety and intelligence in next-generation robotics. We're hiring exceptional PhD students who are passionate about pushing the boundaries of robot learning. 𝗪𝗵𝗮𝘁 𝗺𝗮𝗸𝗲𝘀 𝗦𝗪𝗜𝗥𝗟 𝘂𝗻𝗶𝗾𝘂𝗲? We operate at the exciting convergence of: • Online & offline reinforcement learning • Imitation learning & human demonstrations • Sample-efficient learning methods • Whole-body and soft robotics systems We're 𝗹𝗼𝗼𝗸𝗶𝗻𝗴 𝗳𝗼𝗿 𝗽𝗿𝗼𝘀𝗽𝗲𝗰𝘁𝗶𝘃𝗲 𝗣𝗵𝗗 𝘀𝘁𝘂𝗱𝗲𝗻𝘁𝘀 interested in: • Developing safe exploration algorithms for robotic systems • Creating sample-efficient learning methods that minimize real-world trials • Building foundation models for robotics with safety guarantees • Advancing soft robotics and compliant human-robot interaction • Bridging theory and practice in embodied AI Why now? As robots become more capable and work closer with humans, we need systems that are both intelligent enough to handle complex tasks 𝗔𝗡𝗗 safe enough for real-world deployment. Traditional approaches treat safety and intelligence as competing priorities, we believe they're synergistic. If you're a motivated researcher who wants to develop the theoretical foundations and practical algorithms for tomorrow's safe, intelligent robots, I'd love to hear from you. Want to join? Apply via

As a newly appointed 𝗔𝘀𝘀𝗶𝘀𝘁𝗮𝗻𝘁 𝗣𝗿𝗼𝗳𝗲𝘀𝘀𝗼𝗿 at Imperial College London, I'm thrilled to announce the 𝗦𝗮𝗳𝗲 𝗪𝗵𝗼𝗹𝗲-𝗯𝗼𝗱𝘆 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝘁 𝗥𝗼𝗯𝗼𝘁𝗶𝗰𝘀 𝗟𝗮𝗯 (𝗦𝗪𝗜𝗥𝗟) at 𝗜𝗺𝗽𝗲𝗿𝗶𝗮𝗹 𝗖𝗼𝗹𝗹𝗲𝗴𝗲 𝗟𝗼𝗻𝗱𝗼𝗻. 𝗦𝗮𝗳𝗲 𝗪𝗵𝗼𝗹𝗲-𝗯𝗼𝗱𝘆 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝘁 𝗥𝗼𝗯𝗼𝘁𝗶𝗰𝘀 𝗟𝗮𝗯 (𝗦𝗪𝗜𝗥𝗟) ( is a new research lab focused on the intersection of safety and intelligence in next-generation robotics. We're hiring exceptional PhD students who are passionate about pushing the boundaries of robot learning. 𝗪𝗵𝗮𝘁 𝗺𝗮𝗸𝗲𝘀 𝗦𝗪𝗜𝗥𝗟 𝘂𝗻𝗶𝗾𝘂𝗲? We operate at the exciting convergence of: • Online & offline reinforcement learning • Imitation learning & human demonstrations • Sample-efficient learning methods • Whole-body and soft robotics systems We're 𝗹𝗼𝗼𝗸𝗶𝗻𝗴 𝗳𝗼𝗿 𝗽𝗿𝗼𝘀𝗽𝗲𝗰𝘁𝗶𝘃𝗲 𝗣𝗵𝗗 𝘀𝘁𝘂𝗱𝗲𝗻𝘁𝘀 interested in: • Developing safe exploration algorithms for robotic systems • Creating sample-efficient learning methods that minimize real-world trials • Building foundation models for robotics with safety guarantees • Advancing soft robotics and compliant human-robot interaction • Bridging theory and practice in embodied AI Why now? As robots become more capable and work closer with humans, we need systems that are both intelligent enough to handle complex tasks 𝗔𝗡𝗗 safe enough for real-world deployment. Traditional approaches treat safety and intelligence as competing priorities, we believe they're synergistic. If you're a motivated researcher who wants to develop the theoretical foundations and practical algorithms for tomorrow's safe, intelligent robots, I'd love to hear from you. Want to join? Apply via

Stephen James

16,902 views • 1 year ago

China is smart. China learned from the US's successes, using only the best parts. It's our turn. We should be learning from China. China's economy is growing three times faster than that of the USA. China leads in almost all areas of science, and has the best infrastructure the world has ever seen. The number one priority of China is tell help their people become more prosperous, especially the least well off. Is there still work to be done. Hell yeah. But there difference is that they are trying and we are not. I've been to hundreds of Chinese villages, for more than a decade, and people are becoming better off each year. They call it 'Socialism with Chinese Characteristics.' We don't need to replicate their system. But we should be learning from it. For those ignoring it, you are hurting America. Pretending China's model isn't successful is preventing the US from improving and learning. The China-Haters should be ashamed of the damage that they are doing to the West. Instead of slander and Sinophobia, we need to be learning from the successes of other nations and their peoples.

China is smart. China learned from the US's successes, using only the best parts. It's our turn. We should be learning from China. China's economy is growing three times faster than that of the USA. China leads in almost all areas of science, and has the best infrastructure the world has ever seen. The number one priority of China is tell help their people become more prosperous, especially the least well off. Is there still work to be done. Hell yeah. But there difference is that they are trying and we are not. I've been to hundreds of Chinese villages, for more than a decade, and people are becoming better off each year. They call it 'Socialism with Chinese Characteristics.' We don't need to replicate their system. But we should be learning from it. For those ignoring it, you are hurting America. Pretending China's model isn't successful is preventing the US from improving and learning. The China-Haters should be ashamed of the damage that they are doing to the West. Instead of slander and Sinophobia, we need to be learning from the successes of other nations and their peoples.

Jason Smith - 上官杰文

82,834 views • 1 year ago

Introducing VL-JEPA: Vision-Language Joint Embedding Predictive Architecture for streaming, live action recognition, retrieval, VQA, and classification tasks with better performance and higher efficiency than large VLMs. • VL-JEPA is the first non-generative model that can perform general-domain vision-language tasks in real-time, built on a joint embedding predictive architecture. • We demonstrate in controlled experiments that VL-JEPA, trained with latent space embedding prediction, outperforms VLMs that rely on data space token prediction. • We show that VL-JEPA delivers significant efficiency gains over VLMs for online video streaming applications, thanks to its non-autoregressive design and native support for selective decoding. • We highlight that our VL-JEPA model, with an unified model architecture, can effectively handle a wide range of classification, retrieval, and VQA tasks at the same time. by Delong Chen (陈德龙) Mustafa Shukor Théo Moutakanni Willy Jade Lei Yu Tejaswi Kasarla Allen Bolourchi Yann LeCun Pascale Fung

Introducing VL-JEPA: Vision-Language Joint Embedding Predictive Architecture for streaming, live action recognition, retrieval, VQA, and classification tasks with better performance and higher efficiency than large VLMs. • VL-JEPA is the first non-generative model that can perform general-domain vision-language tasks in real-time, built on a joint embedding predictive architecture. • We demonstrate in controlled experiments that VL-JEPA, trained with latent space embedding prediction, outperforms VLMs that rely on data space token prediction. • We show that VL-JEPA delivers significant efficiency gains over VLMs for online video streaming applications, thanks to its non-autoregressive design and native support for selective decoding. • We highlight that our VL-JEPA model, with an unified model architecture, can effectively handle a wide range of classification, retrieval, and VQA tasks at the same time. by Delong Chen (陈德龙) Mustafa Shukor Théo Moutakanni Willy Jade Lei Yu Tejaswi Kasarla Allen Bolourchi Yann LeCun Pascale Fung

Pascale Fung

90,522 views • 10 months ago

Most imitation learning policies break when the camera moves or the robot changes. NOT THIS ONE 👇 [📍 Bookmark for later ] A new 3D scene representation encoder, tackles this by enabling zero-shot generalization to unseen embodiments and viewpoints… And it works with any IL algorithm. The trick? •Use a 2D foundation model to extract semantic features •Lift them into 3D space for localization (not semantics) •Condition the IL policy on this spatially grounded vector Across 93 simulated and 6 real tasks, Adapt3R: ✅ Maintains IL performance on LIBERO & MimicGen benchmarks ✅ Outperforms DP3 and 3D Diffuser Actor in most settings ✅ Holds >80% success on LIBERO even with large camera rotations Thanks for sharing this, Animesh Garg & Albert Wilcox! 📍Paper: Website: Code:

Most imitation learning policies break when the camera moves or the robot changes. NOT THIS ONE 👇 [📍 Bookmark for later ] A new 3D scene representation encoder, tackles this by enabling zero-shot generalization to unseen embodiments and viewpoints… And it works with any IL algorithm. The trick? •Use a 2D foundation model to extract semantic features •Lift them into 3D space for localization (not semantics) •Condition the IL policy on this spatially grounded vector Across 93 simulated and 6 real tasks, Adapt3R: ✅ Maintains IL performance on LIBERO & MimicGen benchmarks ✅ Outperforms DP3 and 3D Diffuser Actor in most settings ✅ Holds >80% success on LIBERO even with large camera rotations Thanks for sharing this, Animesh Garg & Albert Wilcox! 📍Paper: Website: Code:

Ilir Aliu

12,209 views • 1 year ago

Streaming iPhone data in real-time directly to Rerun 🚀 The collection process is one of the most frustrating parts of building imitation-learning datasets. I’ve got a little army of sensors—📱 iPhone, iPad, Quest 3—but getting them temporally aligned, spatially aligned, AND seeing real-time feedback while recording is tough. I stumbled on a great library from Cake Lab (WPI) called ARFlow. It’s a thin client built on Unity’s ARFoundation that connects over gRPC to a server running Rerun for live data logging. I forked it to: - Log the SLAM translation poses, and - Upgrade rerun to v0.23 for my use case. So far, it works well, but there are still a few hitches: 1. Right now, it’s solid on iPhone and iPad; my Quest 3 client is still slow and not super reliable. 2. I’m using an older ARFlow branch focused on real-time streaming only—no spatial or temporal sync yet. Unity builds for iOS keep failing. 🛠️ 3. Nothing is saved locally to the client, so packet loss is a risk on shaky networks. There’s huge potential in tapping the ubiquitous sensors we carry around every day, and ARFlow is a big step toward making that easy

Streaming iPhone data in real-time directly to Rerun 🚀 The collection process is one of the most frustrating parts of building imitation-learning datasets. I’ve got a little army of sensors—📱 iPhone, iPad, Quest 3—but getting them temporally aligned, spatially aligned, AND seeing real-time feedback while recording is tough. I stumbled on a great library from Cake Lab (WPI) called ARFlow. It’s a thin client built on Unity’s ARFoundation that connects over gRPC to a server running Rerun for live data logging. I forked it to: - Log the SLAM translation poses, and - Upgrade rerun to v0.23 for my use case. So far, it works well, but there are still a few hitches: 1. Right now, it’s solid on iPhone and iPad; my Quest 3 client is still slow and not super reliable. 2. I’m using an older ARFlow branch focused on real-time streaming only—no spatial or temporal sync yet. Unity builds for iOS keep failing. 🛠️ 3. Nothing is saved locally to the client, so packet loss is a risk on shaky networks. There’s huge potential in tapping the ubiquitous sensors we carry around every day, and ARFlow is a big step toward making that easy

Pablo Vela

27,433 views • 1 year ago

$We removed 93.25% of the connections in our Un-0 image model, fully expecting to pay for it in quality. But it got better. FID 7.15 on ImageNet 64x64, roughly 1.9 ahead of the dense baseline at matched size. Same family of model, a fraction of the couplings, a better score. Here is why that is not as strange as it sounds. Un-0 is a coupled oscillator model, and in the dense version every oscillator talks to every other one. That sounds like a strength, but it means the whole system can fall into catastrophic synchronization: everything locks into step, gradients go flat, and learning stalls. Sparser connectivity leaves room for coherent and incoherent activity to coexist. The dynamics stay alive, and the model keeps learning. Connectivity turns out to be a control knob, not a dial you turn up until it stops. Learn more here:$

We removed 93.25% of the connections in our Un-0 image model, fully expecting to pay for it in quality. But it got better. FID 7.15 on ImageNet 64x64, roughly 1.9 ahead of the dense baseline at matched size. Same family of model, a fraction of the couplings, a better score. Here is why that is not as strange as it sounds. Un-0 is a coupled oscillator model, and in the dense version every oscillator talks to every other one. That sounds like a strength, but it means the whole system can fall into catastrophic synchronization: everything locks into step, gradients go flat, and learning stalls. Sparser connectivity leaves room for coherent and incoherent activity to coexist. The dynamics stay alive, and the model keeps learning. Connectivity turns out to be a control knob, not a dial you turn up until it stops. Learn more here:

Unconventional AI

38,121 views • 2 months ago

Here’s a video of a McDonald's pneumatic ketchup/mustard dispenser that never made it to restaurants (post below). For decades we had two separate dispensers: one for ketchup and one for mustard. We’re always looking to make our kitchens faster, one second at a time. They’re pretty efficient, but you had to pick up two different dispensers. Now there’s a single dual-purpose version. It dispenses both at once or either one, depending on how you move the black handle (left for mustard, center for both, right for ketchup). Sorry for the poor video, I’m filming and trying to show the dispenser at once. Attached is a pic of the old-school way where you pick up one in each hand. Yes, it’s probably over-engineered. Yes, it works. Yes, it saves time.

Here’s a video of a McDonald's pneumatic ketchup/mustard dispenser that never made it to restaurants (post below). For decades we had two separate dispensers: one for ketchup and one for mustard. We’re always looking to make our kitchens faster, one second at a time. They’re pretty efficient, but you had to pick up two different dispensers. Now there’s a single dual-purpose version. It dispenses both at once or either one, depending on how you move the black handle (left for mustard, center for both, right for ketchup). Sorry for the poor video, I’m filming and trying to show the dispenser at once. Attached is a pic of the old-school way where you pick up one in each hand. Yes, it’s probably over-engineered. Yes, it works. Yes, it saves time.

McFranchisee

103,317 views • 9 days ago

🤖 Robotic hands are evolving faster than we realize — and they’re learning to master our world. I find this both fascinating and a little unsettling. At first glance, these robotic hands still look awkward - stiff fingers, clumsy motion. But behind the scenes, every iteration brings: ➡️ Tighter sensors ➡️ Smarter control loops ➡️ Better training on human-designed objects Week by week, they move closer to something astonishing — not imitation, but mastery. They’re learning to grip, twist, and adapt to tools made for our anatomy. And once they reach human-level dexterity, they won’t stop there. They’ll surpass it — steadier than surgeons, faster than assembly lines, tireless and precise. This is more than robotics. It’s the moment machines stop mimicking us… and start outperforming us. How do we redefine “human skill” in a world where robots master it too? #AI #Robotics #Innovation #Technology #Automation #FutureOfWork #Engineering

🤖 Robotic hands are evolving faster than we realize — and they’re learning to master our world. I find this both fascinating and a little unsettling. At first glance, these robotic hands still look awkward - stiff fingers, clumsy motion. But behind the scenes, every iteration brings: ➡️ Tighter sensors ➡️ Smarter control loops ➡️ Better training on human-designed objects Week by week, they move closer to something astonishing — not imitation, but mastery. They’re learning to grip, twist, and adapt to tools made for our anatomy. And once they reach human-level dexterity, they won’t stop there. They’ll surpass it — steadier than surgeons, faster than assembly lines, tireless and precise. This is more than robotics. It’s the moment machines stop mimicking us… and start outperforming us. How do we redefine “human skill” in a world where robots master it too? #AI #Robotics #Innovation #Technology #Automation #FutureOfWork #Engineering

Pascal Bornet

97,943 views • 9 months ago

Ok, that's amazing. 🦄 A side by side three.js comparison - with GitMCP and without. GitMCP's story originally started from a tweet by Three.js about their large, non-LLM-digestable, documentation file. Ido Salomon and I ended up creating a cool generic documentation MCP server, but we didn't forget threejs and mrdoob. So we put it to the test. Using GitMCP, I gave this prompt to Cursor + Claude 3.7: "Build a Three.js scene featuring a controllable realistic person navigating a textured dynamic urban environment with realistic lighting and subtle bloom effects. Ensure keyboard controls (WASD) for movement." This is the result. To the right is the one-shot result with our MCP server. To the left is the result without it, based just on Cursor's limited knowledge. The video is pretty conclusive. 🤯 It's really amazing since Cursor already knows a lot about three.js, but with our dedicated documentation MCP server it just outputs better results. And for all other libraries out there - that are not included in Cursor - this is a total game changer. A huge shout out to Cloudflare Developers and Anni Wang, who worked with us yesterday to quickly insert new capabilities in their AutoRAG feature (that enables indexing large amount of documentation) according to our specific needs, and to troubleshoot issues. It was very helpful. Knowledge is power, and that applies for every coding assitant. And not just for three.js! Check out GitMCP for any library you're using - link in the next comment.

Ok, that's amazing. 🦄 A side by side three.js comparison - with GitMCP and without. GitMCP's story originally started from a tweet by Three.js about their large, non-LLM-digestable, documentation file. Ido Salomon and I ended up creating a cool generic documentation MCP server, but we didn't forget threejs and mrdoob. So we put it to the test. Using GitMCP, I gave this prompt to Cursor + Claude 3.7: "Build a Three.js scene featuring a controllable realistic person navigating a textured dynamic urban environment with realistic lighting and subtle bloom effects. Ensure keyboard controls (WASD) for movement." This is the result. To the right is the one-shot result with our MCP server. To the left is the result without it, based just on Cursor's limited knowledge. The video is pretty conclusive. 🤯 It's really amazing since Cursor already knows a lot about three.js, but with our dedicated documentation MCP server it just outputs better results. And for all other libraries out there - that are not included in Cursor - this is a total game changer. A huge shout out to Cloudflare Developers and Anni Wang, who worked with us yesterday to quickly insert new capabilities in their AutoRAG feature (that enables indexing large amount of documentation) according to our specific needs, and to troubleshoot issues. It was very helpful. Knowledge is power, and that applies for every coding assitant. And not just for three.js! Check out GitMCP for any library you're using - link in the next comment.

Liad Yosef

41,745 views • 1 year ago

World modeling and imitation learning have largely been considered two disparate worlds. In our recent work, Unified World Models, just accepted to #RSS2025, Chuning Zhu provides a dead-simple unifying solution: just train a joint diffusion model over actions and future states, but with *decoupled* diffusion time steps across these modalities. Manipulating these decoupled time steps then allows for marginalization or conditioning on actions or states; a single model can serve as a policy, forward dynamics model, video prediction model, or inverse dynamics model by simply setting diffusion timesteps carefully. The resulting model can leverage video datasets along with robot training data much more effectively, and shows improved robustness, generalization, and flexibility. This is exciting because it is frustratingly simple, scalable, and shows strong improvement on real-world robotics problems. Please refer to Chuning Zhu 's excellent thread for more details! More details/code can be found on our website and in the paper -

World modeling and imitation learning have largely been considered two disparate worlds. In our recent work, Unified World Models, just accepted to #RSS2025, Chuning Zhu provides a dead-simple unifying solution: just train a joint diffusion model over actions and future states, but with decoupled diffusion time steps across these modalities. Manipulating these decoupled time steps then allows for marginalization or conditioning on actions or states; a single model can serve as a policy, forward dynamics model, video prediction model, or inverse dynamics model by simply setting diffusion timesteps carefully. The resulting model can leverage video datasets along with robot training data much more effectively, and shows improved robustness, generalization, and flexibility. This is exciting because it is frustratingly simple, scalable, and shows strong improvement on real-world robotics problems. Please refer to Chuning Zhu 's excellent thread for more details! More details/code can be found on our website and in the paper -

Abhishek Gupta

11,430 views • 1 year ago

What happens when a child learns that bad behavior rarely has any real consequences? 👀👀👀 All parents want well behaved children, but how many actually want to be strict and discipline them when they start crossing the lines? It can be easier to give in, ignore the behavior, or hope the child eventually grows out of it. The problem is that children are constantly learning where the boundaries are. If they discover that screaming, throwing a fit, talking back, refusing to listen, or deliberately breaking rules gets them what they want, there is very little reason for them to stop. Young children need boundaries because they are still learning self control. They need to know that when a parent says no, it actually means no. They need consistent consequences when they knowingly cross a line. That does not mean being harsh or losing your temper. Good discipline can be calm, reasonable, and firm. The goal is not to make a child afraid of their parents. It is to teach them that their choices have consequences and that other people deserve respect. There is also a difference between discipline and simply punishing a child. Discipline is about teaching. Sometimes that means taking away a privilege. Sometimes it means making a child correct what they did. Sometimes it means simply refusing to reward bad behavior with attention or a change of rules. Children who grow up with clear expectations often have a better understanding of how to behave when their parents are not around. Children who learn that every boundary can be negotiated may keep pushing until they find the limit. Parents don't have to be perfect, but they have to be willing to lead. If children are never taught where the boundaries are, how can we expect them to know when they've crossed them?

What happens when a child learns that bad behavior rarely has any real consequences? 👀👀👀 All parents want well behaved children, but how many actually want to be strict and discipline them when they start crossing the lines? It can be easier to give in, ignore the behavior, or hope the child eventually grows out of it. The problem is that children are constantly learning where the boundaries are. If they discover that screaming, throwing a fit, talking back, refusing to listen, or deliberately breaking rules gets them what they want, there is very little reason for them to stop. Young children need boundaries because they are still learning self control. They need to know that when a parent says no, it actually means no. They need consistent consequences when they knowingly cross a line. That does not mean being harsh or losing your temper. Good discipline can be calm, reasonable, and firm. The goal is not to make a child afraid of their parents. It is to teach them that their choices have consequences and that other people deserve respect. There is also a difference between discipline and simply punishing a child. Discipline is about teaching. Sometimes that means taking away a privilege. Sometimes it means making a child correct what they did. Sometimes it means simply refusing to reward bad behavior with attention or a change of rules. Children who grow up with clear expectations often have a better understanding of how to behave when their parents are not around. Children who learn that every boundary can be negotiated may keep pushing until they find the limit. Parents don't have to be perfect, but they have to be willing to lead. If children are never taught where the boundaries are, how can we expect them to know when they've crossed them?

Declaration of Memes

28,551 views • 8 days ago

We’ve seen humanoid robots walk around for a while, but when will they actually help with useful tasks in daily life? The challenge here is the diversity and complexity of real-world scenes. Our new work tackles this problem via 3D visuomotor policy learning. Using data from only 1 scene, our Improved 3D Diffusion Policy (iDP3) enables a full-sized humanoid robot to autonomously pick&place objects, pour water, and wipe tables, in the wild open world. (and all these skills are useful, right?) Web: Fully open-sourced code:

We’ve seen humanoid robots walk around for a while, but when will they actually help with useful tasks in daily life? The challenge here is the diversity and complexity of real-world scenes. Our new work tackles this problem via 3D visuomotor policy learning. Using data from only 1 scene, our Improved 3D Diffusion Policy (iDP3) enables a full-sized humanoid robot to autonomously pick&place objects, pour water, and wipe tables, in the wild open world. (and all these skills are useful, right?) Web: Fully open-sourced code:

Yanjie Ze

75,420 views • 2 years ago