Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

The robot is learning several novel tasks instantly, after just ONE demonstration each... Instant Policy makes it possible: no extra training, no weight updates, just pure in-context learning. It just got accepted at ICLR 2025, and it’s changing how robots learn. With just a single demo, a robot can...

IlirAliu_'s profile picture

Ilir Aliu

49,464 subscribers

46,285 Aufrufe • vor 1 Jahr •via X (Twitter)

BildungWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

6 Kommentare

Profilbild von Mark Ruby
Mark Rubyvor 1 Jahr

I'm waiting for the robot to become sentient, then impatient, and slap him.

Profilbild von Luke Liem
Luke Liemvor 1 Jahr

This combines imitation learning and graph representation. Apparently a transformer is also involved in the architecture - to learn context?

Profilbild von The Rundown AI
The Rundown AIvor 1 Jahr

If you're not learning AI in 2025, you're falling behind. Join 1,000,000+ early adopters reading and learn AI in just 5 minutes a day (for free).

Profilbild von Dayder
Daydervor 1 Jahr

Dum-E from Iron Man is so close😁 Amazing progress.

Profilbild von Jamal Jalal
Jamal Jalalvor 1 Jahr

يتعلم ذراع الروبوت من توجيه واحد فقط ولايتأثر بتغير البيئه , اعتقد انهم اقتربوا من صناعة هواتف الموبايل بالروبوتات ,حيث كان الهدف قبل سنوات ان ينفذ الروبوت ما يراه اي يتم تعليمه كما يتعلم البشر من النظر,كان ينبغي على الدول العربيه السير بهذا الطريق وهو مربح للغايه ايضا

Profilbild von Renato Pudo
Renato Pudovor 1 Jahr

🤯

Ähnliche Videos

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER
0:30

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER

Ilir Aliu - eu/acc

10,509 Aufrufe • vor 1 Jahr

The $10k robot arms that can learn from a handful of demos: Most robots still lose the cost equation. They’re slow, expensive, and take months to deploy, which means… they only make sense for huge factories. The The Robot Learning Company (YC X25) is building affordable, general-purpose robot arms that can pick up new tasks from just a few human demonstrations. Why it matters: ✅ Under $10k per pair of stationary arms ✅ Adapts to new tasks with imitation + reinforcement learning ✅ Deployable in days, not months From small manufacturers to service providers, this makes robotic labor accessible to companies that could never justify it before. Saying hi to founder Jannik Grothusen 👋 More here:
0:39

The $10k robot arms that can learn from a handful of demos: Most robots still lose the cost equation. They’re slow, expensive, and take months to deploy, which means… they only make sense for huge factories. The The Robot Learning Company (YC X25) is building affordable, general-purpose robot arms that can pick up new tasks from just a few human demonstrations. Why it matters: ✅ Under $10k per pair of stationary arms ✅ Adapts to new tasks with imitation + reinforcement learning ✅ Deployable in days, not months From small manufacturers to service providers, this makes robotic labor accessible to companies that could never justify it before. Saying hi to founder Jannik Grothusen 👋 More here:

Ilir Aliu - eu/acc

36,336 Aufrufe • vor 10 Monaten

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]
0:25

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Max Fu

40,392 Aufrufe • vor 1 Jahr

How do you teach a robot to handle complex, multi-step tasks, without training it for each one? [Github ⬇️] The team behind ReKep shows that robots can perform bimanual, in-the-wild tasks by reasoning over keypoint constraints: Generated on the fly using vision and language models. No task-specific data, no environment modeling. Why it matters ✅ Encodes tasks as simple Python functions over 3D keypoints ✅ Uses VLMs to generate keypoint constraints from instructions ✅ Plans and replans in real time with a 10 Hz perception-action loop ✅ Works for bimanual, multi-stage tasks without task-specific training Built on open tools like SciPy and BEHAVIOR, ReKep brings reactive, general-purpose reasoning closer to real-world robot control. Project website: Paper: Code: Walkthrough video: Thank you, Wenlong Huang for sharing 🫶
1:56

How do you teach a robot to handle complex, multi-step tasks, without training it for each one? [Github ⬇️] The team behind ReKep shows that robots can perform bimanual, in-the-wild tasks by reasoning over keypoint constraints: Generated on the fly using vision and language models. No task-specific data, no environment modeling. Why it matters ✅ Encodes tasks as simple Python functions over 3D keypoints ✅ Uses VLMs to generate keypoint constraints from instructions ✅ Plans and replans in real time with a 10 Hz perception-action loop ✅ Works for bimanual, multi-stage tasks without task-specific training Built on open tools like SciPy and BEHAVIOR, ReKep brings reactive, general-purpose reasoning closer to real-world robot control. Project website: Paper: Code: Walkthrough video: Thank you, Wenlong Huang for sharing 🫶

Ilir Aliu - eu/acc

25,348 Aufrufe • vor 1 Jahr

Robot AI brains, aka Vision-Language-Action models, cannot adapt to new tasks as easily as LLMs like Gemini, ChatGPT, or Grok. LLMs can adapt quickly with their in-context learning (ICL) capabilities. But can we inject ICL abilities into a pre-trained VLA like pi0? Yes! Introducing RICL (Retraining for In-Context Learning), our Conference on Robot Learning (CoRL) 2025 paper. Our RICL-pi0 model can adapt to unseen objects, novel motions, and new scenes with just ICL and RAG (retrieval-augmented generation). RICL-pi0 also boosts performance on the long-tail of tasks. A quick 1 minute video summary:
1:03

Robot AI brains, aka Vision-Language-Action models, cannot adapt to new tasks as easily as LLMs like Gemini, ChatGPT, or Grok. LLMs can adapt quickly with their in-context learning (ICL) capabilities. But can we inject ICL abilities into a pre-trained VLA like pi0? Yes! Introducing RICL (Retraining for In-Context Learning), our Conference on Robot Learning (CoRL) 2025 paper. Our RICL-pi0 model can adapt to unseen objects, novel motions, and new scenes with just ICL and RAG (retrieval-augmented generation). RICL-pi0 also boosts performance on the long-tail of tasks. A quick 1 minute video summary:

Kaustubh Sridhar

52,158 Aufrufe • vor 10 Monaten

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:
0:29

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

42,804 Aufrufe • vor 4 Monaten

Introducing RoboCat, a new AI model designed to operate multiple robots. 🤖 It learns to solve new tasks on different robotic arms with as few as 100 demonstrations - and improves skills from self-generated training data. Find out more:
0:52

Introducing RoboCat, a new AI model designed to operate multiple robots. 🤖 It learns to solve new tasks on different robotic arms with as few as 100 demonstrations - and improves skills from self-generated training data. Find out more:

Google DeepMind

410,272 Aufrufe • vor 3 Jahren

How precise can a robot get… without ever seeing the object move? Turns out, just a geometric model is enough. A robot hit 73 precise 3D goals in a row, using only a geometric model. No demonstrations. No imitation learning. No fancy vision models. The secret? Sample where the robot could make contact. Let local contact-implicit control take over from there. That’s it. ✅ Real-time performance ✅ Natural contact making and breaking ✅ Works on tricky tasks like pushing and rotating This approach mixes global planning with local MPC. Fast. Accurate. Sample-efficient. Thanks for sharing, Michael Posa ! 📍Watch it in action: Paper:
0:46

How precise can a robot get… without ever seeing the object move? Turns out, just a geometric model is enough. A robot hit 73 precise 3D goals in a row, using only a geometric model. No demonstrations. No imitation learning. No fancy vision models. The secret? Sample where the robot could make contact. Let local contact-implicit control take over from there. That’s it. ✅ Real-time performance ✅ Natural contact making and breaking ✅ Works on tricky tasks like pushing and rotating This approach mixes global planning with local MPC. Fast. Accurate. Sample-efficient. Thanks for sharing, Michael Posa ! 📍Watch it in action: Paper:

Ilir Aliu

19,012 Aufrufe • vor 11 Monaten

Scientists developed HumanX, a system that lets humanoid robots learn new skills by watching just one short human video. It learns directly from demonstrations instead of large datasets or complex training rewards. Using this, the Unitree G1 humanoid robot learned 10 different real-world skills without extra training.
0:34

Scientists developed HumanX, a system that lets humanoid robots learn new skills by watching just one short human video. It learns directly from demonstrations instead of large datasets or complex training rewards. Using this, the Unitree G1 humanoid robot learned 10 different real-world skills without extra training.

Space and Technology

27,686 Aufrufe • vor 2 Monaten

🚀 We just shipped a big upgrade to our imitation-learning-in-simulation playground in LeRobot, built together with Lightwheel ! You can now teleoperate robots in sim (keyboard or real robot) and collect training demos instantly. This makes it possible to run real IL research on harder, more realistic manipulation tasks, even if you don’t have hardware. New tasks: 🟠 pick orange to the plate 🧺 fold cloth (yes!) 📦 pick 2 “e” toys to the box 🔴 lift red cube Through our partnership with Lightwheel, LeIsaac was integrated into EnvHub on day one, a best-practice integration that strengthens both ecosystems and pushes simulation-first robotics forward. 👉 Load a task via EnvHub, start teleop-ing, record your demo, upload the data on the hub and you’re ready to train or test.
0:22

🚀 We just shipped a big upgrade to our imitation-learning-in-simulation playground in LeRobot, built together with Lightwheel ! You can now teleoperate robots in sim (keyboard or real robot) and collect training demos instantly. This makes it possible to run real IL research on harder, more realistic manipulation tasks, even if you don’t have hardware. New tasks: 🟠 pick orange to the plate 🧺 fold cloth (yes!) 📦 pick 2 “e” toys to the box 🔴 lift red cube Through our partnership with Lightwheel, LeIsaac was integrated into EnvHub on day one, a best-practice integration that strengthens both ecosystems and pushes simulation-first robotics forward. 👉 Load a task via EnvHub, start teleop-ing, record your demo, upload the data on the hub and you’re ready to train or test.

LeRobot

44,647 Aufrufe • vor 6 Monaten

A new robot policy just cleaned up a kitchen it had never seen before [watch what happens. paper included ⬇️] Pi-0.5 builds on top of Pi-0 and shows how smart co-training with diverse data can unlock real generalization in the home. It doesn’t just learn from one setup but adapts to many, including homes it’s never seen. What it does ✅ Handles new homes without training in them ✅ Follows complex language instructions ✅ Cleans, places dishes, handles spills ✅ Matches in-home training models using cross-embodiment and web data Robots that understand tasks and adapt to new spaces are finally within reach. More in the blog: Read the paper ⬇️ Physical Intelligence, co-founded by UC Berkeley professor Sergey Levine, is a robotics startup developing general-purpose AI foundation models that enable robots to perform a wide variety of real-world tasks with human-like adaptability, recently raising $400 million to advance this vision.
1:22

A new robot policy just cleaned up a kitchen it had never seen before [watch what happens. paper included ⬇️] Pi-0.5 builds on top of Pi-0 and shows how smart co-training with diverse data can unlock real generalization in the home. It doesn’t just learn from one setup but adapts to many, including homes it’s never seen. What it does ✅ Handles new homes without training in them ✅ Follows complex language instructions ✅ Cleans, places dishes, handles spills ✅ Matches in-home training models using cross-embodiment and web data Robots that understand tasks and adapt to new spaces are finally within reach. More in the blog: Read the paper ⬇️ Physical Intelligence, co-founded by UC Berkeley professor Sergey Levine, is a robotics startup developing general-purpose AI foundation models that enable robots to perform a wide variety of real-world tasks with human-like adaptability, recently raising $400 million to advance this vision.

Ilir Aliu - eu/acc

18,120 Aufrufe • vor 1 Jahr

🚨🇺🇸FIGURE ROBOT LEARNS TO WALK IN HOURS Turns out you can teach a robot to walk—if you’ve got a few years to burn in a physics simulator (or a few hours of GPU time). Figure’s team trained a neural network using reinforcement learning to mimic natural human locomotion, and the wild part is it worked on the first try with real hardware. No tuning. No fiddling. Just plug and strut. Thanks to sim-to-real transfer and some sneaky domain randomization, the robot didn’t just wobble out of the lab—it walked like it had somewhere to be. Next up: running. Not terrifying at all. Source: Figure
0:54

🚨🇺🇸FIGURE ROBOT LEARNS TO WALK IN HOURS Turns out you can teach a robot to walk—if you’ve got a few years to burn in a physics simulator (or a few hours of GPU time). Figure’s team trained a neural network using reinforcement learning to mimic natural human locomotion, and the wild part is it worked on the first try with real hardware. No tuning. No fiddling. Just plug and strut. Thanks to sim-to-real transfer and some sneaky domain randomization, the robot didn’t just wobble out of the lab—it walked like it had somewhere to be. Next up: running. Not terrifying at all. Source: Figure

Mario Nawfal

117,866 Aufrufe • vor 1 Jahr

Force is arguably the most overlooked ingredient in modern robot learning. Introducing FACTR 2: it turns *any* commodity robot into a force-aware system with no force sensors required. Train a tiny force network in <1min with <10mins of data and drop it into any existing teleop pipelines: ✅ Free force sensing for both the robot and the operator arm ✅ Makes demos higher-quality → fewer of them needed. ✅ A new force-aware learning algorithm (FIRST) uses those recovered forces to figure out which parts of a demo actually matter, making learning data-efficient. ✅ Strong performance on complex tasks with fewer demos and even no pretraining! More details below.
0:40

Force is arguably the most overlooked ingredient in modern robot learning. Introducing FACTR 2: it turns *any* commodity robot into a force-aware system with no force sensors required. Train a tiny force network in <1min with <10mins of data and drop it into any existing teleop pipelines: ✅ Free force sensing for both the robot and the operator arm ✅ Makes demos higher-quality → fewer of them needed. ✅ A new force-aware learning algorithm (FIRST) uses those recovered forces to figure out which parts of a demo actually matter, making learning data-efficient. ✅ Strong performance on complex tasks with fewer demos and even no pretraining! More details below.

Deepak Pathak

37,741 Aufrufe • vor 10 Tagen

2. NVIDIA is building the GPT of humanoid robots. They just launched Isaac GR00T N1.5 - a foundation model for general purpose robotics. Here’s how it works: → A human demos the task once → Cosmos (their physics AI model) generates 1,000s of variations → Omniverse simulates the motions in high fidelity → The robot trains entirely in simulation → Then fine-tunes itself in the real world Robots can now learn general skills across tasks, tools, even body types with just one human demo. AI isn’t just thinking in text anymore. It’s perceiving. Reasoning. Moving. Physical AI is here and it’s training itself.
2:21

2. NVIDIA is building the GPT of humanoid robots. They just launched Isaac GR00T N1.5 - a foundation model for general purpose robotics. Here’s how it works: → A human demos the task once → Cosmos (their physics AI model) generates 1,000s of variations → Omniverse simulates the motions in high fidelity → The robot trains entirely in simulation → Then fine-tunes itself in the real world Robots can now learn general skills across tasks, tools, even body types with just one human demo. AI isn’t just thinking in text anymore. It’s perceiving. Reasoning. Moving. Physical AI is here and it’s training itself.

Shruti

70,359 Aufrufe • vor 1 Jahr

Everyone is scaling VLAs with more robot data. TiPToP shows another path. No robot training, no policy learning. Just RGB + language → 3D scene → GPU TAMP planner → trajectory. Foundation models + planning alone can run real manipulation tasks.
1:38

Everyone is scaling VLAs with more robot data. TiPToP shows another path. No robot training, no policy learning. Just RGB + language → 3D scene → GPU TAMP planner → trajectory. Foundation models + planning alone can run real manipulation tasks.

Robots Digest 🤖

10,362 Aufrufe • vor 3 Monaten

A robot that works with 1 to 6 legs? This reconfigurable robot adapts like a spider, switching between different movement styles on the fly. SnapBot isn’t just a legged robot: it’s a shape-shifting machine. With up to 700 configurations, it can walk, crawl, or row its way through any terrain. Why this is exciting: ✅ Changes its setup from one to six legs for different tasks ✅ Moves in multiple ways, including walking, rowing, and crawling ✅ Adapts to its environment like nature’s best climbers and crawlers ✅ A step toward robots that adjust to real-world challenges instantly Nature-inspired robotics is unlocking new ways to move. SnapBot proves that flexibility is key to the future of locomotion.
0:25

A robot that works with 1 to 6 legs? This reconfigurable robot adapts like a spider, switching between different movement styles on the fly. SnapBot isn’t just a legged robot: it’s a shape-shifting machine. With up to 700 configurations, it can walk, crawl, or row its way through any terrain. Why this is exciting: ✅ Changes its setup from one to six legs for different tasks ✅ Moves in multiple ways, including walking, rowing, and crawling ✅ Adapts to its environment like nature’s best climbers and crawlers ✅ A step toward robots that adjust to real-world challenges instantly Nature-inspired robotics is unlocking new ways to move. SnapBot proves that flexibility is key to the future of locomotion.

Ilir Aliu

77,991 Aufrufe • vor 1 Jahr

In order for robots to be deployed in the real world, performing tasks of real value, they must be reliable. Unfortunately, even more, most robotic demos work maybe 70-80% of the time at best. The way to get better reliability is to do real-world reinforcement learning: having the robot teach itself how to perform the task up to a high level of success. The key to doing this is to start with a core of expert human data, use that to train a policy then iteratively improve it, until finally finishing with on-policy reinforcement learning. Kun Lei talks through a unified framework for imitation and reinforcement learning based on PPO, which enables this improvement process. In this episode, Kun Lei explains the theory behind his reinforcement learning method and how it allowed his robot to run in a shopping mall juicing oranges for seven hours at a time, among experiments on a wide variety of tasks and embodiments. Watch episode 58 of RoboPapers now, hosted by Michael Cho - Rbt/Acc and Chris Paxton!
1:11:51

In order for robots to be deployed in the real world, performing tasks of real value, they must be reliable. Unfortunately, even more, most robotic demos work maybe 70-80% of the time at best. The way to get better reliability is to do real-world reinforcement learning: having the robot teach itself how to perform the task up to a high level of success. The key to doing this is to start with a core of expert human data, use that to train a policy then iteratively improve it, until finally finishing with on-policy reinforcement learning. Kun Lei talks through a unified framework for imitation and reinforcement learning based on PPO, which enables this improvement process. In this episode, Kun Lei explains the theory behind his reinforcement learning method and how it allowed his robot to run in a shopping mall juicing oranges for seven hours at a time, among experiments on a wide variety of tasks and embodiments. Watch episode 58 of RoboPapers now, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

RoboPapers

18,541 Aufrufe • vor 5 Monaten

This week, Amazon introduced Vulcan! 🌋 Amazon has created a new robot called Vulcan, and it's the first one with a real sense of touch. I visited one of the fulfillment centers of Amazon to shoot their latest picking robot for you! Vulcan doesn’t just “see” like most robots—it can also feel what it’s touching, like a human hand. 🪬 This helps it do tricky tasks, such as picking items out of crowded bins, without damaging anything nearby. Vulcan is powered by physical AI, which means it uses both sight and touch to understand and move through the world. In Amazon’s warehouses, items are stored tightly in small compartments, which used to be hard for robots to handle. Their new system with Universal Robots arms changes that, it can gently pick up or place items in these tight spaces by knowing how much pressure to apply. The robot uses special tools and sensors to know when it touches something and how hard it’s pressing. 🤏🏼 It helps make the workplace safer by doing tasks that used to require employees to use ladders or reach overhead. Over time, Vulcan will become even smarter by learning from its mistakes and gaining more experience, just like a person does.
0:49

This week, Amazon introduced Vulcan! 🌋 Amazon has created a new robot called Vulcan, and it's the first one with a real sense of touch. I visited one of the fulfillment centers of Amazon to shoot their latest picking robot for you! Vulcan doesn’t just “see” like most robots—it can also feel what it’s touching, like a human hand. 🪬 This helps it do tricky tasks, such as picking items out of crowded bins, without damaging anything nearby. Vulcan is powered by physical AI, which means it uses both sight and touch to understand and move through the world. In Amazon’s warehouses, items are stored tightly in small compartments, which used to be hard for robots to handle. Their new system with Universal Robots arms changes that, it can gently pick up or place items in these tight spaces by knowing how much pressure to apply. The robot uses special tools and sensors to know when it touches something and how hard it’s pressing. 🤏🏼 It helps make the workplace safer by doing tasks that used to require employees to use ladders or reach overhead. Over time, Vulcan will become even smarter by learning from its mistakes and gaining more experience, just like a person does.

Lukas Ziegler

16,413 Aufrufe • vor 1 Jahr

🤖What if a robot could perform a new task just from a natural language command, with zero demonstrations? Our new work, NovaFlow, makes it possible! We use pre-trained video generative model to create a video of the task, then translate it into a plan for real-world robot execution. 1/6 #Robotics #AI #ZeroShot #Manipulation
0:27

🤖What if a robot could perform a new task just from a natural language command, with zero demonstrations? Our new work, NovaFlow, makes it possible! We use pre-trained video generative model to create a video of the task, then translate it into a plan for real-world robot execution. 1/6 #Robotics #AI #ZeroShot #Manipulation

Hongyu Li

105,428 Aufrufe • vor 8 Monaten