正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

R+X was accepted at ICRA 2025! Robots can now do in-context imitation learning, just by observing humans going about their daily lives... No more need to label and train - just RETRIEVE and EXECUTE! 🧵👇 (1/5)

Edward Johns

3,870 subscribers

10,402 次观看 • 1 年前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

6 条评论

Edward Johns 的头像

Edward Johns1 年前

Given a language command at deployment: (1) Use Gemini to retrieve all the relevant human videos and their human hand trajectories, (2) Condition our in-context IL method (KAT) on these trajectories, (3) Predict the robot hand trajectories, and execute! (2/5)

Edward Johns 的头像

Edward Johns1 年前

Importantly: (1) We don't require the human videos to be labelled. (2) Since we do in-context imitation learning "at test time" rather than training an explicit policy, new human videos can easily be added on the fly and used immediately for retrieval. This is very scalable! (3/5)

Edward Johns 的头像

Edward Johns1 年前

We also found that this "Retrieval + Execution" idea performs much better than training an explicit, language-conditioned policy, such as when fine-tuning R3M or Octo on this same dataset of human videos. (4/5)

Edward Johns 的头像

Edward Johns1 年前

R+X was jointly led by Georgios Papagiannis (@geopgs) and Norman Di Palo (@normandipalo). For the paper and further videos, please visit: Thanks for reading! (5/5)

The Rundown AI 的头像

The Rundown AI2 年前

If you're not learning AI in 2024, you're falling behind. Join 500,000+ readers and learn how to use AI in just 5 minutes a day (for free).

Michael Cho - Rbt/Acc 的头像

Michael Cho - Rbt/Acc1 年前

Super cool work! 2 qn: must the human videos be recorded at the same environment where the physical robots do in-context learning at inference? Also, how important is the Depth info (saw that u guys are using rgbd cameras in this setup)?

相关视频

At #ICRA2025, I'm about to present: "R+X: Retrieval and Execution from Everyday Human Videos". By using a VLM for retrieval and in-context IL for execution, robots can now learn from just videos of humans! Led by George Papagiannis and Norman Di Palo. See:

At #ICRA2025, I'm about to present: "R+X: Retrieval and Execution from Everyday Human Videos". By using a VLM for retrieval and in-context IL for execution, robots can now learn from just videos of humans! Led by George Papagiannis and Norman Di Palo. See:

Edward Johns

11,962 次观看 • 1 年前

Very excited to announce: Keypoint Action Tokens! We found that LLMs can be repurposed as "imitation learning engines" for robots, by representing both observations & actions as 3D keypoints, and feeding into an LLM for in-context learning. See: More 👇

Very excited to announce: Keypoint Action Tokens! We found that LLMs can be repurposed as "imitation learning engines" for robots, by representing both observations & actions as 3D keypoints, and feeding into an LLM for in-context learning. See: More 👇

Edward Johns

32,569 次观看 • 2 年前

This is a single uncut video, showing a robot learning several tasks instantly, after just one demonstration each ... This is possible because we've now been able to achieve in-context learning for everyday robotics tasks, and I'm very excited to announce our latest paper: 🎆 Instant Policy: In-Context Imitation Learning via Graph Diffusion 🎆 (1/6) 🧵👇

This is a single uncut video, showing a robot learning several tasks instantly, after just one demonstration each ... This is possible because we've now been able to achieve in-context learning for everyday robotics tasks, and I'm very excited to announce our latest paper: 🎆 Instant Policy: In-Context Imitation Learning via Graph Diffusion 🎆 (1/6) 🧵👇

Edward Johns

74,683 次观看 • 1 年前

The robot is learning several novel tasks instantly, after just ONE demonstration each... Instant Policy makes it possible: no extra training, no weight updates, just pure in-context learning. It just got accepted at ICLR 2025, and it’s changing how robots learn. With just a single demo, a robot can pick up a new task and start performing it right away. Why this is a big deal: ✅ Learns tasks instantly with just one or a few demonstrations ✅ Improves over time as more demonstrations are given ✅ Uses simulation-based training with “pseudo-demonstrations” for scalability ✅ Can transfer skills across different robots and even follow language-defined tasks It brings in-context learning to robotics, opening up new possibilities for flexible, real-world automation. You can try it yourself: code and weights are available at • • • • Thank you to Edward Johns, Director of the Robot Learning Lab at Imperial College for sharing their work! 🙏

The robot is learning several novel tasks instantly, after just ONE demonstration each... Instant Policy makes it possible: no extra training, no weight updates, just pure in-context learning. It just got accepted at ICLR 2025, and it’s changing how robots learn. With just a single demo, a robot can pick up a new task and start performing it right away. Why this is a big deal: ✅ Learns tasks instantly with just one or a few demonstrations ✅ Improves over time as more demonstrations are given ✅ Uses simulation-based training with “pseudo-demonstrations” for scalability ✅ Can transfer skills across different robots and even follow language-defined tasks It brings in-context learning to robotics, opening up new possibilities for flexible, real-world automation. You can try it yourself: code and weights are available at • • • • Thank you to Edward Johns, Director of the Robot Learning Lab at Imperial College for sharing their work! 🙏

Ilir Aliu

46,285 次观看 • 1 年前

Lots of people think it's obvious that humans learn by imitation, starting with babies imitating their parents. Richard Sutton thinks this is completely wrong. Sutton basically thinks direct imitation plays almost no role in animal learning, and at most a small role in human intelligence. When one animal ends up doing the same thing as another, he thinks this is the result of RL-style trial-and-error, not direct imitation. And if imitation isn't really a kind of learning, then training LLMs to imitate humans isn't really Bitter Lesson–pilled.

Lots of people think it's obvious that humans learn by imitation, starting with babies imitating their parents. Richard Sutton thinks this is completely wrong. Sutton basically thinks direct imitation plays almost no role in animal learning, and at most a small role in human intelligence. When one animal ends up doing the same thing as another, he thinks this is the result of RL-style trial-and-error, not direct imitation. And if imitation isn't really a kind of learning, then training LLMs to imitate humans isn't really Bitter Lesson–pilled.

Dwarkesh Patel

55,137 次观看 • 1 个月前

Robots need to be able to apply pressure and make contact with objects as needed in order to accomplish their tasks. From compliance to working safely around humans to whole-body manipulation of heavy objects, combining force and position control can dramatically expand the capabilities of robots. This is especially true for legged robots, which have so much ability to exert forces on the world around them. But how do we train robots which can do this? Baoxiong Jia tells us more in our discussion of his team’s recent, Best Paper Award winning work on learning a unified policy for position and force control, called UniFP. To learn more, watch Episode #49 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton.

Robots need to be able to apply pressure and make contact with objects as needed in order to accomplish their tasks. From compliance to working safely around humans to whole-body manipulation of heavy objects, combining force and position control can dramatically expand the capabilities of robots. This is especially true for legged robots, which have so much ability to exert forces on the world around them. But how do we train robots which can do this? Baoxiong Jia tells us more in our discussion of his team’s recent, Best Paper Award winning work on learning a unified policy for position and force control, called UniFP. To learn more, watch Episode #49 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton.

RoboPapers

44,803 次观看 • 8 个月前

Derrick Henry on how the Ravens offense can improve its overall execution: "What do we need to do as an offense? Play better. Execute. I mean, I don't know what else to say. We just need to play better, and everybody just do their job. It takes all 11 to be able to execute, put plays together, put drives together and get more points." via Baltimore Ravens

Derrick Henry on how the Ravens offense can improve its overall execution: "What do we need to do as an offense? Play better. Execute. I mean, I don't know what else to say. We just need to play better, and everybody just do their job. It takes all 11 to be able to execute, put plays together, put drives together and get more points." via Baltimore Ravens

Bobby Trosset

204,165 次观看 • 8 个月前

🤖ROBOTS ARE GETTING SMARTER AT TOUCHING THE REAL WORLD Researchers from UC Berkeley, NVIDIA and Stanford introduced T-Rex, a framework that combines vision, language and tactile sensing. Instead of relying on cameras alone, robots can now respond to physical contact in real time. Robots are no longer just seeing objects. They’re learning how to FEEL them.

🤖ROBOTS ARE GETTING SMARTER AT TOUCHING THE REAL WORLD Researchers from UC Berkeley, NVIDIA and Stanford introduced T-Rex, a framework that combines vision, language and tactile sensing. Instead of relying on cameras alone, robots can now respond to physical contact in real time. Robots are no longer just seeing objects. They’re learning how to FEEL them.

Coin Bureau

28,275 次观看 • 1 个月前

ELON: WILL OUR LIVES STILL HAVE MEANING IF COMPUTERS AND ROBOTS OUTPERFORM US? “At some point, as AI and robotics get better, eventually AI will be able to do everything that humans can do. Any task you do will be optional, like a hobby. It does beg the question, will our lives have meaning if the computers and the robots can do everything better than we can? That is a real question. I do wonder about that myself. That's maybe why we need the Neuralink, so we can enhance human capability, so we can keep up with the machines.” Source: Interview with Mark Penn at CES, January 8, 2025

ELON: WILL OUR LIVES STILL HAVE MEANING IF COMPUTERS AND ROBOTS OUTPERFORM US? “At some point, as AI and robotics get better, eventually AI will be able to do everything that humans can do. Any task you do will be optional, like a hobby. It does beg the question, will our lives have meaning if the computers and the robots can do everything better than we can? That is a real question. I do wonder about that myself. That's maybe why we need the Neuralink, so we can enhance human capability, so we can keep up with the machines.” Source: Interview with Mark Penn at CES, January 8, 2025

Mario Nawfal

329,978 次观看 • 1 年前

Robots and smart devices are about to get a massive upgrade. 🤖 Forget clunky GPS or special markers. We’re talking about machines that can understand their exact position & orientation in the real world just by SEEING IT. This is the future of machine perception, and it's happening now. 👇

Robots and smart devices are about to get a massive upgrade. 🤖 Forget clunky GPS or special markers. We’re talking about machines that can understand their exact position & orientation in the real world just by SEEING IT. This is the future of machine perception, and it's happening now. 👇

Over the Reality 🌐

266,553 次观看 • 10 个月前

TXT is clearly overworked, since their debut they have not had a long break and in recent months their fatigue began to be noticed accompanied by injuries and illnesses, and now with Soobin's hiatus this can no longer continue, THEY ARE HUMANS, NOT ROBOTS #BighitLetTXTRest

TXT is clearly overworked, since their debut they have not had a long break and in recent months their fatigue began to be noticed accompanied by injuries and illnesses, and now with Soobin's hiatus this can no longer continue, THEY ARE HUMANS, NOT ROBOTS #BighitLetTXTRest

maggie 𖣂 7TH YEAR

41,370 次观看 • 1 年前

We just made blockchain dev ridiculously easier at Chromia | Power to the Public Now you can write Rell code automatically in Cursor (takes literally 1 min to set up) No more diving through docs. Just prompt. 🧵

We just made blockchain dev ridiculously easier at Chromia | Power to the Public Now you can write Rell code automatically in Cursor (takes literally 1 min to set up) No more diving through docs. Just prompt. 🧵

superoo7

14,824 次观看 • 1 年前

Sheik Ahmad Gumi: "Kidnapping Children From School is less Evil Than Going Into A Town And Ransacking It." “Kidnapping children from school is a lesser evil because, in the end, you can negotiate and now bandits are very careful about human lives. “Before, the mission of bandits was to go into a town, ransack it and kill people. By this, I can say our preaching is working and hopefully, we are coming to an end of banditry in Zamfara and other states. “Bandits are more careful about lives now and just want to do sensational attacks which would bring attention to themselves

Sheik Ahmad Gumi: "Kidnapping Children From School is less Evil Than Going Into A Town And Ransacking It." “Kidnapping children from school is a lesser evil because, in the end, you can negotiate and now bandits are very careful about human lives. “Before, the mission of bandits was to go into a town, ransack it and kill people. By this, I can say our preaching is working and hopefully, we are coming to an end of banditry in Zamfara and other states. “Bandits are more careful about lives now and just want to do sensational attacks which would bring attention to themselves

Ozor Ndi Ozor

860,525 次观看 • 9 个月前

How do you call someone 20mins when they’re at work and expect them to answer 20times??? And when they answer, how can your response be that… can we just agree that people need to get their lives together

How do you call someone 20mins when they’re at work and expect them to answer 20times??? And when they answer, how can your response be that… can we just agree that people need to get their lives together

Dr. God Abeg ooo

188,710 次观看 • 1 年前

BREAKING: Elon Musk just said what nobody in Washington wants to hear. "The government is basically unfixable." "If AI and robots don't solve our national debt. We're toast." $39,000,000,000,000 in national debt. Adding trillions every year. The man who built Tesla. SpaceX. And tried to fix the government himself. Just admitted it can't be fixed by humans. Only two things can save America now. AI and robots. Or bankruptcy. There is no third option.

BREAKING: Elon Musk just said what nobody in Washington wants to hear. "The government is basically unfixable." "If AI and robots don't solve our national debt. We're toast." $39,000,000,000,000 in national debt. Adding trillions every year. The man who built Tesla. SpaceX. And tried to fix the government himself. Just admitted it can't be fixed by humans. Only two things can save America now. AI and robots. Or bankruptcy. There is no third option.

Crypto Tice

584,141 次观看 • 2 个月前

ELON: EVERY OPTIMIST NEEDS AN OPTIMUS . . . OR TWO "Most people would like to have their own personal C-3PO R2-D2. I think humanoid robots will be the biggest product in history by far. You can just say every human is going to want one. Some will want two. There'll be all of the industry in terms of providing products and services. What's the ratio of humanoid robots to humans? My guess is it's at least three to one, four to one, maybe five to one. We're talking about 20, 30 billion humanoid robots." Source: Elon Musk, Mark Penn, January 9th 2025

ELON: EVERY OPTIMIST NEEDS AN OPTIMUS . . . OR TWO "Most people would like to have their own personal C-3PO R2-D2. I think humanoid robots will be the biggest product in history by far. You can just say every human is going to want one. Some will want two. There'll be all of the industry in terms of providing products and services. What's the ratio of humanoid robots to humans? My guess is it's at least three to one, four to one, maybe five to one. We're talking about 20, 30 billion humanoid robots." Source: Elon Musk, Mark Penn, January 9th 2025

Mario Nawfal

190,653 次观看 • 9 个月前

Trump: Now, we have no inflation. Gas just hit $1.99 in 5 states. It was up to $4 and going up to 5,6,7. In California, it was $7.70

Trump: Now, we have no inflation. Gas just hit $1.99 in 5 states. It was up to $4 and going up to 5,6,7. In California, it was $7.70

Acyn

3,035,517 次观看 • 1 年前

This one is all about control, strength, and mobility ✨🔥🔥 *Motivational / Strong* 1. *Strength + Mobility = Everything* No equipment, just bodyweight 2. From squat to handstand. Your body can do more than you think 3. Train to move, not just to look. 🔥🔥

This one is all about control, strength, and mobility ✨🔥🔥 Motivational / Strong 1. Strength + Mobility = Everything No equipment, just bodyweight 2. From squat to handstand. Your body can do more than you think 3. Train to move, not just to look. 🔥🔥

The_Fitness & Fitfoods

22,780 次观看 • 26 天前

🚨DO THIS NOW🚨 Many projects are rejoining us by changing their names. Thanks to Alphagate, you can learn about projects and people who have changed their names and bios What You Need to Do: 1⃣ Go to and register with your email address. 2⃣Install the plugin and log in. 3⃣Enter the X profile you want to review 🤍To reach more people, RT-Like

Coinisan

20,309 次观看 • 1 年前