Загрузка видео...

Не удалось загрузить видео

На главную

At #ICRA2025, I'm about to present: "R+X: Retrieval and Execution from Everyday Human Videos". By using a VLM for retrieval and in-context IL for execution, robots can now learn from just videos of humans! Led by George Papagiannis and Norman Di Palo. See:

Ed__Johns's profile picture

Edward Johns

3,811 subscribers

11,962 просмотров • 1 год назад •via X (Twitter)

#ICRA2025
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 3

Фото профиля Phil Trubey
Phil Trubey1 год назад

@geopgs @normandipalo Nice! Can you learn multi step actions. Like open a box, then stack cups, then put them into the box, then close the box.

Фото профиля The Rundown AI
The Rundown AI2 лет назад

If you're not learning AI in 2024, you're falling behind. Join 500,000+ readers and learn how to use AI in just 5 minutes a day (for free).

Фото профиля Michael Neuvirth
Michael Neuvirth1 год назад

@geopgs @normandipalo This is very useful. Thank you

Похожие видео

Researchers at RAI Institute developed the Koala gripper to help robots handle tools and objects more naturally. The system includes a handheld version for humans and a motorized version for robots, both using the same design and sensors. By learning from human demonstrations recorded with cameras and force sensors, robots can repeat the same tasks using the Koala gripper.
0:24

Researchers at RAI Institute developed the Koala gripper to help robots handle tools and objects more naturally. The system includes a handheld version for humans and a motorized version for robots, both using the same design and sensors. By learning from human demonstrations recorded with cameras and force sensors, robots can repeat the same tasks using the Koala gripper.

Space and Technology

21,980 просмотров • 1 месяц назад

How can #robots remember? 🤖 💭 For robots to understand and respond to questions that require complex multi-step reasoning in scenarios over long periods of time, we built ReMEmbR, a retrieval-augmented memory for embodied robots. 👀 Technical deep dive from #NVIDIAResearch ➡️
1:36

How can #robots remember? 🤖 💭 For robots to understand and respond to questions that require complex multi-step reasoning in scenarios over long periods of time, we built ReMEmbR, a retrieval-augmented memory for embodied robots. 👀 Technical deep dive from #NVIDIAResearch ➡️

NVIDIA AI Developer

30,485 просмотров • 1 год назад

Using computer vision techniques, the robot analyzed each frame of video and was able to identify the different objects and features. In the future, robots will be able to learn human behaviors just by watching videos.
1:03

Using computer vision techniques, the robot analyzed each frame of video and was able to identify the different objects and features. In the future, robots will be able to learn human behaviors just by watching videos.

Aakash Gupta

55,257 просмотров • 3 лет назад

A fully scoped project. Built from what your workspace already knows about your product and customers. Learn how to use Linear Agent to turn context into execution by drafting project requirements, suggesting milestones, and organizing issues.
2:27

A fully scoped project. Built from what your workspace already knows about your product and customers. Learn how to use Linear Agent to turn context into execution by drafting project requirements, suggesting milestones, and organizing issues.

Linear

75,740 просмотров • 3 месяцев назад

Exa CEO Will Bryk explains why retrieval can help solve the tokenpocalypse: "We should not be using gigantic models for every task." "You should use a family of models of different sizes. The big model decides what to do, and it dishes out commands to the small models, and those small models can be way more accurate and reliable if they're using retrieval." "Retrieval helps small models act like big models... We do save our customers a huge amount of tokens because they can use smaller models and use retrieval." "We could save 20x on cost for customers compared to other providers by being very efficient in what information from the web the agent actually sees." Will Bryk with Sarah Wang
1:21

Exa CEO Will Bryk explains why retrieval can help solve the tokenpocalypse: "We should not be using gigantic models for every task." "You should use a family of models of different sizes. The big model decides what to do, and it dishes out commands to the small models, and those small models can be way more accurate and reliable if they're using retrieval." "Retrieval helps small models act like big models... We do save our customers a huge amount of tokens because they can use smaller models and use retrieval." "We could save 20x on cost for customers compared to other providers by being very efficient in what information from the web the agent actually sees." Will Bryk with Sarah Wang

a16z

25,144 просмотров • 1 месяц назад

This is big. The Retrieval Plugin allows ChatGPT to have a memory! The model can now remember information from conversations and store it in the retrieval plugin for later use. This feature is a must if you want to develop GPT-based tools.
0:38

This is big. The Retrieval Plugin allows ChatGPT to have a memory! The model can now remember information from conversations and store it in the retrieval plugin for later use. This feature is a must if you want to develop GPT-based tools.

Lior Alexander

227,586 просмотров • 3 лет назад

The AutoFi stack is a turning point for crypto. It's a radical shift in how we monetize execution and generate revenue for our ecosystem. Learn all about it at the keynote by Josh Tobkin (SUPRA) on June 24, live from Permissionless Wealth (自由财富) IV by
5:15

The AutoFi stack is a turning point for crypto. It's a radical shift in how we monetize execution and generate revenue for our ecosystem. Learn all about it at the keynote by Josh Tobkin (SUPRA) on June 24, live from Permissionless Wealth (自由财富) IV by

Supra

10,714 просмотров • 1 год назад

The most frustrating part of imitation learning is collecting huge amounts of teleop data. But why teleop robots when robots can learn by watching us? Introducing Point Policy, a novel framework that enables robots to learn from human videos without any teleop, sim2real, or RL.
0:40

The most frustrating part of imitation learning is collecting huge amounts of teleop data. But why teleop robots when robots can learn by watching us? Introducing Point Policy, a novel framework that enables robots to learn from human videos without any teleop, sim2real, or RL.

Siddhant Haldar

69,056 просмотров • 1 год назад

I spent the past year building AI for robots at Tesla Optimus and Dyna Now I'm introducing ANDROID DREAMS: an essay of my predictions for the next 20 years of robotics, inspired by Situational Awareness and AI2027. I predict EGI by 2031 and more robots than humans by 2045👇🏼
0:24

I spent the past year building AI for robots at Tesla Optimus and Dyna Now I'm introducing ANDROID DREAMS: an essay of my predictions for the next 20 years of robotics, inspired by Situational Awareness and AI2027. I predict EGI by 2031 and more robots than humans by 2045👇🏼

anandmaj

345,443 просмотров • 8 месяцев назад

New short course on advanced retrieval for RAG (retrieval augmented generation)! RAG fetches relevant documents to give context to an LLM. In Advanced Retrieval for AI with Chroma, taught by Chroma founder anton 🇺🇸, you’ll learn: (i) Query expansion using an LLM to rewrite and improve a query, by either generating either additional relevant queries or a hypothetical answer to the query. (ii) Reranking using a cross-encoder - a model trained to measure similarity between two inputs presented simultaneously. Reranking reorders retrieved documents based on the cross-encoder similarity measure. (iii) Constructing and training an Embedding Adaptor, which is a model that adapts the embedding values to be more relevant to your use case. Each of these techniques can help you build much better RAG systems. Please sign up for the course here:
2:36

New short course on advanced retrieval for RAG (retrieval augmented generation)! RAG fetches relevant documents to give context to an LLM. In Advanced Retrieval for AI with Chroma, taught by Chroma founder anton 🇺🇸, you’ll learn: (i) Query expansion using an LLM to rewrite and improve a query, by either generating either additional relevant queries or a hypothetical answer to the query. (ii) Reranking using a cross-encoder - a model trained to measure similarity between two inputs presented simultaneously. Reranking reorders retrieved documents based on the cross-encoder similarity measure. (iii) Constructing and training an Embedding Adaptor, which is a model that adapts the embedding values to be more relevant to your use case. Each of these techniques can help you build much better RAG systems. Please sign up for the course here:

Andrew Ng

191,219 просмотров • 2 лет назад

Teaching robots to perform dexterous manipulation tasks currently requires teleoperation, which limits demonstration quality, speed, and scalability. Instead, why not use human videos? The problem is that a human hand isn’t a robot hand, so data must be retargeted using simulation to resolve issues like collisions and interpenetration when controlling the hand. In VideoManip, Hongyi Chen and co-authors built a system to solve this problem, taking in RGB videos of humans performing manipulation tasks and using them to create accurate simulations with which to learn robot policies. Watch episode #73 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton, now to learn more!
44:20

Teaching robots to perform dexterous manipulation tasks currently requires teleoperation, which limits demonstration quality, speed, and scalability. Instead, why not use human videos? The problem is that a human hand isn’t a robot hand, so data must be retargeted using simulation to resolve issues like collisions and interpenetration when controlling the hand. In VideoManip, Hongyi Chen and co-authors built a system to solve this problem, taking in RGB videos of humans performing manipulation tasks and using them to create accurate simulations with which to learn robot policies. Watch episode #73 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton, now to learn more!

RoboPapers

27,249 просмотров • 3 месяцев назад

Engineers have built a robotic face that can learn to move its lips in sync with speech and singing by watching itself in a mirror and then observing humans in online videos, aiming to make humanoid robots appear less 'uncanny' in face-to-face interaction
2:47

Engineers have built a robotic face that can learn to move its lips in sync with speech and singing by watching itself in a mirror and then observing humans in online videos, aiming to make humanoid robots appear less 'uncanny' in face-to-face interaction

Reuters

27,638 просмотров • 5 месяцев назад

🚨Breaking: Google just dropped a new Guided Learning feature in Gemini. Perfect for students instead of just giving answers, it actually teaches. You can get images, diagrams, videos, and quizzes to learn step by step and see how much you’ve understood.
0:36

🚨Breaking: Google just dropped a new Guided Learning feature in Gemini. Perfect for students instead of just giving answers, it actually teaches. You can get images, diagrams, videos, and quizzes to learn step by step and see how much you’ve understood.

AshutoshShrivastava

79,363 просмотров • 11 месяцев назад

RF-DETR by Roboflow now on MLX It can do realtime instance segmentation on-device and enable some cool use cases for visual analysis, monitoring and robotics like Reachy Mini. Also augmented VLM and VLA by preprocessing image and video with areas of interest. New release coming soon on mlx-vlm 🚀 For those who can’t wait you can install mlx-vlm from source.
1:16

RF-DETR by Roboflow now on MLX It can do realtime instance segmentation on-device and enable some cool use cases for visual analysis, monitoring and robotics like Reachy Mini. Also augmented VLM and VLA by preprocessing image and video with areas of interest. New release coming soon on mlx-vlm 🚀 For those who can’t wait you can install mlx-vlm from source.

Prince Canuma

30,499 просмотров • 3 месяцев назад

We discovered an emergent property of VLAs like π0/π0.5/π0.6: as we scale up pre-training, the model learns to align human videos and robot data! This gives us a simple way to leverage human videos. Once π0.5 knows how to control robots, it can naturally learn from human video.
2:02

We discovered an emergent property of VLAs like π0/π0.5/π0.6: as we scale up pre-training, the model learns to align human videos and robot data! This gives us a simple way to leverage human videos. Once π0.5 knows how to control robots, it can naturally learn from human video.

Physical Intelligence

1,183,087 просмотров • 7 месяцев назад

Excited to announce that we have raised $2.3M led by Sam Altman Peak XV Partners and an incredible set of investors for Induced We let anyone create virtual AI workers that can automate the execution of workflows on a browser in the cloud with human-like reasoning.
0:26

Excited to announce that we have raised $2.3M led by Sam Altman Peak XV Partners and an incredible set of investors for Induced We let anyone create virtual AI workers that can automate the execution of workflows on a browser in the cloud with human-like reasoning.

Aryan Sharma

465,500 просмотров • 2 лет назад

Learning dexterous policies from human videos is challenging due to differences between human and robot hands. We present HuDOR, a method that learns dexterous policies within the robot's physical constraints using just one human video and an hour of online interactions! [1/n]
1:48

Learning dexterous policies from human videos is challenging due to differences between human and robot hands. We present HuDOR, a method that learns dexterous policies within the robot's physical constraints using just one human video and an hour of online interactions! [1/n]

Irmak Guzey

65,121 просмотров • 1 год назад

Just collecting manipulation data isn’t enough for robots - they need to be able to move around in the world, which has a whole different set of challenges from pure manipulation. And bringing navigation and manipulation together in a single framework is even more challenging. Enter HERMES, from Zhecheng Yuan and Tianming Wei. This is a four-stage process in which human videos are used to set up an RL sim-to-real training pipeline in order to overcome differences between robot and human kinematics, and used together with a navigation foundation model to move around in a variety of environments. To learn more, join us as Zhecheng Yuan and Tianming Wei tell us about how they built their system to perform mobile dexterous manipulation from human videos in a variety of environments. Watch Episode #45 of RoboPapers today, hosted by Michael Cho - Rbt/Acc and Chris Paxton!
1:01:08

Just collecting manipulation data isn’t enough for robots - they need to be able to move around in the world, which has a whole different set of challenges from pure manipulation. And bringing navigation and manipulation together in a single framework is even more challenging. Enter HERMES, from Zhecheng Yuan and Tianming Wei. This is a four-stage process in which human videos are used to set up an RL sim-to-real training pipeline in order to overcome differences between robot and human kinematics, and used together with a navigation foundation model to move around in a variety of environments. To learn more, join us as Zhecheng Yuan and Tianming Wei tell us about how they built their system to perform mobile dexterous manipulation from human videos in a variety of environments. Watch Episode #45 of RoboPapers today, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

RoboPapers

15,085 просмотров • 7 месяцев назад