Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

At #ICRA2025, I'm about to present: "R+X: Retrieval and Execution from Everyday Human Videos". By using a VLM for retrieval and in-context IL for execution, robots can now learn from just videos of humans! Led by George Papagiannis and Norman Di Palo. See:

Edward Johns

3,811 subscribers

11,962 görüntüleme • 1 yıl önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

3 Yorum

Phil Trubey profil fotoğrafı

Phil Trubey1 yıl önce

@geopgs @normandipalo Nice! Can you learn multi step actions. Like open a box, then stack cups, then put them into the box, then close the box.

The Rundown AI profil fotoğrafı

The Rundown AI2 yıl önce

If you're not learning AI in 2024, you're falling behind. Join 500,000+ readers and learn how to use AI in just 5 minutes a day (for free).

Michael Neuvirth profil fotoğrafı

Michael Neuvirth1 yıl önce

@geopgs @normandipalo This is very useful. Thank you

Benzer Videolar

R+X was accepted at ICRA 2025! Robots can now do in-context imitation learning, just by observing humans going about their daily lives... No more need to *label* and *train* - just *RETRIEVE* and *EXECUTE*! 🧵👇 (1/5)

R+X was accepted at ICRA 2025! Robots can now do in-context imitation learning, just by observing humans going about their daily lives... No more need to label and train - just RETRIEVE and EXECUTE! 🧵👇 (1/5)

Edward Johns

10,402 görüntüleme • 1 yıl önce

Researchers at RAI Institute developed the Koala gripper to help robots handle tools and objects more naturally. The system includes a handheld version for humans and a motorized version for robots, both using the same design and sensors. By learning from human demonstrations recorded with cameras and force sensors, robots can repeat the same tasks using the Koala gripper.

Researchers at RAI Institute developed the Koala gripper to help robots handle tools and objects more naturally. The system includes a handheld version for humans and a motorized version for robots, both using the same design and sensors. By learning from human demonstrations recorded with cameras and force sensors, robots can repeat the same tasks using the Koala gripper.

Space and Technology

22,161 görüntüleme • 3 ay önce

How can #robots remember? 🤖 💭 For robots to understand and respond to questions that require complex multi-step reasoning in scenarios over long periods of time, we built ReMEmbR, a retrieval-augmented memory for embodied robots. 👀 Technical deep dive from #NVIDIAResearch ➡️

How can #robots remember? 🤖 💭 For robots to understand and respond to questions that require complex multi-step reasoning in scenarios over long periods of time, we built ReMEmbR, a retrieval-augmented memory for embodied robots. 👀 Technical deep dive from #NVIDIAResearch ➡️

NVIDIA AI Developer

30,520 görüntüleme • 2 yıl önce

Using computer vision techniques, the robot analyzed each frame of video and was able to identify the different objects and features. In the future, robots will be able to learn human behaviors just by watching videos.

Using computer vision techniques, the robot analyzed each frame of video and was able to identify the different objects and features. In the future, robots will be able to learn human behaviors just by watching videos.

Aakash Gupta

55,274 görüntüleme • 3 yıl önce

A fully scoped project. Built from what your workspace already knows about your product and customers. Learn how to use Linear Agent to turn context into execution by drafting project requirements, suggesting milestones, and organizing issues.

A fully scoped project. Built from what your workspace already knows about your product and customers. Learn how to use Linear Agent to turn context into execution by drafting project requirements, suggesting milestones, and organizing issues.

Linear

75,941 görüntüleme • 6 ay önce

Exa CEO Will Bryk explains why retrieval can help solve the tokenpocalypse: "We should not be using gigantic models for every task." "You should use a family of models of different sizes. The big model decides what to do, and it dishes out commands to the small models, and those small models can be way more accurate and reliable if they're using retrieval." "Retrieval helps small models act like big models... We do save our customers a huge amount of tokens because they can use smaller models and use retrieval." "We could save 20x on cost for customers compared to other providers by being very efficient in what information from the web the agent actually sees." Will Bryk with Sarah Wang

Exa CEO Will Bryk explains why retrieval can help solve the tokenpocalypse: "We should not be using gigantic models for every task." "You should use a family of models of different sizes. The big model decides what to do, and it dishes out commands to the small models, and those small models can be way more accurate and reliable if they're using retrieval." "Retrieval helps small models act like big models... We do save our customers a huge amount of tokens because they can use smaller models and use retrieval." "We could save 20x on cost for customers compared to other providers by being very efficient in what information from the web the agent actually sees." Will Bryk with Sarah Wang

a16z

25,144 görüntüleme • 3 ay önce

This is big. The Retrieval Plugin allows ChatGPT to have a memory! The model can now remember information from conversations and store it in the retrieval plugin for later use. This feature is a must if you want to develop GPT-based tools.

This is big. The Retrieval Plugin allows ChatGPT to have a memory! The model can now remember information from conversations and store it in the retrieval plugin for later use. This feature is a must if you want to develop GPT-based tools.

Lior Alexander

227,586 görüntüleme • 3 yıl önce

The AutoFi stack is a turning point for crypto. It's a radical shift in how we monetize execution and generate revenue for our ecosystem. Learn all about it at the keynote by Josh Tobkin (SUPRA) on June 24, live from Permissionless Wealth (自由财富) IV by

The AutoFi stack is a turning point for crypto. It's a radical shift in how we monetize execution and generate revenue for our ecosystem. Learn all about it at the keynote by Josh Tobkin (SUPRA) on June 24, live from Permissionless Wealth (自由财富) IV by

Supra

10,714 görüntüleme • 1 yıl önce

The most frustrating part of imitation learning is collecting huge amounts of teleop data. But why teleop robots when robots can learn by watching us? Introducing Point Policy, a novel framework that enables robots to learn from human videos without any teleop, sim2real, or RL.

The most frustrating part of imitation learning is collecting huge amounts of teleop data. But why teleop robots when robots can learn by watching us? Introducing Point Policy, a novel framework that enables robots to learn from human videos without any teleop, sim2real, or RL.

Siddhant Haldar

69,056 görüntüleme • 1 yıl önce

I spent the past year building AI for robots at Tesla Optimus and Dyna Now I'm introducing ANDROID DREAMS: an essay of my predictions for the next 20 years of robotics, inspired by Situational Awareness and AI2027. I predict EGI by 2031 and more robots than humans by 2045👇🏼

I spent the past year building AI for robots at Tesla Optimus and Dyna Now I'm introducing ANDROID DREAMS: an essay of my predictions for the next 20 years of robotics, inspired by Situational Awareness and AI2027. I predict EGI by 2031 and more robots than humans by 2045👇🏼

anandmaj

346,308 görüntüleme • 10 ay önce

New short course on advanced retrieval for RAG (retrieval augmented generation)! RAG fetches relevant documents to give context to an LLM. In Advanced Retrieval for AI with Chroma, taught by Chroma founder anton 🇺🇸, you’ll learn: (i) Query expansion using an LLM to rewrite and improve a query, by either generating either additional relevant queries or a hypothetical answer to the query. (ii) Reranking using a cross-encoder - a model trained to measure similarity between two inputs presented simultaneously. Reranking reorders retrieved documents based on the cross-encoder similarity measure. (iii) Constructing and training an Embedding Adaptor, which is a model that adapts the embedding values to be more relevant to your use case. Each of these techniques can help you build much better RAG systems. Please sign up for the course here:

New short course on advanced retrieval for RAG (retrieval augmented generation)! RAG fetches relevant documents to give context to an LLM. In Advanced Retrieval for AI with Chroma, taught by Chroma founder anton 🇺🇸, you’ll learn: (i) Query expansion using an LLM to rewrite and improve a query, by either generating either additional relevant queries or a hypothetical answer to the query. (ii) Reranking using a cross-encoder - a model trained to measure similarity between two inputs presented simultaneously. Reranking reorders retrieved documents based on the cross-encoder similarity measure. (iii) Constructing and training an Embedding Adaptor, which is a model that adapts the embedding values to be more relevant to your use case. Each of these techniques can help you build much better RAG systems. Please sign up for the course here:

Andrew Ng

191,219 görüntüleme • 2 yıl önce

Teaching robots to perform dexterous manipulation tasks currently requires teleoperation, which limits demonstration quality, speed, and scalability. Instead, why not use human videos? The problem is that a human hand isn’t a robot hand, so data must be retargeted using simulation to resolve issues like collisions and interpenetration when controlling the hand. In VideoManip, Hongyi Chen and co-authors built a system to solve this problem, taking in RGB videos of humans performing manipulation tasks and using them to create accurate simulations with which to learn robot policies. Watch episode #73 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton, now to learn more!

Teaching robots to perform dexterous manipulation tasks currently requires teleoperation, which limits demonstration quality, speed, and scalability. Instead, why not use human videos? The problem is that a human hand isn’t a robot hand, so data must be retargeted using simulation to resolve issues like collisions and interpenetration when controlling the hand. In VideoManip, Hongyi Chen and co-authors built a system to solve this problem, taking in RGB videos of humans performing manipulation tasks and using them to create accurate simulations with which to learn robot policies. Watch episode #73 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton, now to learn more!

RoboPapers

27,360 görüntüleme • 5 ay önce

𝗛𝘂𝗺𝘆𝗻𝗹𝗮𝗯𝘀 𝗶𝘀 𝗰𝗿𝗲𝗮𝘁𝗶𝗻𝗴 𝗮 𝗱𝗮𝘁𝗮 𝗽𝗹𝗮𝘁𝗳𝗼𝗿𝗺 𝗳𝗼𝗿 𝗽𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗔𝗜. They collect thousands of hours of real human activities like what people see, hear, move, and feel and turn it into clean training data for robots. Unlike language models that learn from the internet, robots need real world experience for messy everyday tasks such as ironing while a child runs past, restocking shelves in a crowded store, or assembling parts on a busy factory floor. Humyn Labs' approach uses four sensory streams captured in actual environments so robots can learn practical skills from human experience.

𝗛𝘂𝗺𝘆𝗻𝗹𝗮𝗯𝘀 𝗶𝘀 𝗰𝗿𝗲𝗮𝘁𝗶𝗻𝗴 𝗮 𝗱𝗮𝘁𝗮 𝗽𝗹𝗮𝘁𝗳𝗼𝗿𝗺 𝗳𝗼𝗿 𝗽𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗔𝗜. They collect thousands of hours of real human activities like what people see, hear, move, and feel and turn it into clean training data for robots. Unlike language models that learn from the internet, robots need real world experience for messy everyday tasks such as ironing while a child runs past, restocking shelves in a crowded store, or assembling parts on a busy factory floor. Humyn Labs' approach uses four sensory streams captured in actual environments so robots can learn practical skills from human experience.

𝐓𝐡𝐞𝐎𝐧𝐥𝐲𝟏𝐇𝐞𝐫𝐨👑

18,514 görüntüleme • 1 ay önce

Engineers have built a robotic face that can learn to move its lips in sync with speech and singing by watching itself in a mirror and then observing humans in online videos, aiming to make humanoid robots appear less 'uncanny' in face-to-face interaction

Engineers have built a robotic face that can learn to move its lips in sync with speech and singing by watching itself in a mirror and then observing humans in online videos, aiming to make humanoid robots appear less 'uncanny' in face-to-face interaction

Reuters

27,638 görüntüleme • 8 ay önce

For general-purpose robots to be useful, we must be able to teach them new tasks quickly and repurpose them to new roles. But most techniques for teaching robots — even from human demonstrations — take a great deal of new data. But it turns out that it is possible, in part through decomposing tasks into component subtasks, that it’s possible to learn a robot skills quickly. Kamil Dreczkowski Pietro Vitiello and Edward Johns join us to talk about their extensive study on how to teach robots new skills efficiently, while also generalizing to novel object instances, using a combination of task decomposition and retrieval, work that was published in Science Robotics. Watch episode #94 of RoboPapers now, with Michael Cho - Rbt/Acc and Jiafei Duan, to learn more!

For general-purpose robots to be useful, we must be able to teach them new tasks quickly and repurpose them to new roles. But most techniques for teaching robots — even from human demonstrations — take a great deal of new data. But it turns out that it is possible, in part through decomposing tasks into component subtasks, that it’s possible to learn a robot skills quickly. Kamil Dreczkowski Pietro Vitiello and Edward Johns join us to talk about their extensive study on how to teach robots new skills efficiently, while also generalizing to novel object instances, using a combination of task decomposition and retrieval, work that was published in Science Robotics. Watch episode #94 of RoboPapers now, with Michael Cho - Rbt/Acc and Jiafei Duan, to learn more!

RoboPapers

19,495 görüntüleme • 1 ay önce

🚨Breaking: Google just dropped a new Guided Learning feature in Gemini. Perfect for students instead of just giving answers, it actually teaches. You can get images, diagrams, videos, and quizzes to learn step by step and see how much you’ve understood.

🚨Breaking: Google just dropped a new Guided Learning feature in Gemini. Perfect for students instead of just giving answers, it actually teaches. You can get images, diagrams, videos, and quizzes to learn step by step and see how much you’ve understood.

AshutoshShrivastava

79,363 görüntüleme • 1 yıl önce

RF-DETR by Roboflow now on MLX It can do realtime instance segmentation on-device and enable some cool use cases for visual analysis, monitoring and robotics like Reachy Mini. Also augmented VLM and VLA by preprocessing image and video with areas of interest. New release coming soon on mlx-vlm 🚀 For those who can’t wait you can install mlx-vlm from source.

RF-DETR by Roboflow now on MLX It can do realtime instance segmentation on-device and enable some cool use cases for visual analysis, monitoring and robotics like Reachy Mini. Also augmented VLM and VLA by preprocessing image and video with areas of interest. New release coming soon on mlx-vlm 🚀 For those who can’t wait you can install mlx-vlm from source.

Prince Canuma

30,499 görüntüleme • 5 ay önce

Excited to announce that we have raised $2.3M led by Sam Altman Peak XV Partners and an incredible set of investors for Induced We let anyone create virtual AI workers that can automate the execution of workflows on a browser in the cloud with human-like reasoning.

Excited to announce that we have raised $2.3M led by Sam Altman Peak XV Partners and an incredible set of investors for Induced We let anyone create virtual AI workers that can automate the execution of workflows on a browser in the cloud with human-like reasoning.

Aryan Sharma

465,671 görüntüleme • 3 yıl önce

We discovered an emergent property of VLAs like π0/π0.5/π0.6: as we scale up pre-training, the model learns to align human videos and robot data! This gives us a simple way to leverage human videos. Once π0.5 knows how to control robots, it can naturally learn from human video.

We discovered an emergent property of VLAs like π0/π0.5/π0.6: as we scale up pre-training, the model learns to align human videos and robot data! This gives us a simple way to leverage human videos. Once π0.5 knows how to control robots, it can naturally learn from human video.

Physical Intelligence

1,187,363 görüntüleme • 9 ay önce