Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Let’s do 🍒 Cherry Picking with Reinforcement Learning - 🥢 Dynamic fine manipulation with chopsticks - 🤖 Only 30 minutes of real world interactions - ⛔️ Too lazy for parameter tuning = off-the-shelf RL algo + default params + 3 seeds in real world

Kay - Liyiming Ke

1,650 subscribers

51,389 görüntüleme • 3 yıl önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

11 Yorum

Kay - Liyiming Ke profil fotoğrafı

Kay - Liyiming Ke3 yıl önce

1/3 - We developed a system for fine manipulation (small objects! 🍒) without rigid surface support (floating 🌊, windy 🍃, etc) using chopsticks🥢 (simple tool for fine manipulation). Our agent learned to be reactive & robust to dynamic scenes in the real world.

Kay - Liyiming Ke profil fotoğrafı

Kay - Liyiming Ke3 yıl önce

2/3 - Our proposal has 3 steps. First, pre-train in sim and leverage demos from heuristic-based visual servo and randomized dynamics. Then, fine-tune in the real world on a single object but make it haaarrd. Finally, deploy to downstream tasks with test-time perception modules.

Kay - Liyiming Ke profil fotoğrafı

Kay - Liyiming Ke3 yıl önce

3/3 - Our secret sauce ㊙️ is to understand the impact of each the design choice. We documented our journey + ablation studies in

Kay - Liyiming Ke profil fotoğrafı

Kay - Liyiming Ke3 yıl önce

♥️Kudos to all collaborators @yunchuzh @ab_deshpande @abhishekunique7 @siddhss5, and 🙇everyone who has helped along the way! This project gave me the most fun I had in grad school.

Chenhao Li profil fotoğrafı

Chenhao Li3 yıl önce

Hope one day it can reproduce Kung Fu Panda chopstick skills 😆

C. Zhang profil fotoğrafı

C. Zhang3 yıl önce

Chopsticks master!

Turing Post (Ksenia Se) profil fotoğrafı

Turing Post (Ksenia Se)3 yıl önce

Precise!

Necromancer profil fotoğrafı

Necromancer3 yıl önce

Amazing fine manipulation results!

Samarth Brahmbhatt profil fotoğrafı

Samarth Brahmbhatt3 yıl önce

Very interesting work @xkelym! I very much appreciate: - description of the steps needed for real-robot fine tuning - sim system identification - comprehensive experiments! We also had a recent paper about a contact-rich task (inserting plates and cups), though 0-shot transfer.

Kay - Liyiming Ke profil fotoğrafı

Kay - Liyiming Ke3 yıl önce

Thanks! I am also fascinated by contact-rich tasks. Looking forward to read your work.

Yanko profil fotoğrafı

Yanko1 yıl önce

Cool :)

Benzer Videolar

RL-100 Performant Robotic Manipulation with Real-World Reinforcement Learning

RL-100 Performant Robotic Manipulation with Real-World Reinforcement Learning

AK

15,364 görüntüleme • 1 yıl önce

Announcing RoboFuME🤖💨, a system for autonomous & efficient real-world robot learning that 1. pre-trains a VLM reward model and a multi-task RL policy from diverse off-the-shelf demo data; 2. runs RL fine-tuning online with the VLM reward model. 🔗 🧵↓

Announcing RoboFuME🤖💨, a system for autonomous & efficient real-world robot learning that 1. pre-trains a VLM reward model and a multi-task RL policy from diverse off-the-shelf demo data; 2. runs RL fine-tuning online with the VLM reward model. 🔗 🧵↓

Jingyun Yang

88,312 görüntüleme • 3 yıl önce

What if robots could improve themselves by learning from their own failures in the real-world? Introducing 𝗣𝗟𝗗 (𝗣𝗿𝗼𝗯𝗲, 𝗟𝗲𝗮𝗿𝗻, 𝗗𝗶𝘀𝘁𝗶𝗹𝗹) — a recipe that enables Vision-Language-Action (VLA) models to self-improve for high-precision manipulation tasks. PLD couples real-world residual reinforcement learning with standard supervised fine-tuning — letting robots discover, recover, and distill their own data flywheel. Quick 🧵

What if robots could improve themselves by learning from their own failures in the real-world? Introducing 𝗣𝗟𝗗 (𝗣𝗿𝗼𝗯𝗲, 𝗟𝗲𝗮𝗿𝗻, 𝗗𝗶𝘀𝘁𝗶𝗹𝗹) — a recipe that enables Vision-Language-Action (VLA) models to self-improve for high-precision manipulation tasks. PLD couples real-world residual reinforcement learning with standard supervised fine-tuning — letting robots discover, recover, and distill their own data flywheel. Quick 🧵

Wenli Xiao

186,165 görüntüleme • 11 ay önce

The best way to get robust, high-quality robot performance is through reinforcement learning; but RL in either the real world or a traditional simulation has lots of limitations. Instead, Jiazhi Yang in RISE does RL in a compositional world model. Learn more ->

The best way to get robust, high-quality robot performance is through reinforcement learning; but RL in either the real world or a traditional simulation has lots of limitations. Instead, Jiazhi Yang in RISE does RL in a compositional world model. Learn more ->

Chris Paxton

34,102 görüntüleme • 4 ay önce

So we did a bunch of projects with real world reinforcement learning - but it was often too inefficient to be practical to train tabula rasa. This suggests we need better priors, but acquiring these from on-robot data can often be expensive as well. In our recent work, we show that despite being fundamentally inaccurate, simulation can guide provide a cheap way to guide real-world RL finetuning to be super efficient! We propose Simulation-Guided Fine-Tuning (SGFT) - a simple paradigm for sim2real finetuning that uses simulation to provide reward shaping that accelerates real world RL finetuning *beyond* just providing an initialization. TLDR: Use value functions from sim to shape rewards for real-world RL, see large sample efficiency improvements 🧵(1/6)

So we did a bunch of projects with real world reinforcement learning - but it was often too inefficient to be practical to train tabula rasa. This suggests we need better priors, but acquiring these from on-robot data can often be expensive as well. In our recent work, we show that despite being fundamentally inaccurate, simulation can guide provide a cheap way to guide real-world RL finetuning to be super efficient! We propose Simulation-Guided Fine-Tuning (SGFT) - a simple paradigm for sim2real finetuning that uses simulation to provide reward shaping that accelerates real world RL finetuning beyond just providing an initialization. TLDR: Use value functions from sim to shape rewards for real-world RL, see large sample efficiency improvements 🧵(1/6)

Abhishek Gupta

13,637 görüntüleme • 1 yıl önce

Introducing RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learning. 7 real robot tasks, 900/900 successes. Up to 250 consecutive trials in one task, running 2 hours nonstop without failure. High success rate against physical disturbances, zero-shot, and few-shot adaptation Our first step toward a deployable robot learning system.

Introducing RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learning. 7 real robot tasks, 900/900 successes. Up to 250 consecutive trials in one task, running 2 hours nonstop without failure. High success rate against physical disturbances, zero-shot, and few-shot adaptation Our first step toward a deployable robot learning system.

Kun Lei

92,764 görüntüleme • 1 yıl önce

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

38,867 görüntüleme • 4 ay önce

🚀 🔥 AgiBot deploys Real-World Reinforcement Learning (RW-RL) in industrial robotics with Longcheer Technology. Robots now learn new skills in tens of MINUTES (not weeks), adapt to variations autonomously, and reconfigure flexibly, solving rigid automation pain points in precision manufacturing. A giant leap of intelligent automation for precision manufacturing! #AgiBot #ReinforcementLearning #RealWorldRL #Robotics #AI #IndustrialRobotics

🚀 🔥 AgiBot deploys Real-World Reinforcement Learning (RW-RL) in industrial robotics with Longcheer Technology. Robots now learn new skills in tens of MINUTES (not weeks), adapt to variations autonomously, and reconfigure flexibly, solving rigid automation pain points in precision manufacturing. A giant leap of intelligent automation for precision manufacturing! #AgiBot #ReinforcementLearning #RealWorldRL #Robotics #AI #IndustrialRobotics

AGIBOT

131,958 görüntüleme • 11 ay önce

Simulation provides the ability to unlock autonomy at scale. Our new blog shows how multi-agent reinforcement learning allows us to populate our Wayve Infinity Simulator with the emergent behaviour critical to represent dynamic urban cities in the real world.

Simulation provides the ability to unlock autonomy at scale. Our new blog shows how multi-agent reinforcement learning allows us to populate our Wayve Infinity Simulator with the emergent behaviour critical to represent dynamic urban cities in the real world.

Alex Kendall

66,941 görüntüleme • 3 yıl önce

🛠️ Did someone ask for some sunlight? 🌤️ Dynamic sunlight can be based on real-world geographic coordinates and change in real time. Dynamic weather can be enabled in real time too, with rain coming during the match. Here's a test with real-time manual adjustments 👇 #efootball

🛠️ Did someone ask for some sunlight? 🌤️ Dynamic sunlight can be based on real-world geographic coordinates and change in real time. Dynamic weather can be enabled in real time too, with rain coming during the match. Here's a test with real-time manual adjustments 👇 #efootball

Durandil PES 🎣

110,795 görüntüleme • 11 gün önce

🍒Hello Cherry Team! 🍒Today we learn about Real World Assets (RWA-s). How do Real World Assets contribute to the economy? What are some of the examples of RWA-s? Watch the video to find answer to your questions! Watch the video closely to gather all the clues! #Cherry $CHERRY

🍒Hello Cherry Team! 🍒Today we learn about Real World Assets (RWA-s). How do Real World Assets contribute to the economy? What are some of the examples of RWA-s? Watch the video to find answer to your questions! Watch the video closely to gather all the clues! #Cherry $CHERRY

Cherry Trending

84,227 görüntüleme • 2 yıl önce

We Luis Felipe Casas have successfully deployed an in-hand manipulation policy to the LEAP hand following RL training in MuJoCo Playground. Sim-to-real isn’t as straightforward as it seems. Policy design and real-world tuning matter. Working on improving the policy further.

We Luis Felipe Casas have successfully deployed an in-hand manipulation policy to the LEAP hand following RL training in MuJoCo Playground. Sim-to-real isn’t as straightforward as it seems. Policy design and real-world tuning matter. Working on improving the policy further.

Yu Xiang

11,142 görüntüleme • 6 ay önce

Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning investigated the application of Deep Reinforcement Learning (Deep RL) for low-cost, miniature humanoid hardware in a dynamic environment, showing the method can synthesize sophisticated and safe movement skills making up complex behavioral strategies in a simplified one-versus-one (1v1) soccer game abs: project page:

Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning investigated the application of Deep Reinforcement Learning (Deep RL) for low-cost, miniature humanoid hardware in a dynamic environment, showing the method can synthesize sophisticated and safe movement skills making up complex behavioral strategies in a simplified one-versus-one (1v1) soccer game abs: project page:

AK

293,474 görüntüleme • 3 yıl önce

We asked Sholto Douglas from Anthropic about the costs of RL (Reinforcement Learning) runs. "In Dario Amodei's essay, he said that RL runs cost only $1M back in December." "RL is a more naively parallelizable and scalable than pre-training." "With pre-training, you need everything in one big data center ideally. For RL, in theory, you could scale all over the world."

We asked Sholto Douglas from Anthropic about the costs of RL (Reinforcement Learning) runs. "In Dario Amodei's essay, he said that RL runs cost only $1M back in December." "RL is a more naively parallelizable and scalable than pre-training." "With pre-training, you need everything in one big data center ideally. For RL, in theory, you could scale all over the world."

TBPN

76,696 görüntüleme • 1 yıl önce

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes evencheaper with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

Abhishek Gupta

13,358 görüntüleme • 1 yıl önce

Haven't been to a conference in a while, really excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)

Haven't been to a conference in a while, really excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)

Abhishek Gupta

10,803 görüntüleme • 1 yıl önce

Introducing HiLMa-Res: a hierarchical RL framework for quadrupeds to tackle loco-manipulation tasks with sustained mobility! Designed for general learning tasks (vision-based, state-based, real-world data, etc), the robot now can step over stones🐾/navigate boxes📦/dribble⚽.

Introducing HiLMa-Res: a hierarchical RL framework for quadrupeds to tackle loco-manipulation tasks with sustained mobility! Designed for general learning tasks (vision-based, state-based, real-world data, etc), the robot now can step over stones🐾/navigate boxes📦/dribble⚽.

Zhongyu Li

11,720 görüntüleme • 2 yıl önce

How do you explain world models to your sibling? 🤖 NVIDIA Cosmos helps physical AI reason about real-world interactions and simulate possible futures before acting in the real world. Or, as they might put it, it teaches robots to dream. 💭

How do you explain world models to your sibling? 🤖 NVIDIA Cosmos helps physical AI reason about real-world interactions and simulate possible futures before acting in the real world. Or, as they might put it, it teaches robots to dream. 💭

NVIDIA

89,292 görüntüleme • 2 ay önce