Загрузка видео...

Не удалось загрузить видео

На главную

(1/n) Since its publication in 2017, PPO has essentially become synonymous with RL. Today, we are excited to provide you with a better alternative - EPO.

wang_jianren's profile picture

Jianren Wang

1,483 subscribers

145,520 просмотров • 1 год назад •via X (Twitter)

Наука и технологииОбразование
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

PPO has long dominated robot locomotion training in simulation. SAC, despite its sample efficiency, couldn't keep up. We analyze why: 🔗 🔥Integrated into RSL-RL, our approach requires only minimal changes, making SAC a drop-in alternative out of the box.
0:19

PPO has long dominated robot locomotion training in simulation. SAC, despite its sample efficiency, couldn't keep up. We analyze why: 🔗 🔥Integrated into RSL-RL, our approach requires only minimal changes, making SAC a drop-in alternative out of the box.

Robotic Systems Lab

43,448 просмотров • 25 дней назад

We found a new way to get language models to reason. 🤯 No RL, no training, no verifiers, no prompting. ❌ With better sampling, base models can achieve single-shot reasoning on par with (or better than!) GRPO while avoiding its characteristic loss in generation diversity.
0:11

We found a new way to get language models to reason. 🤯 No RL, no training, no verifiers, no prompting. ❌ With better sampling, base models can achieve single-shot reasoning on par with (or better than!) GRPO while avoiding its characteristic loss in generation diversity.

Aayush Karan

277,459 просмотров • 8 месяцев назад

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)
0:10

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)

Animesh Garg

52,279 просмотров • 2 лет назад

Are you excited to be with God forever in paradise? A. Yes B. No
0:10

Are you excited to be with God forever in paradise? A. Yes B. No

Faith Alone Saves ✟

231,127 просмотров • 1 год назад

RL HAS STARTED SHARING THEIR LOCATION WITH YOU #rollingloud
0:16

RL HAS STARTED SHARING THEIR LOCATION WITH YOU #rollingloud

Rolling Loud

13,214 просмотров • 1 год назад

1/ 📣 Native USDC is Going xChain: Wormhole integrates Circle’s CCTP We're incredibly excited to announce that Wormhole has partnered with Circle to provide a seamless cross-chain development experience. Read everything about it in this 🧵
0:16

1/ 📣 Native USDC is Going xChain: Wormhole integrates Circle’s CCTP We're incredibly excited to announce that Wormhole has partnered with Circle to provide a seamless cross-chain development experience. Read everything about it in this 🧵

Wormhole

96,981 просмотров • 3 лет назад

Excited to have partnered with @MLBRivalsUS to provide color commentary for the hit mobile game MLB Rivals. Download and play today:
0:15

Excited to have partnered with @MLBRivalsUS to provide color commentary for the hit mobile game MLB Rivals. Download and play today:

Orel Hershiser

22,336 просмотров • 1 год назад

Today, we're launching the $EURC Liquid vault. Earn on your Euros with 💶 We're excited to partner with K3 Capital and Midas to provide yield for $EURC holders. Start earning today at
0:14

Today, we're launching the $EURC Liquid vault. Earn on your Euros with 💶 We're excited to partner with K3 Capital and Midas to provide yield for $EURC holders. Start earning today at

ether.fi

117,150 просмотров • 1 месяц назад

Imagine believing that millions of these things erected almost silently overnight in every corner of The World are simply here to provide you with better internet.
0:14

Imagine believing that millions of these things erected almost silently overnight in every corner of The World are simply here to provide you with better internet.

Concerned Citizen

1,595,733 просмотров • 1 год назад

you will be able to become friends with jaoba. STOP BEING RUDE IN MY DMS WE ARE DOING IT
0:16

you will be able to become friends with jaoba. STOP BEING RUDE IN MY DMS WE ARE DOING IT

not yeucc

133,718 просмотров • 7 месяцев назад

Making some business deals with my new partner Hailey Blush today, I'm really excited to partner with her and her associates with "X" and I think she's excited too! In fact we decided that signing with a smooch was better than pen!! 🤗😘🥰💯✨❤️‍🔥💗🤍 #blondegirl #blondeshavemorefun #sealitwithakiss #newpartnerships #cutegirls
0:15

Making some business deals with my new partner Hailey Blush today, I'm really excited to partner with her and her associates with "X" and I think she's excited too! In fact we decided that signing with a smooch was better than pen!! 🤗😘🥰💯✨❤️‍🔥💗🤍 #blondegirl #blondeshavemorefun #sealitwithakiss #newpartnerships #cutegirls

Tisha

45,079 просмотров • 2 месяцев назад

Come out to Alternative Products Expo in Miami at Mana! We will be there today and tomorrow with all the goodies!
0:13

Sensitive content

Come out to Alternative Products Expo in Miami at Mana! We will be there today and tomorrow with all the goodies!

Bare Self Care

23,095 просмотров • 1 год назад

✨ WE DID IT in Araneta, BABY! ✨ With all your help in the light box project fundraising, we are excited to execute this project with you! 💖 FAM, are you ready to make history? Donation still open: 🔗 See you on July 4! #FreenBeckyAromagicare
0:14

✨ WE DID IT in Araneta, BABY! ✨ With all your help in the light box project fundraising, we are excited to execute this project with you! 💖 FAM, are you ready to make history? Donation still open: 🔗 See you on July 4! #FreenBeckyAromagicare

FreenBecky INTL Fanclub

441,260 просмотров • 3 лет назад

Ever since seeing Daniel getting along so well with a girl on campus, Kevin has been acting differently. "Hyung, are you upset with me about something today?" "Why won't you say anything?"
0:11

Ever since seeing Daniel getting along so well with a girl on campus, Kevin has been acting differently. "Hyung, are you upset with me about something today?" "Why won't you say anything?"

Kevin_boxer

49,995 просмотров • 6 дней назад

Gun Fu Art with a Capital "A". In Tensor City, survival demands you become its maestro.
0:17

Gun Fu Art with a Capital "A". In Tensor City, survival demands you become its maestro.

SPINE — This is Gun Fu

28,320 просмотров • 6 месяцев назад

just a reminder that conspiracy theorists are the easiest people to sell to now you can become one in 2 mins with AI crazy times we live in
0:12

just a reminder that conspiracy theorists are the easiest people to sell to now you can become one in 2 mins with AI crazy times we live in

MAX

76,427 просмотров • 5 месяцев назад

For the 8th year in row, and thanks to you, Air France has received the N°1 Customer Relationship Award in the « Transport » category. Year after year, we do our very best to maintain a strong link with you and to keep you satisfied. Thank you!
0:11

For the 8th year in row, and thanks to you, Air France has received the N°1 Customer Relationship Award in the « Transport » category. Year after year, we do our very best to maintain a strong link with you and to keep you satisfied. Thank you!

Air France

19,020 просмотров • 3 лет назад

Class of 2027, we are excited to get on the phone with you guys very soon! #GoWildcats
0:17

Class of 2027, we are excited to get on the phone with you guys very soon! #GoWildcats

ACU Women’s Basketball

14,401 просмотров • 1 год назад

Supercell has fired near Kress, Texas with a tornado watch issued. We are live in 1 minute
0:13

Supercell has fired near Kress, Texas with a tornado watch issued. We are live in 1 minute

Reed Timmer, PhD

45,050 просмотров • 1 год назад