Загрузка видео...

Не удалось загрузить видео

На главную

(1/n) Since its publication in 2017, PPO has essentially become synonymous with RL. Today, we are excited to provide you with a better alternative - EPO.

wang_jianren's profile picture

Jianren Wang

1,497 subscribers

145,870 просмотров • 1 год назад •via X (Twitter)

Наука и технологииОбразование
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

PPO has long dominated robot locomotion training in simulation. SAC, despite its sample efficiency, couldn't keep up. We analyze why: 🔗 🔥Integrated into RSL-RL, our approach requires only minimal changes, making SAC a drop-in alternative out of the box.
0:19

PPO has long dominated robot locomotion training in simulation. SAC, despite its sample efficiency, couldn't keep up. We analyze why: 🔗 🔥Integrated into RSL-RL, our approach requires only minimal changes, making SAC a drop-in alternative out of the box.

Robotic Systems Lab

45,107 просмотров • 2 месяцев назад

We found a new way to get language models to reason. 🤯 No RL, no training, no verifiers, no prompting. ❌ With better sampling, base models can achieve single-shot reasoning on par with (or better than!) GRPO while avoiding its characteristic loss in generation diversity.
0:11

We found a new way to get language models to reason. 🤯 No RL, no training, no verifiers, no prompting. ❌ With better sampling, base models can achieve single-shot reasoning on par with (or better than!) GRPO while avoiding its characteristic loss in generation diversity.

Aayush Karan

278,309 просмотров • 10 месяцев назад

Today, we are excited to announce a major partnership with GSK to deploy the latest Boltz models across GSK’s research organization!
0:15

Today, we are excited to announce a major partnership with GSK to deploy the latest Boltz models across GSK’s research organization!

Gabriele Corso

40,855 просмотров • 2 месяцев назад

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)
0:10

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)

Animesh Garg

52,308 просмотров • 2 лет назад

Are you excited to be with God forever in paradise? A. Yes B. No
0:10

Are you excited to be with God forever in paradise? A. Yes B. No

Faith Alone Saves ✟

231,151 просмотров • 2 лет назад

Trump: When we do well, they shouldn't have to raise interest rates. They should in many cases lower because you know why? We become a better credit. When we're doing well, we become a better credit. When you become a better credit, you're supposed to cut rates.
0:19

Trump: When we do well, they shouldn't have to raise interest rates. They should in many cases lower because you know why? We become a better credit. When we're doing well, we become a better credit. When you become a better credit, you're supposed to cut rates.

Acyn

134,397 просмотров • 10 часов назад

1/ 📣 Native USDC is Going xChain: Wormhole integrates Circle’s CCTP We're incredibly excited to announce that Wormhole has partnered with Circle to provide a seamless cross-chain development experience. Read everything about it in this 🧵
0:16

1/ 📣 Native USDC is Going xChain: Wormhole integrates Circle’s CCTP We're incredibly excited to announce that Wormhole has partnered with Circle to provide a seamless cross-chain development experience. Read everything about it in this 🧵

Wormhole

96,995 просмотров • 3 лет назад

you will be able to become friends with jaoba. STOP BEING RUDE IN MY DMS WE ARE DOING IT
0:16

you will be able to become friends with jaoba. STOP BEING RUDE IN MY DMS WE ARE DOING IT

not yeucc

133,718 просмотров • 9 месяцев назад

Imagine believing that millions of these things erected almost silently overnight in every corner of The World are simply here to provide you with better internet.
0:14

Imagine believing that millions of these things erected almost silently overnight in every corner of The World are simply here to provide you with better internet.

Concerned Citizen

1,595,759 просмотров • 1 год назад

Making some business deals with my new partner Hailey Blush today, I'm really excited to partner with her and her associates with "X" and I think she's excited too! In fact we decided that signing with a smooch was better than pen!! 🤗😘🥰💯✨❤️‍🔥💗🤍 #blondegirl #blondeshavemorefun #sealitwithakiss #newpartnerships #cutegirls
0:15

Making some business deals with my new partner Hailey Blush today, I'm really excited to partner with her and her associates with "X" and I think she's excited too! In fact we decided that signing with a smooch was better than pen!! 🤗😘🥰💯✨❤️‍🔥💗🤍 #blondegirl #blondeshavemorefun #sealitwithakiss #newpartnerships #cutegirls

Tisha

45,079 просмотров • 4 месяцев назад

Ever since seeing Daniel getting along so well with a girl on campus, Kevin has been acting differently. "Hyung, are you upset with me about something today?" "Why won't you say anything?"
0:11

Ever since seeing Daniel getting along so well with a girl on campus, Kevin has been acting differently. "Hyung, are you upset with me about something today?" "Why won't you say anything?"

Kevin_boxer

50,859 просмотров • 2 месяцев назад

For the 8th year in row, and thanks to you, Air France has received the N°1 Customer Relationship Award in the « Transport » category. Year after year, we do our very best to maintain a strong link with you and to keep you satisfied. Thank you!
0:11

For the 8th year in row, and thanks to you, Air France has received the N°1 Customer Relationship Award in the « Transport » category. Year after year, we do our very best to maintain a strong link with you and to keep you satisfied. Thank you!

Air France

19,021 просмотров • 3 лет назад

Gun Fu Art with a Capital "A". In Tensor City, survival demands you become its maestro.
0:17

Gun Fu Art with a Capital "A". In Tensor City, survival demands you become its maestro.

SPINE — This is Gun Fu

28,320 просмотров • 8 месяцев назад

just a reminder that conspiracy theorists are the easiest people to sell to now you can become one in 2 mins with AI crazy times we live in
0:12

just a reminder that conspiracy theorists are the easiest people to sell to now you can become one in 2 mins with AI crazy times we live in

MAX

76,427 просмотров • 7 месяцев назад

Class of 2027, we are excited to get on the phone with you guys very soon! #GoWildcats
0:17

Class of 2027, we are excited to get on the phone with you guys very soon! #GoWildcats

ACU Women’s Basketball

14,408 просмотров • 1 год назад

ARE YOU KIDDING?!! Tyler Black preserves the no-hitter with a highlight grab to end the game in left! Brandon Sproat and Tate Kuehner combine for the first Sounds no-hitter since 2017!
0:17

ARE YOU KIDDING?!! Tyler Black preserves the no-hitter with a highlight grab to end the game in left! Brandon Sproat and Tate Kuehner combine for the first Sounds no-hitter since 2017!

Nashville Sounds

118,338 просмотров • 8 дней назад