正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

(1/n) Since its publication in 2017, PPO has essentially become synonymous with RL. Today, we are excited to provide you with a better alternative - EPO.

Jianren Wang

1,497 subscribers

145,870 次观看 • 1 年前 •via X (Twitter)

科学技术教育

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

PPO has long dominated robot locomotion training in simulation. SAC, despite its sample efficiency, couldn't keep up. We analyze why: 🔗 🔥Integrated into RSL-RL, our approach requires only minimal changes, making SAC a drop-in alternative out of the box.

PPO has long dominated robot locomotion training in simulation. SAC, despite its sample efficiency, couldn't keep up. We analyze why: 🔗 🔥Integrated into RSL-RL, our approach requires only minimal changes, making SAC a drop-in alternative out of the box.

Robotic Systems Lab

45,107 次观看 • 2 个月前

We found a new way to get language models to reason. 🤯 No RL, no training, no verifiers, no prompting. ❌ With better sampling, base models can achieve single-shot reasoning on par with (or better than!) GRPO while avoiding its characteristic loss in generation diversity.

We found a new way to get language models to reason. 🤯 No RL, no training, no verifiers, no prompting. ❌ With better sampling, base models can achieve single-shot reasoning on par with (or better than!) GRPO while avoiding its characteristic loss in generation diversity.

Aayush Karan

278,309 次观看 • 10 个月前

Today, we are excited to announce a major partnership with GSK to deploy the latest Boltz models across GSK’s research organization!

Today, we are excited to announce a major partnership with GSK to deploy the latest Boltz models across GSK’s research organization!

Gabriele Corso

40,855 次观看 • 2 个月前

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)

Animesh Garg

52,308 次观看 • 2 年前

Are you excited to be with God forever in paradise? A. Yes B. No

Are you excited to be with God forever in paradise? A. Yes B. No

Faith Alone Saves ✟

231,151 次观看 • 2 年前

Trump: When we do well, they shouldn't have to raise interest rates. They should in many cases lower because you know why? We become a better credit. When we're doing well, we become a better credit. When you become a better credit, you're supposed to cut rates.

Trump: When we do well, they shouldn't have to raise interest rates. They should in many cases lower because you know why? We become a better credit. When we're doing well, we become a better credit. When you become a better credit, you're supposed to cut rates.

Acyn

104,422 次观看 • 4 小时前

1/ 📣 Native USDC is Going xChain: Wormhole integrates Circle’s CCTP We're incredibly excited to announce that Wormhole has partnered with Circle to provide a seamless cross-chain development experience. Read everything about it in this 🧵

1/ 📣 Native USDC is Going xChain: Wormhole integrates Circle’s CCTP We're incredibly excited to announce that Wormhole has partnered with Circle to provide a seamless cross-chain development experience. Read everything about it in this 🧵

Wormhole

96,995 次观看 • 3 年前

Imagine believing that millions of these things erected almost silently overnight in every corner of The World are simply here to provide you with better internet.

Imagine believing that millions of these things erected almost silently overnight in every corner of The World are simply here to provide you with better internet.

Concerned Citizen

1,595,759 次观看 • 1 年前

you will be able to become friends with jaoba. STOP BEING RUDE IN MY DMS WE ARE DOING IT

you will be able to become friends with jaoba. STOP BEING RUDE IN MY DMS WE ARE DOING IT

not yeucc

133,718 次观看 • 9 个月前

Making some business deals with my new partner Hailey Blush today, I'm really excited to partner with her and her associates with "X" and I think she's excited too! In fact we decided that signing with a smooch was better than pen!! 🤗😘🥰💯✨❤️‍🔥💗🤍 #blondegirl #blondeshavemorefun #sealitwithakiss #newpartnerships #cutegirls

Making some business deals with my new partner Hailey Blush today, I'm really excited to partner with her and her associates with "X" and I think she's excited too! In fact we decided that signing with a smooch was better than pen!! 🤗😘🥰💯✨❤️‍🔥💗🤍 #blondegirl #blondeshavemorefun #sealitwithakiss #newpartnerships #cutegirls

Tisha

45,079 次观看 • 4 个月前

Ever since seeing Daniel getting along so well with a girl on campus, Kevin has been acting differently. "Hyung, are you upset with me about something today?" "Why won't you say anything?"

Ever since seeing Daniel getting along so well with a girl on campus, Kevin has been acting differently. "Hyung, are you upset with me about something today?" "Why won't you say anything?"

Kevin_boxer

50,859 次观看 • 2 个月前

just a reminder that conspiracy theorists are the easiest people to sell to now you can become one in 2 mins with AI crazy times we live in

just a reminder that conspiracy theorists are the easiest people to sell to now you can become one in 2 mins with AI crazy times we live in

MAX

76,427 次观看 • 7 个月前

Gun Fu Art with a Capital "A". In Tensor City, survival demands you become its maestro.

Gun Fu Art with a Capital "A". In Tensor City, survival demands you become its maestro.

SPINE — This is Gun Fu

28,320 次观看 • 8 个月前

For the 8th year in row, and thanks to you, Air France has received the N°1 Customer Relationship Award in the « Transport » category. Year after year, we do our very best to maintain a strong link with you and to keep you satisfied. Thank you!

For the 8th year in row, and thanks to you, Air France has received the N°1 Customer Relationship Award in the « Transport » category. Year after year, we do our very best to maintain a strong link with you and to keep you satisfied. Thank you!

Air France

19,021 次观看 • 3 年前

Class of 2027, we are excited to get on the phone with you guys very soon! #GoWildcats

Class of 2027, we are excited to get on the phone with you guys very soon! #GoWildcats

ACU Women’s Basketball

14,408 次观看 • 1 年前

We are incredibly excited to announce that MANSA has raised $10M, led by and Polymorphic Capital , to revolutionize cross-border payments with with stablecoin-based liquidity solutions.

We are incredibly excited to announce that MANSA has raised $10M, led by and Polymorphic Capital , to revolutionize cross-border payments with with stablecoin-based liquidity solutions.

MANSA

124,359 次观看 • 1 年前