Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

(1/n) Since its publication in 2017, PPO has essentially become synonymous with RL. Today, we are excited to provide you with a better alternative - EPO.

wang_jianren's profile picture

Jianren Wang

1,482 subscribers

145,659 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & TechnologieBildung
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

PPO has long dominated robot locomotion training in simulation. SAC, despite its sample efficiency, couldn't keep up. We analyze why: 🔗 🔥Integrated into RSL-RL, our approach requires only minimal changes, making SAC a drop-in alternative out of the box.
0:19

PPO has long dominated robot locomotion training in simulation. SAC, despite its sample efficiency, couldn't keep up. We analyze why: 🔗 🔥Integrated into RSL-RL, our approach requires only minimal changes, making SAC a drop-in alternative out of the box.

Robotic Systems Lab

44,447 Aufrufe • vor 1 Monat

We found a new way to get language models to reason. 🤯 No RL, no training, no verifiers, no prompting. ❌ With better sampling, base models can achieve single-shot reasoning on par with (or better than!) GRPO while avoiding its characteristic loss in generation diversity.
0:11

We found a new way to get language models to reason. 🤯 No RL, no training, no verifiers, no prompting. ❌ With better sampling, base models can achieve single-shot reasoning on par with (or better than!) GRPO while avoiding its characteristic loss in generation diversity.

Aayush Karan

277,680 Aufrufe • vor 9 Monaten

Today, we are excited to announce a major partnership with GSK to deploy the latest Boltz models across GSK’s research organization!
0:15

Today, we are excited to announce a major partnership with GSK to deploy the latest Boltz models across GSK’s research organization!

Gabriele Corso

40,236 Aufrufe • vor 1 Monat

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)
0:10

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)

Animesh Garg

52,308 Aufrufe • vor 2 Jahren

Are you excited to be with God forever in paradise? A. Yes B. No
0:10

Are you excited to be with God forever in paradise? A. Yes B. No

Faith Alone Saves ✟

231,151 Aufrufe • vor 2 Jahren

1/ 📣 Native USDC is Going xChain: Wormhole integrates Circle’s CCTP We're incredibly excited to announce that Wormhole has partnered with Circle to provide a seamless cross-chain development experience. Read everything about it in this 🧵
0:16

1/ 📣 Native USDC is Going xChain: Wormhole integrates Circle’s CCTP We're incredibly excited to announce that Wormhole has partnered with Circle to provide a seamless cross-chain development experience. Read everything about it in this 🧵

Wormhole

96,993 Aufrufe • vor 3 Jahren

you will be able to become friends with jaoba. STOP BEING RUDE IN MY DMS WE ARE DOING IT
0:16

you will be able to become friends with jaoba. STOP BEING RUDE IN MY DMS WE ARE DOING IT

not yeucc

133,718 Aufrufe • vor 8 Monaten

Imagine believing that millions of these things erected almost silently overnight in every corner of The World are simply here to provide you with better internet.
0:14

Imagine believing that millions of these things erected almost silently overnight in every corner of The World are simply here to provide you with better internet.

Concerned Citizen

1,595,733 Aufrufe • vor 1 Jahr

Making some business deals with my new partner Hailey Blush today, I'm really excited to partner with her and her associates with "X" and I think she's excited too! In fact we decided that signing with a smooch was better than pen!! 🤗😘🥰💯✨❤️‍🔥💗🤍 #blondegirl #blondeshavemorefun #sealitwithakiss #newpartnerships #cutegirls
0:15

Making some business deals with my new partner Hailey Blush today, I'm really excited to partner with her and her associates with "X" and I think she's excited too! In fact we decided that signing with a smooch was better than pen!! 🤗😘🥰💯✨❤️‍🔥💗🤍 #blondegirl #blondeshavemorefun #sealitwithakiss #newpartnerships #cutegirls

Tisha

45,079 Aufrufe • vor 3 Monaten

Come out to Alternative Products Expo in Miami at Mana! We will be there today and tomorrow with all the goodies!
0:13

Sensitive content

Come out to Alternative Products Expo in Miami at Mana! We will be there today and tomorrow with all the goodies!

Bare Self Care

23,095 Aufrufe • vor 1 Jahr

Ever since seeing Daniel getting along so well with a girl on campus, Kevin has been acting differently. "Hyung, are you upset with me about something today?" "Why won't you say anything?"
0:11

Ever since seeing Daniel getting along so well with a girl on campus, Kevin has been acting differently. "Hyung, are you upset with me about something today?" "Why won't you say anything?"

Kevin_boxer

50,748 Aufrufe • vor 1 Monat

✨ WE DID IT in Araneta, BABY! ✨ With all your help in the light box project fundraising, we are excited to execute this project with you! 💖 FAM, are you ready to make history? Donation still open: 🔗 See you on July 4! #FreenBeckyAromagicare
0:14

✨ WE DID IT in Araneta, BABY! ✨ With all your help in the light box project fundraising, we are excited to execute this project with you! 💖 FAM, are you ready to make history? Donation still open: 🔗 See you on July 4! #FreenBeckyAromagicare

FreenBecky INTL Fanclub

441,260 Aufrufe • vor 3 Jahren

Gun Fu Art with a Capital "A". In Tensor City, survival demands you become its maestro.
0:17

Gun Fu Art with a Capital "A". In Tensor City, survival demands you become its maestro.

SPINE — This is Gun Fu

28,320 Aufrufe • vor 7 Monaten

just a reminder that conspiracy theorists are the easiest people to sell to now you can become one in 2 mins with AI crazy times we live in
0:12

just a reminder that conspiracy theorists are the easiest people to sell to now you can become one in 2 mins with AI crazy times we live in

MAX

76,427 Aufrufe • vor 6 Monaten

For the 8th year in row, and thanks to you, Air France has received the N°1 Customer Relationship Award in the « Transport » category. Year after year, we do our very best to maintain a strong link with you and to keep you satisfied. Thank you!
0:11

For the 8th year in row, and thanks to you, Air France has received the N°1 Customer Relationship Award in the « Transport » category. Year after year, we do our very best to maintain a strong link with you and to keep you satisfied. Thank you!

Air France

19,021 Aufrufe • vor 3 Jahren

Class of 2027, we are excited to get on the phone with you guys very soon! #GoWildcats
0:17

Class of 2027, we are excited to get on the phone with you guys very soon! #GoWildcats

ACU Women’s Basketball

14,401 Aufrufe • vor 1 Jahr

We are incredibly excited to announce that MANSA has raised $10M, led by and Polymorphic Capital , to revolutionize cross-border payments with with stablecoin-based liquidity solutions.
0:19

We are incredibly excited to announce that MANSA has raised $10M, led by and Polymorphic Capital , to revolutionize cross-border payments with with stablecoin-based liquidity solutions.

MANSA

124,157 Aufrufe • vor 1 Jahr

With all the feedback everyone has given us over the past months since beta, we've landed on a vision of Omega Strikers that we believe is perfect. We've gone back to the fundamentals and are sticking with them. I'm so excited to share a new quick clip of these developments!
0:15

With all the feedback everyone has given us over the past months since beta, we've landed on a vision of Omega Strikers that we believe is perfect. We've gone back to the fundamentals and are sticking with them. I'm so excited to share a new quick clip of these developments!

Omega Strikers

66,883 Aufrufe • vor 3 Jahren