Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Congratulations also to Patrick 👍 for his #ICLR paper on Temporal Difference (TD) learning , in it We solve the decades-old puzzle of why TD can solve complex RL tasks that Gradient Descent cannot.

Thuerey Group at TUM

2,817 subscribers

22,074 views • 1 year ago •via X (Twitter)

Science & Technology Education Health & Wellness #ICLR

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

RL is back! But is it always the best choice? In a new paper, we investigate under what circumstances neuroevolution outperforms RL in transfer learning tasks. See more details in the thread below 🧵 While NE performs best in simpler domains, it will be interesting to see if the lessons learned here can also be applied to more complex systems/tasks (LLMs?).

RL is back! But is it always the best choice? In a new paper, we investigate under what circumstances neuroevolution outperforms RL in transfer learning tasks. See more details in the thread below 🧵 While NE performs best in simpler domains, it will be interesting to see if the lessons learned here can also be applied to more complex systems/tasks (LLMs?).

Sebastian Risi

11,220 views • 1 year ago

1/ While most RL methods use shallow MLPs (~2–5 layers), we show that scaling up to 1000-layers for contrastive RL (CRL) can significantly boost performance, ranging from doubling performance to 50x on a diverse suite of robotic tasks. Webpage+Paper+Code:

1/ While most RL methods use shallow MLPs (~2–5 layers), we show that scaling up to 1000-layers for contrastive RL (CRL) can significantly boost performance, ranging from doubling performance to 50x on a diverse suite of robotic tasks. Webpage+Paper+Code:

Kevin Wang

155,134 views • 1 year ago

Robots struggle with strict action rules…memory and symbols help them learn fast. [Project + Full video link ⬇️] Robots struggle when tasks require specific steps in a fixed order. What if memory helped them think symbolically and learn faster? Solving tasks like unlocking a door then opening it is hard for deep RL. But by learning constraint relationships and storing them in memory, robots can solve these tasks much faster; with fewer trials and less training. Why it works ✅ Learns symbolic rules about action constraints ✅ Uses memory to transfer what it learned across tasks ✅ Handles real-world exploration with just 30 minutes of data ✅ Needs 10x fewer episodes than deep RL approaches This memory-based method shows a promising path forward for robots learning structured, real-world tasks. Full video: Paper: Thank you, Mrinal Verghese for sharing this amazing work! 🙏

Robots struggle with strict action rules…memory and symbols help them learn fast. [Project + Full video link ⬇️] Robots struggle when tasks require specific steps in a fixed order. What if memory helped them think symbolically and learn faster? Solving tasks like unlocking a door then opening it is hard for deep RL. But by learning constraint relationships and storing them in memory, robots can solve these tasks much faster; with fewer trials and less training. Why it works ✅ Learns symbolic rules about action constraints ✅ Uses memory to transfer what it learned across tasks ✅ Handles real-world exploration with just 30 minutes of data ✅ Needs 10x fewer episodes than deep RL approaches This memory-based method shows a promising path forward for robots learning structured, real-world tasks. Full video: Paper: Thank you, Mrinal Verghese for sharing this amazing work! 🙏

Ilir Aliu - eu/acc

10,241 views • 1 year ago

Yes, such a great BioRob meeting. Enjoyed catching up with everyone. Great to see all the RL+biomechanics here. Also honored that our work on imitation learning for gait analysis was another best paper finalist Nice updates in MuJoCo coming soon!

Yes, such a great BioRob meeting. Enjoyed catching up with everyone. Great to see all the RL+biomechanics here. Also honored that our work on imitation learning for gait analysis was another best paper finalist Nice updates in MuJoCo coming soon!

R. James Cotton

12,006 views • 1 year ago

#Browns Shedeur Sanders on his near TD pass to Jerry Jeudy that was snatched out of his hands for an INT vs the #Bears in #Browns 31-3 loss:

#Browns Shedeur Sanders on his near TD pass to Jerry Jeudy that was snatched out of his hands for an INT vs the #Bears in #Browns 31-3 loss:

Mary Kay Cabot

19,953 views • 7 months ago

Test-time learning has a catch that scaling alone does not solve: the state a system writes to in order to learn is also the record a user must inspect to audit it. Optimize it only for learning, and auditability usually degrades. Blog: Part I treats the learning state and the audit record as the same object. This starts the ADAPT series, exploring the architecture section by section.

Test-time learning has a catch that scaling alone does not solve: the state a system writes to in order to learn is also the record a user must inspect to audit it. Optimize it only for learning, and auditability usually degrades. Blog: Part I treats the learning state and the audit record as the same object. This starts the ADAPT series, exploring the architecture section by section.

Rei

22,864 views • 2 days ago

Ever watch an agent solve a problem with 100% confidence and get it completely wrong? You can fix it… if you catch it. But then you need to watch every… single… run. New paper: We show that a meta-agent can do this for you, so your agents get ~2x more done on their own. 🧵

Ever watch an agent solve a problem with 100% confidence and get it completely wrong? You can fix it… if you catch it. But then you need to watch every… single… run. New paper: We show that a meta-agent can do this for you, so your agents get ~2x more done on their own. 🧵

Weiyan Shi

62,373 views • 1 month ago

.Rachel Millward: "You cannot actually solve the housing crisis in the current planning system because it begins and ends with profit. Developer profit is protected in this system. We need to have council housing at scale & that will take a change of govt legislation" 🎯

.Rachel Millward: "You cannot actually solve the housing crisis in the current planning system because it begins and ends with profit. Developer profit is protected in this system. We need to have council housing at scale & that will take a change of govt legislation" 🎯

Saul Staniforth

363,337 views • 3 months ago

Just presented my paper titled “Can Al-Ghazali Solve the Problem of Fake News?” at the conference in Berlin 🇩🇪 Grateful to Der Divan for organizing this enriching event! Der Divan | الديوان Also had a great discussion with Stefan Weidner on challenges and building bridges between the Arab world and Europe. Special thanks to Dr. Nour Hariri for hosting us on her podcast! Nour Hariri

Just presented my paper titled “Can Al-Ghazali Solve the Problem of Fake News?” at the conference in Berlin 🇩🇪 Grateful to Der Divan for organizing this enriching event! Der Divan | الديوان Also had a great discussion with Stefan Weidner on challenges and building bridges between the Arab world and Europe. Special thanks to Dr. Nour Hariri for hosting us on her podcast! Nour Hariri

Mesfer Alhayyani | مسفر حمد الحياني

12,445 views • 1 year ago

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)

Animesh Garg

52,300 views • 2 years ago

we are officially working on opentui - a library for building terminal UIs in typescript it is built on the work of kmdrfx - we are sponsoring him to solve this problem once and for all clip is not sped up and it's running entirely in the terminal - link to repo in reply

we are officially working on opentui - a library for building terminal UIs in typescript it is built on the work of kmdrfx - we are sponsoring him to solve this problem once and for all clip is not sped up and it's running entirely in the terminal - link to repo in reply

dax

286,942 views • 1 year ago

One of my fav X plots. Sonic would just take the problem & solve it without telling anyone, which often led to him being labeled a villain. But for him, it literally makes no sense to explain things that seem so obvious. His sensitivity to changes in nature is also a good detail

One of my fav X plots. Sonic would just take the problem & solve it without telling anyone, which often led to him being labeled a villain. But for him, it literally makes no sense to explain things that seem so obvious. His sensitivity to changes in nature is also a good detail

Vito🌿

11,661 views • 8 months ago

Introducing RL Environment Creator Skill Now any one can create RL environments $ npx skills add adithya-s-k/RL_Envs_101 > You can create environments across multiple frameworks like OpenEnv, OpenReward, Verifiers, NemoGym ... > the repo has live working examples of environments that your coding agent can reference > The skill is design to first understand what type of model you are training and create an environment while keeping that in mind ps. There’s a lot more to building RL environments that can be used for training. One major aspect is the data, which this skill can’t directly solve. However, the skill will help with implementing tools, rewards, and other components of an RL environment, making it easier to go from idea to implementation quickly across different frameworks. Let me know if you’d be interested in a detailed, end-to-end blog/tutorial on building an environment and actually training a model for a useful use case.

Introducing RL Environment Creator Skill Now any one can create RL environments $ npx skills add adithya-s-k/RL_Envs_101 > You can create environments across multiple frameworks like OpenEnv, OpenReward, Verifiers, NemoGym ... > the repo has live working examples of environments that your coding agent can reference > The skill is design to first understand what type of model you are training and create an environment while keeping that in mind ps. There’s a lot more to building RL environments that can be used for training. One major aspect is the data, which this skill can’t directly solve. However, the skill will help with implementing tools, rewards, and other components of an RL environment, making it easier to go from idea to implementation quickly across different frameworks. Let me know if you’d be interested in a detailed, end-to-end blog/tutorial on building an environment and actually training a model for a useful use case.

Adithya S K

46,556 views • 2 months ago

The difference???? In ays jungkook asked for a new bottle because jm touched his mouth to it, and now we have jungkook drinking from the same bottle that taehyung also drank from 😭

The difference???? In ays jungkook asked for a new bottle because jm touched his mouth to it, and now we have jungkook drinking from the same bottle that taehyung also drank from 😭

wanessa

79,288 views • 2 months ago

Labor is delivering serious consequences for youth violence with Adult Time for Violent Crime. But the causes of violence are complex – we also need to invest in prevention to stop crime before it starts. That’s why we’re investing in Blue Light Victoria.

Labor is delivering serious consequences for youth violence with Adult Time for Violent Crime. But the causes of violence are complex – we also need to invest in prevention to stop crime before it starts. That’s why we’re investing in Blue Light Victoria.

Jacinta Allan

36,230 views • 1 month ago

For those paying attention to the Bengals' position battle at Right Guard: 🎥 Lucas Patrick missed a block & allowed pressure on 3rd & 5 (first possession). Joe Burrow - with arms in his face - still completed a pass to Ja'Marr Chase for 1st down. This was the first of 6 consecutive completions on the Bengals' opening TD drive. #Bengals WCPO 9

For those paying attention to the Bengals' position battle at Right Guard: 🎥 Lucas Patrick missed a block & allowed pressure on 3rd & 5 (first possession). Joe Burrow - with arms in his face - still completed a pass to Ja'Marr Chase for 1st down. This was the first of 6 consecutive completions on the Bengals' opening TD drive. #Bengals WCPO 9

Caleb Noe

55,355 views • 11 months ago

On my way to Ukraine, I met with Ireland’s Taoiseach Simon Harris TD. We discussed support for Ukraine and continued cooperation in demining and cybersecurity. We also talked about the next steps to implement the decisions based on the outcomes of the Peace Summit and discussed the efforts needed to increase global attention to the issue of children illegally deported by Russia. I thank Ireland for supporting the Peace Summit in Switzerland and the Taoiseach for his strong personal attention and commitment to the efforts to bring the children home. 🇺🇦 🇮🇪

On my way to Ukraine, I met with Ireland’s Taoiseach Simon Harris TD. We discussed support for Ukraine and continued cooperation in demining and cybersecurity. We also talked about the next steps to implement the decisions based on the outcomes of the Peace Summit and discussed the efforts needed to increase global attention to the issue of children illegally deported by Russia. I thank Ireland for supporting the Peace Summit in Switzerland and the Taoiseach for his strong personal attention and commitment to the efforts to bring the children home. 🇺🇦 🇮🇪

Volodymyr Zelenskyy / Володимир Зеленський

341,326 views • 2 years ago

🚨 GPT-5.6 features flag can be enabled in the codex > So we can see ui but can't use the model directly > Only trusted users are given access for now Personal life update: my adhd get worse with vacation as it fuck up my routine that's why I was not active lately, suggest something to solve this 🫂

🚨 GPT-5.6 features flag can be enabled in the codex > So we can see ui but can't use the model directly > Only trusted users are given access for now Personal life update: my adhd get worse with vacation as it fuck up my routine that's why I was not active lately, suggest something to solve this 🫂

Chetaslua

354,783 views • 26 days ago