Congratulations also to Patrick 👍 for his #ICLR paper... on Temporal Difference (TD) learning , in it We solve the decades-old puzzle of why TD can solve complex RL tasks that Gradient Descent cannot.show more

RL is back! But is it always the best... choice? In a new paper, we investigate under what circumstances neuroevolution outperforms RL in transfer learning tasks. See more details in the thread below 🧵 While NE performs best in simpler domains, it will be interesting to see if the lessons learned here can also be applied to more complex systems/tasks (LLMs?).show more

1/ While most RL methods use shallow MLPs (~2–5... layers), we show that scaling up to 1000-layers for contrastive RL (CRL) can significantly boost performance, ranging from doubling performance to 50x on a diverse suite of robotic tasks. Webpage+Paper+Code:show more

Elon Musk announced that Grok 4.7 will exceed all... current AI models when it launches in 3 to 4 weeks. The upcoming 2.1-trillion-parameter model is designed to prioritize superior token efficiency at the expense of serving speed. xAI plans to train the system on complex engineering data to solve challenging tasks in the physical world.show more

Yes, such a great BioRob meeting. Enjoyed catching up... with everyone. Great to see all the RL+biomechanics here. Also honored that our work on imitation learning for gait analysis was another best paper finalist Nice updates in MuJoCo coming soon!show more

#Browns Shedeur Sanders on his near TD pass to... Jerry Jeudy that was snatched out of his hands for an INT vs the #Bears in #Browns 31-3 loss:show more

Test-time learning has a catch that scaling alone does... not solve: the state a system writes to in order to learn is also the record a user must inspect to audit it. Optimize it only for learning, and auditability usually degrades. Blog: Part I treats the learning state and the audit record as the same object. This starts the ADAPT series, exploring the architecture section by section.show more

Ever watch an agent solve a problem with 100%... confidence and get it completely wrong? You can fix it… if you catch it. But then you need to watch every… single… run. New paper: We show that a meta-agent can do this for you, so your agents get ~2x more done on their own. 🧵show more

.Rachel Millward: "You cannot actually solve the housing crisis... in the current planning system because it begins and ends with profit. Developer profit is protected in this system. We need to have council housing at scale & that will take a change of govt legislation" 🎯show more

Just presented my paper titled “Can Al-Ghazali Solve the... Problem of Fake News?” at the conference in Berlin 🇩🇪 Grateful to Der Divan for organizing this enriching event! Der Divan | الديوان Also had a great discussion with Stefan Weidner on challenges and building bridges between the Arab world and Europe. Special thanks to Dr. Nour Hariri for hosting us on her podcast! Nour Haririshow more

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with... supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)show more

we are officially working on opentui - a library... for building terminal UIs in typescript it is built on the work of kmdrfx - we are sponsoring him to solve this problem once and for all clip is not sped up and it's running entirely in the terminal - link to repo in replyshow more

One of my fav X plots. Sonic would just... take the problem & solve it without telling anyone, which often led to him being labeled a villain. But for him, it literally makes no sense to explain things that seem so obvious. His sensitivity to changes in nature is also a good detailshow more

Introducing RL Environment Creator Skill Now any one can... create RL environments $ npx skills add adithya-s-k/RL_Envs_101 > You can create environments across multiple frameworks like OpenEnv, OpenReward, Verifiers, NemoGym ... > the repo has live working examples of environments that your coding agent can reference > The skill is design to first understand what type of model you are training and create an environment while keeping that in mind ps. There’s a lot more to building RL environments that can be used for training. One major aspect is the data, which this skill can’t directly solve. However, the skill will help with implementing tools, rewards, and other components of an RL environment, making it easier to go from idea to implementation quickly across different frameworks. Let me know if you’d be interested in a detailed, end-to-end blog/tutorial on building an environment and actually training a model for a useful use case.show more

The difference???? In ays jungkook asked for a new... bottle because jm touched his mouth to it, and now we have jungkook drinking from the same bottle that taehyung also drank from 😭show more

Labor is delivering serious consequences for youth violence with... Adult Time for Violent Crime. But the causes of violence are complex – we also need to invest in prevention to stop crime before it starts. That’s why we’re investing in Blue Light Victoria.show more

For those paying attention to the Bengals' position battle... at Right Guard: 🎥 Lucas Patrick missed a block & allowed pressure on 3rd & 5 (first possession). Joe Burrow - with arms in his face - still completed a pass to Ja'Marr Chase for 1st down. This was the first of 6 consecutive completions on the Bengals' opening TD drive. #Bengals WCPO 9show more

On my way to Ukraine, I met with Ireland’s... Taoiseach Simon Harris TD. We discussed support for Ukraine and continued cooperation in demining and cybersecurity. We also talked about the next steps to implement the decisions based on the outcomes of the Peace Summit and discussed the efforts needed to increase global attention to the issue of children illegally deported by Russia. I thank Ireland for supporting the Peace Summit in Switzerland and the Taoiseach for his strong personal attention and commitment to the efforts to bring the children home. 🇺🇦 🇮🇪show more