Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

How robust can model predictive control be if we can solve each trajectory optimization to global optimality? On the contact-rich push-T problem, we show that model-based global optimization is so robust that it never fails, even if the model is not even correct! We achieve global optimality via sparse...

28,718 görüntüleme • 1 yıl önce •via X (Twitter)

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)

Animesh Garg

52,300 görüntüleme • 2 yıl önce

Dear Tarun Chitra 1. We are the original creators of DeSci back in 2016. What DeSci has become today is largely unrelated with its original model of producing rigorous peer-reviewed scientific studies published in reputable medical journals. The model we introduced. We are tirelessly fighting against pseudoscience, and we are showing the world that people can understand the difference between legit science and pseudoscience with the success of $INNBCV. Yes, meritocracy is possible in crypto. Even against all odds. 2. We are the only project in the entire crypto space that ever funded, performed, and published highly innovative HIV cure research ( We are the project that produced the first peer-reviewed study on blockchain-based biomedical data storage in the world’s most reputable scientific network, Springer Nature ( $INNBCV is not for the privileged few; it is for the many. We resisted all the pressure from those who wanted us to provide big allocations to VIPs of other DAOs “because it is good for the marketing” and put our users first, ensuring a fair launch, a launch for the people, and they turned $70k into $2,000,000. $INNBCV shows that you can have a sustainable model, provided you are backed by actual science. And thanks to the amazing guys at daos.fun baoskee and Solana community. Behind our project there is the sweat and blood of years of work to produce publications in the most reputable medical journals. Just to put things into perspective, it took us 3 years to publish our latest work in Springer Nature. 3. Unlike many other projects, we had no ICO/VCs, meaning we had to prove ourselves every single day because we are only supported by our community. If we deliver products, we survive; it is either publish or perish for us, and that’s why we have such a close connection to our community. $INNBCV is a struggler, $INNBCV is a survivor, $INNBCV is not for the privilege of the few but for the people. Our community makes it possible by supporting us. You guys are the real heroes.

InnovativeBioresearch🇮🇹

10,867 görüntüleme • 1 yıl önce