Загрузка видео...

Не удалось загрузить видео

На главную

Thanks AK! Finally, robot can do continuous, agile, autonomous, adaptive jumping over stair and stepping stone Key idea: combine the pros of model-free RL and model-based control. RL (for CoM refs) + QP (for GRF) + WBC (for torque) Open-sourced:

32,155 просмотров • 1 год назад •via X (Twitter)

Комментарии: 4

Фото профиля Allen Z. Ren
Allen Z. Ren1 год назад

@_akhaliq It reminds me I was trying to get the Minitaur jumping the stairs (scripted controller) somewhere in Hamerschlag I think in 2018...Amazing to see how both software and hardware have gotten so much better since then!

Фото профиля Yaoyao(Freax) Qian
Yaoyao(Freax) Qian1 год назад

@_akhaliq Interesting work!

Фото профиля Kashu Yamazaki
Kashu Yamazaki1 год назад

@_akhaliq This is impressive!

Фото профиля Heng (Alfredo)Zhang
Heng (Alfredo)Zhang1 год назад

@_akhaliq very impressive!

Похожие видео

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)

Animesh Garg

52,308 просмотров • 2 лет назад

LEONARDO, also called LEO, was built by researchers at Caltech’s Center for Autonomous Systems and Technologies. Its full name means LEgs ONboARD drOne. The idea is simple but unusual: • Build a small biped robot • Give it drone-style thrust • Use the legs for ground contact • Use the propellers for balance and lift • Combine walking, hopping and flying in one system LEO is basically a hybrid between a walking robot and a flying drone. How it was built: • Two lightweight legs • Three actuated joints in each leg • Four propeller thrusters near the shoulders • A lightweight body • Leg motors for ground movement • Propellers for balance, lift and aerial control • Real-time control software that synchronizes the legs and propellers How it walks: • The legs move the robot forward • The feet touch the ground like a normal biped • The propellers constantly correct balance from above • The robot can stay upright even in unstable situations • The thrust reduces the risk of falling during difficult motions How it flies: • The legs stop being the main locomotion system • The four propellers generate lift • The robot behaves more like a drone • It can take off, fly over obstacles and land back on its legs What makes it different: • It does not walk like a normal humanoid • It does not fly like a normal drone • It blends both systems • The legs handle contact with the ground • The propellers act like fast stabilizers • The control system decides how much help comes from the legs and how much comes from thrust That is why LEO can: • Walk • Hop • Fly over obstacles • Ride a skateboard • Balance on a slackline The key idea is walking with aerial stabilization.

Techniahqrobot | humanoid robots

135,515 просмотров • 1 месяц назад

🧬 We have many foundation models or language models for DNAs, but can we control them? We introduce Ctrl-DNA: Controllable Cell-Type-Specific Regulatory DNA Design via Constrained RL — a reinforcement learning framework for controllable cis-regulatory sequence generation. Paper: Code: 🔬What’s the challenge? Designing regulatory DNA that is both highly expressive in target cell types and inactive in others is essential for synthetic biology, gene therapy, and precision medicine. Yet, controlling these trade-offs is challenging due to sparse, sequence-level rewards and biological constraints. 🔥Why Ctrl-DNA? Ctrl-DNA fine-tunes pre-trained DNA language models using a value model free, Lagrangian-guided RL framework, enabling flexible and customizable constraint optimization. Users can define application-specific thresholds across cell types, balancing expression strength with specificity. ✅ Maximize target-cell expression ✅ Constrain off-target activity under user-defined thresholds ✅ Preserve cell-type-specific TF motif structure Benchmarked on human enhancer and promoter datasets, Ctrl-DNA consistently outperforms prior methods, achieving stronger specificity, higher fitness, and more biologically grounded sequence generation — all with direct control over regulatory trade-offs. Shoutout to the PhD students Xingyu Chen (Xingyu Chen ) and Rex Ma (Rex Ma) for their amazing work leading this project!

Bo Wang

30,719 просмотров • 1 год назад