Check out our #ICRA2024 paper "Actor-Critic Model Predictive Control." Model-free…
Davide Scaramuzza
在 X 上查看@davsca12万 位关注者
34,962 次观看 #ICRA2024 #reinforcementlearning #ModelPredictiveControl
Check out our #ICRA2024 paper "Actor-Critic Model Predictive Control." Model-free #reinforcementlearning (RL) is known for its strong task performance and flexibility in optimizing general reward formulations. On the other hand, #ModelPredictiveControl (MPC) benefits from robustness and online replanning capabilities. We combine both approaches by introducing a new framework called Actor-Critic Model Predictive Control. The key idea is to embed a differentiable MPC within an Actor-Critic RL framework. The proposed approach leverages the short-term predictive optimization capabilities of MPC with the exploratory and end-to-end training properties of RL. The resulting policy effectively manages both short-term decisions through the MPC-based actor and long-term prediction via the critic network, unifying the benefits of both model-based control and end-to-end learning. We validate our method in simulation and the real world with a quadcopter across various high-level tasks. We show that the proposed architecture can achieve real-time control performance, learn complex behaviors via trial and error, and retain the predictive properties of the MPC to better handle out-of-distribution behavior.
Paper:
Full Video with more details:
Kudos to Ángel Romero, Yunlong Song
IEEE ICRA University of Zurich UZH Science UZH Space Hub Aerial Core AUTOASSESS European Research Council (ERC)
4 条评论
评论来自 X (Twitter) 上的原帖

@_k_sridhar
Hi Davide, we wrote a paper with a very similar idea but focused on discrete optimization a year ago: I believe it’s highly relevant to your interesting ICRA paper.
7 个赞

@davsca1
Thanks for the pointer! We will read it!
1 个赞

@PeterSoetens
Davide, big fan of your work and publications, but the music in this video makes me nervous. I'm trying to focus on the contents and I get the constant feeling that something will explode any second. [/2cents]
1 个赞

@yrikk
Do you plan to open-source the code or code structure?
