跳过导航
Check out our #ICRA2024 paper "Actor-Critic Model Predictive Control." Model-free…
0:00 / 1:26

Check out our #ICRA2024 paper "Actor-Critic Model Predictive Control." Model-free…

Davide Scaramuzza

@davsca12万 位关注者

在 X 上查看
441 个赞下载原帖

34,962 次观看 #ICRA2024 #reinforcementlearning #ModelPredictiveControl

Check out our #ICRA2024 paper "Actor-Critic Model Predictive Control." Model-free #reinforcementlearning (RL) is known for its strong task performance and flexibility in optimizing general reward formulations. On the other hand, #ModelPredictiveControl (MPC) benefits from robustness and online replanning capabilities. We combine both approaches by introducing a new framework called Actor-Critic Model Predictive Control. The key idea is to embed a differentiable MPC within an Actor-Critic RL framework. The proposed approach leverages the short-term predictive optimization capabilities of MPC with the exploratory and end-to-end training properties of RL. The resulting policy effectively manages both short-term decisions through the MPC-based actor and long-term prediction via the critic network, unifying the benefits of both model-based control and end-to-end learning. We validate our method in simulation and the real world with a quadcopter across various high-level tasks. We show that the proposed architecture can achieve real-time control performance, learn complex behaviors via trial and error, and retain the predictive properties of the MPC to better handle out-of-distribution behavior. Paper: Full Video with more details: Kudos to Ángel Romero, Yunlong Song IEEE ICRA University of Zurich UZH Science UZH Space Hub Aerial Core AUTOASSESS European Research Council (ERC)

4 条评论

评论来自 X (Twitter) 上的原帖

  • Kaustubh Sridhar 的头像

    @_k_sridhar

    Hi Davide, we wrote a paper with a very similar idea but focused on discrete optimization a year ago: I believe it’s highly relevant to your interesting ICRA paper.

    7 个赞

  • Davide Scaramuzza 的头像

    @davsca1

    Thanks for the pointer! We will read it!

    1 个赞

  • Peter Soetens⚡ 的头像

    @PeterSoetens

    Davide, big fan of your work and publications, but the music in this video makes me nervous. I'm trying to focus on the contents and I get the constant feeling that something will explode any second. [/2cents]

    1 个赞

  • Yura Kriachko 的头像

    @yrikk

    Do you plan to open-source the code or code structure?