Heng Yang
@hankyang94 • 5,005 subscribers
Assistant Professor @Harvard SEAS @hseas, Lead the Harvard Computational Robotics Lab. #Robotics, #Optimization, #Control, #Vision, #Learning
Shorts
Videos
Glad that our work “Inference-Time Enhancement of Generative Robot Policies via Predictive World Modeling”, led by Han Qi, has been accepted to IEEE Robotics and Automation Letters! 🎉 We propose Generative Predictive Control (GPC): sample action proposals from a pretrained diffusion policy (“look back”), roll them out with a diffusion-based action-conditioned video world model (“look forward”), then rank or optimize the actions using either a learned reward model or VLM preferences. Conceptually, this is trajectory optimization / MPC with hybrid sampling + gradient optimization, interpreted through modern diffusion priors and video world models. Interestingly, we first posted the paper on arXiv in Feb 2025, when action-conditioned video world models for planning were still rare—now this direction is rapidly gaining traction. Still many open questions, e.g., • how to avoid local minima in planning • what representations work best for world models • how to balance physics priors vs. data-driven learning Paper:
Heng Yang18,994 次观看 • 4 个月前
Diffusion has shown great promise for generating robot **actions**, can it act as a **world model** to generate the future conditioned on actions? In our work led by han qi Haocheng Yin and in collaboration with Yilun Du, we show a **controllable** action-conditioned video diffusion model can produce photorealistic and (near) physics-accurate future predictions. This ability strengthens the policy via: - ranking different action proposals and selecting the best, or - **visual** trajectory optimization by optimizing the action proposals using gradient ascent. Learn more about Generative Predictive Control (GPC) at:
Heng Yang38,428 次观看 • 1 年前
"Building Rome with Convex Optimization" has been accepted to #RSS2025! Try XM, our new structure from motion pipeline powered by GPU-accelerated convex semidefinite optimization: XM solves large-scale (nonconvex) global bundle adjustment problem via learned depth and a tight convex semidefinite relaxation. By implementing the Burer-Monteiro low-rank factorization algorithm in CUDA, XM can solve bundle adjustment problems with more than 10,000 images/views. Technical details in the paper: Kudos to Haoyu Han
Heng Yang27,486 次观看 • 1 年前
没有更多内容可加载