跳过导航
Introducing ✨RigidFormer: Learning Rigid Dynamics with Transformers - our attempt to scale…
0:00 / 0:26

Introducing ✨RigidFormer: Learning Rigid Dynamics with Transformers - our attempt to scale…

Zhiyang (Frank) Dou

@frankzydou2463 位关注者

在 X 上查看
298 个赞下载原帖

583,988 次观看

Introducing ✨RigidFormer: Learning Rigid Dynamics with Transformers - our attempt to scale learning-based physical dynamics with Transformers. RigidFormer learns rigid dynamics with Transformers. It is a mesh-free, object-centric Transformer for multi-object rigid-body contact dynamics from point clouds. Learning physics with purely neural simulators, without relying on traditional physics engines, is an important and widely studied problem. Prior SOTA methods often use graph neural networks for accuracy and generalization, but still struggle with efficient, high-fidelity simulation at scale. RigidFormer uses only point inputs, matches or outperforms mesh-based baselines on standard benchmarks, runs much faster, generalizes across point resolutions and datasets, and scales to 200+ objects. We also show a preliminary extension to command-conditioned articulated bodies by treating body parts as interacting object-level components. RigidFormer is mesh-free: it does not require mesh connectivity, SDFs, or vertex-level message passing, making it well-suited for point-cloud observations and scalable simulation. This architecture can also be adapted to learn soft-body dynamics by replacing the rigid-body module (differentiable Kabsch alignment). 🎬See our video for more details. Many thanks to my amazing collaborators: Minghao Guo Minghao Guo, Haixu Wu Haixu Wu, Doug Roble, Tuur Stuyck Tuur Stuyck, and Wojciech Matusik Wojciech Matusik. Project page: Paper:

8 条评论

评论来自 X (Twitter) 上的原帖

  • Zhiyang (Frank) Dou 的头像

    @frankzydou

    One direction we are especially excited about is extending this type of dynamics modeling with richer action conditioning (I showcased action-condition dynamics modeling for humanoid robots in the video). We believe that such object-centric neural dynamics models can serve as a useful building block for future world models, where physical interaction, control signals, and scene-level/object-level dynamics are learned jointly from observations in a scalable way.

    6 个赞

  • Yushi LAN 的头像

    @yushilan98

    Impressive work!

    1 个赞

  • Zhiyang (Frank) Dou 的头像

    @frankzydou

    @GROS17121524 Thank you, Yushi!

    1 个赞

  • Nurvai - The Data Layer for Physical AI 的头像

    @nurvai_ai

    Mesh free, object centric dynamics feels like a really important direction for scalable learned physics. Interesting that transformers are starting to compete with graph approaches here. How far do you think this can scale before physical consistency becomes harder to preserve?

    1 个赞

  • Arnon ∈ ℝ³ 的头像

    @desert_mouse

    Amazing work, physical understanding is the way forward

    1 个赞

  • Shilong Liu 的头像

    @Shilong_Liu_AI

    impressive!

    1 个赞

  • Zhiyang (Frank) Dou 的头像

    @frankzydou

    @atasteoff Thanks, Shilong!

  • EB1A Experts 的头像

    @eb1aexperts

    Impressive work!