Happy to share our new work on Navigation World Models! 🔥🔥 Navigation is a fundamental skill of agents with visual-motor capabilities. We train a single World Model across multiple environments and diverse agent data. w/ Gaoyue Zhou, Danny Tran, trevordarrell and Yann LeCun.

Our Navigation World Model can simulate trajectories by generating video. This capability unlocks planning: simply find the sequence of actions that leads from the input image to a target goal. In unknown environments, our model can hallucinate navigation trajectories.

For more information, see our paper and project page: Project Page: Preprint: work is a collaboration between @AIatMeta and @berkeley_ai 😊

@berkeley_ai A lot of our work is also built on the work of others. Our Conditional Diffusion Transformer model (CDiT) extends DiT by @billpeeb and @sainingxie, and much of the data and inspiration is based on the works of our @berkeley_ai colleagues Noriaki Hirose and @shahdhruv_

@berkeley_ai @billpeeb @sainingxie @shahdhruv_ Finally, it is exciting to see the space of world models blowing up in the past week with other cool works from @theworldlabs and the genie project by @_rockt @GoogleDeepMind. Looking forward to what comes next!

@GaoyueZhou @trevordarrell @ylecun sick

@GaoyueZhou @trevordarrell @ylecun Thank you sir 🙏🫡

@GaoyueZhou @trevordarrell @ylecun Super cool! Btw have u come across this dataset? DM me if u r keen on our next big release 🙏

@GaoyueZhou @trevordarrell @ylecun