📢Introducing 360Anything, our method for lifting any perspective image or video to gravity-aligned 360° panoramas without using any camera or 3D information. This enables consistent novel view synthesis and 3D scene reconstruction. Project page: 🧵

*What is the task?* Unlike standard perspective data with a limited field of view, 360° panoramas capture the full scene with 360° horizontal and 180° vertical FoV. Our task: We take a perspective image or video and "hallucinate" the rest of that missing sphere. (2/N)

*Why panorama?* Standard video models struggle with object permanence—if a camera pans away and comes back, objects may disappear. With panoramas, the model is forced to generate everything in the scene. This serves as a "working memory" for consistent world generation. (3/N)

*Is our result 3D consistent?* Yes! We lift monocular video to 360° and directly train 3D Gaussian Splatting (3DGS) on it. 360Anything generates unobserved regions (e.g., behind the camera) with high consistency which blends seamlessly with observed areas in 3D. (4/N)

*Gravity-aligned generation* We always produce panoramas in a gravity-aligned canonical pose. Below we show results with large camera or object motion in the input video. 360Anything consistently tracks the camera trajectory indicated by the red border. (5/N)

*Method* Embracing the "bitter lesson", we replace geometric inductive biases in existing methods (e.g. camera projection) with a simple sequence-to-sequence DiT that learns camera geometry purely from data. 360Anything achieves new SOTA without requiring any camera info. (6/N)

Check out more results at: This work was done during my Student Researcher internship at @GoogleDeepMind Toronto, and it’s likely the last work of my PhD! Shout out to the amazing team @watson_nn, @taiyasaki, @fleet_dj, @marcusabrubaker, @srbhsxn! (N/N)

Big guy 🚀

Bonus: seam artifact is a long-standing issue in pano gen. We attribute this to the zero-padding in Conv when encoding image/video latents with VAE, and propose a fix. Below, we concat both sides of the panorama to show they are continuous. Check out our paper for details!

@Dazitu_616, your method for 360° panoramas is a clever approach for novel view synthesis; the project page will be useful.

cool work! Where can I experience the model with custom inputs?

Thanks for your interest! We are kicking off the code release process internally but nothing is guaranteed. Google has been strict about weight release of generative models (safety & copyright issues). Stay tuned!

Hey ziyi! I'm working on a project with @abundanceinst and this model is exactly what we need - I would love to talk to you about potential access! DM me!

It's been almost three months—haven't you decided to release it as open source yet?

🤯🤯🤯🤯🤯🤯🤯🤯🤯🤯🤯🤯

Hi. If you are looking for AI testers in Japan let me know. I would love to try it(if possible).

Will this generate Stereo 360°? Can it generate 180° stereo from a non fisheye mono or non fisheye stereo pair?

That would be an interesting direction--having more than a single input/output. Could be a future direction to explore (if the right data exists)

Wow. Super. Any change we can get the code to play with?

We are kicking off the code release process internally but nothing is guaranteed. Google has been strict about weight release of generative models (safety & copyright issues). Stay tuned!

Cool work!

still waiting for realesed

nice stuff! i found the "Circular Latent Encoding" technique quite ingenious to ensure the seams of the 360 panorama stay consistent

WTF ! 😲