Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Generative Novel View Synthesis with 3D-Aware Diffusion Models abs: project page:

AK

465,832 subscribers

304,708 views • 3 years ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

8 Comments

Tom Bielecki3 years ago

Maybe this can be used to enrich existing NeRFs with generative data?

Emmanuel3 years ago

AI already made all rotators obsolete @tszzl

Aakash Rao3 years ago

This is so cool

Aakash Rao3 years ago

Amazing work!!

Samurai Capital3 years ago

🤯

Anton3 years ago

This is huge!

Bullseye Web3 Studio3 years ago

Impressive (and scary)

Ryan Gordon3 years ago

generative novel view synthesis of fire hydrants

Related Videos

3D-aware Conditional Image Synthesis abs: project page:

3D-aware Conditional Image Synthesis abs: project page:

AK

301,457 views • 3 years ago

Towards Realistic Generative 3D Face Models abs: project page:

Towards Realistic Generative 3D Face Models abs: project page:

AK

48,089 views • 3 years ago

Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion abs: project page:

Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion abs: project page:

AK

262,780 views • 3 years ago

Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior abs: project page:

Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior abs: project page:

AK

163,868 views • 3 years ago

Diffuman4D 4D Consistent Human View Synthesis from Sparse-View Videos with Spatio-Temporal Diffusion Models

Diffuman4D 4D Consistent Human View Synthesis from Sparse-View Videos with Spatio-Temporal Diffusion Models

AK

12,518 views • 11 months ago

Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models turn the publicly available, state-of-the-art text-to-image LDM Stable Diffusion into an efficient and expressive text-to-video model with resolution up to 1280 x 2048 abs: project page:

Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models turn the publicly available, state-of-the-art text-to-image LDM Stable Diffusion into an efficient and expressive text-to-video model with resolution up to 1280 x 2048 abs: project page:

AK

718,682 views • 3 years ago

Single Motion Diffusion abs: project page:

Single Motion Diffusion abs: project page:

AK

72,767 views • 3 years ago

📢Introducing 360Anything, our method for lifting any perspective image or video to gravity-aligned 360° panoramas without using any camera or 3D information. This enables consistent novel view synthesis and 3D scene reconstruction. Project page: 🧵

📢Introducing 360Anything, our method for lifting any perspective image or video to gravity-aligned 360° panoramas without using any camera or 3D information. This enables consistent novel view synthesis and 3D scene reconstruction. Project page: 🧵

Ziyi Wu

46,859 views • 4 months ago

DreamBooth3D: Subject-Driven Text-to-3D Generation Personalized 3D models from just a few casual photos, with text-driven modifications abs: project page:

DreamBooth3D: Subject-Driven Text-to-3D Generation Personalized 3D models from just a few casual photos, with text-driven modifications abs: project page:

AK

107,503 views • 3 years ago

Dreamix: Video Diffusion Models are General Video Editors abs: project page: present diffusion-based method that is able to perform text-based motion and appearance editing of general videos

Dreamix: Video Diffusion Models are General Video Editors abs: project page: present diffusion-based method that is able to perform text-based motion and appearance editing of general videos

AK

398,132 views • 3 years ago

Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model abs: project page:

Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model abs: project page:

AK

90,072 views • 3 years ago

Scaling up GANs for Text-to-Image Synthesis present our 1B-parameter GigaGAN, achieving lower FID than Stable Diffusion v1.5, DALL·E 2, and Parti-750M. It generates 512px outputs at 0.13s, orders of magnitude faster than diffusion and autoregressive models, and inherits the disentangled, continuous, and controllable latent space of GANs abs: project page:

Scaling up GANs for Text-to-Image Synthesis present our 1B-parameter GigaGAN, achieving lower FID than Stable Diffusion v1.5, DALL·E 2, and Parti-750M. It generates 512px outputs at 0.13s, orders of magnitude faster than diffusion and autoregressive models, and inherits the disentangled, continuous, and controllable latent space of GANs abs: project page:

AK

278,115 views • 3 years ago

🍺 LagerNVS (CVPR 2026) 🍺 LagerNVS is a generalizable, feed-forward, real-time Novel View Synthesis network which - performs rendering in real time, - generalizes to in-the-wild data, - works with and without known source cameras, - sets a new state-of-the-art among deterministic methods, - can be paired with a diffusion decoder for generative extrapolation. LagerNVS shows that 3D biases are useful for Novel View Synthesis but explicit 3D representations are not required to achieve them. We use 3D biases in (1) architecture design and (2) pre-training: (1) In NVS with explicit 3D representations (3DGS, NeRF) reconstruction is typically difficult and slow, but rendering is much faster and simpler. We mimic this process in the network design: we use a large (1B params) encoder and a small, lightweight decoder (ViT-B). This allows increasing the network capacity while still achieving real-time rendering. (2) The encoder, initialized from VGGT, was pre-trained with 3D reconstruction objectives, making the initial features 3D aware. Both substantially improve performance. Project page: Code: Paper: Models: Work done with Jianyuan Minghao Chen Christian Rupprecht and Andrea Vedaldi

🍺 LagerNVS (CVPR 2026) 🍺 LagerNVS is a generalizable, feed-forward, real-time Novel View Synthesis network which - performs rendering in real time, - generalizes to in-the-wild data, - works with and without known source cameras, - sets a new state-of-the-art among deterministic methods, - can be paired with a diffusion decoder for generative extrapolation. LagerNVS shows that 3D biases are useful for Novel View Synthesis but explicit 3D representations are not required to achieve them. We use 3D biases in (1) architecture design and (2) pre-training: (1) In NVS with explicit 3D representations (3DGS, NeRF) reconstruction is typically difficult and slow, but rendering is much faster and simpler. We mimic this process in the network design: we use a large (1B params) encoder and a small, lightweight decoder (ViT-B). This allows increasing the network capacity while still achieving real-time rendering. (2) The encoder, initialized from VGGT, was pre-trained with 3D reconstruction objectives, making the initial features 3D aware. Both substantially improve performance. Project page: Code: Paper: Models: Work done with Jianyuan Minghao Chen Christian Rupprecht and Andrea Vedaldi

Stan Szymanowicz

31,404 views • 2 months ago

VSTAR Generative Temporal Nursing for Longer Dynamic Video Synthesis Despite tremendous progress in the field of text-to-video (T2V) synthesis, open-sourced T2V diffusion models struggle to generate longer videos with dynamically varying and evolving content. They tend to

VSTAR Generative Temporal Nursing for Longer Dynamic Video Synthesis Despite tremendous progress in the field of text-to-video (T2V) synthesis, open-sourced T2V diffusion models struggle to generate longer videos with dynamically varying and evolving content. They tend to

AK

36,982 views • 2 years ago

Oh wow! "[SIGGRAPH '24] DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models" Paper: Project: Code (MIT) :

Oh wow! "[SIGGRAPH '24] DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models" Paper: Project: Code (MIT) :

MrNeRF

59,480 views • 2 years ago

🚀High-Res 3D Text-to-Texture🚀 🎨#MVPaint delivers high-res 3D textures with minimal UV wrapping dependency and high versatility using synchronized multi-view diffusion 📑 Paper: 🏠 Project Page: 🧑‍💻 Code:

🚀High-Res 3D Text-to-Texture🚀 🎨#MVPaint delivers high-res 3D textures with minimal UV wrapping dependency and high versatility using synchronized multi-view diffusion 📑 Paper: 🏠 Project Page: 🧑‍💻 Code:

Ziwei Liu

28,122 views • 1 year ago

Excited to share Norman Müller's DiffRF: Rendering-guided 3D Radiance Field Diffusion #CVPR2023 highlight! 2D diffusion is great, but what about 3D? We show radiance field diffusion with rendering guidance for consistent and editable 3D synthesis. Vid:

Excited to share Norman Müller's DiffRF: Rendering-guided 3D Radiance Field Diffusion #CVPR2023 highlight! 2D diffusion is great, but what about 3D? We show radiance field diffusion with rendering guidance for consistent and editable 3D synthesis. Vid:

Matthias Niessner

117,759 views • 3 years ago

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

Sherwin Bahmani

66,417 views • 8 months ago

🚀 Introducing InterDyn — our newly accepted CVPR work that explores controllable synthesis of interactive dynamics! Building upon powerful video diffusion models, InterDyn infers future motion and interactions directly from an input image and a dynamic control signal (e.g., a moving hand mask). Check out how we push the boundaries of intuitive physics with video generative models. Project page: Arxiv: #GenAI #AIGC #VideoGen #ML #ComputerVision #CVPR2025 🧵1/6

🚀 Introducing InterDyn — our newly accepted CVPR work that explores controllable synthesis of interactive dynamics! Building upon powerful video diffusion models, InterDyn infers future motion and interactions directly from an input image and a dynamic control signal (e.g., a moving hand mask). Check out how we push the boundaries of intuitive physics with video generative models. Project page: Arxiv: #GenAI #AIGC #VideoGen #ML #ComputerVision #CVPR2025 🧵1/6

Haven (Haiwen) Feng

44,898 views • 1 year ago

SceneDreamer: Unbounded 3D Scene Generation from 2D Image Collections abs: project page:

SceneDreamer: Unbounded 3D Scene Generation from 2D Image Collections abs: project page:

AK

317,619 views • 3 years ago