Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

📢📢📢 𝐀𝐂𝟑𝐃: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers TL;DR: for 3D camera control in generative video, it really helps knowing which part of your model you should mess with Internship by Sherwin Bahmani at Snap

Andrea Tagliasacchi 🇨🇦→✈️→🇺🇲 (@CVPR)

16,653 subscribers

23,040 görüntüleme • 1 yıl önce •via X (Twitter)

Bilim & Teknoloji Haberler & Politika Eğitim

Anya Rossi• Live Now

Private livecam show

5 Yorum

Andrea Tagliasacchi 🇨🇦 profil fotoğrafı

Andrea Tagliasacchi 🇨🇦1 yıl önce

TL;DR (expanded): 1) "when" in the diffusion process you condition for camera matters (i.e. noise scheduler) 2) "how" in the diffusion process you condition for camera maters (i.e. architecture) 3) "what data" you give to your diffusion model to condition camera matters

Andrea Tagliasacchi 🇨🇦 profil fotoğrafı

Andrea Tagliasacchi 🇨🇦1 yıl önce

Why, you ask? 1) camera motion is low-frequency... early denoising iterations deal with low-frequency content 2) early DiT blocks are enough to fine-tune for camera control... more and you lose quality 3) model needs to know what a static view of the dynamic world looks like

Andrea Tagliasacchi 🇨🇦 profil fotoğrafı

Andrea Tagliasacchi 🇨🇦1 yıl önce

A shout to the collaborators @isskoro @guocheng_qian A. Siarohin @willimenapace @SergeyTulyakov at Snap and @DaveLindell at UofT.

Samarth Sinha profil fotoğrafı

Samarth Sinha1 yıl önce

@sherwinbahmani Congrats @sherwinbahmani !!

Abdullah Hamdi profil fotoğrafı

Abdullah Hamdi1 yıl önce

@sherwinbahmani Congrats to the team ! Amazing work

Benzer Videolar

📢Excited to be at #ICLR2025 for our paper: VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control Poster: Thu 3-5:30 PM (#134) Website: Code: Also check out our #CVPR2025 follow-up AC3D:

📢Excited to be at #ICLR2025 for our paper: VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control Poster: Thu 3-5:30 PM (#134) Website: Code: Also check out our #CVPR2025 follow-up AC3D:

Sherwin Bahmani

11,560 görüntüleme • 1 yıl önce

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

Sherwin Bahmani

66,417 görüntüleme • 8 ay önce

Generative 3D worlds are amazing! Here is a generative AI video showing fine grained camera control. Camera movements are all hand places within a generated 3D world.

Generative 3D worlds are amazing! Here is a generative AI video showing fine grained camera control. Camera movements are all hand places within a generated 3D world.

Brittani Natali

21,043 görüntüleme • 1 yıl önce

📢𝐋𝟑𝐃𝐆: 𝐋𝐚𝐭𝐞𝐧𝐭 𝟑𝐃 𝐆𝐚𝐮𝐬𝐬𝐢𝐚𝐧 𝐃𝐢𝐟𝐟𝐮𝐬𝐢𝐨𝐧📢 #SIGGRAPHAsia We propose a generative diffusion model for 3D Gaussians. Key is a learnt latent space which substantially reduces the complexity of the diffusion process, thus facilitating room-scale scene generation! Great work by Barbara Roessle in with Norman Müller, Angela Dai, Lorenzo Porzi, Samuel Rota Bulò, Peter Kontschieder!

📢𝐋𝟑𝐃𝐆: 𝐋𝐚𝐭𝐞𝐧𝐭 𝟑𝐃 𝐆𝐚𝐮𝐬𝐬𝐢𝐚𝐧 𝐃𝐢𝐟𝐟𝐮𝐬𝐢𝐨𝐧📢 #SIGGRAPHAsia We propose a generative diffusion model for 3D Gaussians. Key is a learnt latent space which substantially reduces the complexity of the diffusion process, thus facilitating room-scale scene generation! Great work by Barbara Roessle in with Norman Müller, Angela Dai, Lorenzo Porzi, Samuel Rota Bulò, Peter Kontschieder!

Matthias Niessner

39,489 görüntüleme • 1 yıl önce

Full creative control is here in 3D-to-Video. 3D-to-Video lets you manage every detail, from objects to camera angles, with total precision. Consistency is guaranteed.

Full creative control is here in 3D-to-Video. 3D-to-Video lets you manage every detail, from objects to camera angles, with total precision. Consistency is guaranteed.

MeshyAI

11,276 görüntüleme • 8 ay önce

Light-X Generative 4D Video Rendering with Camera and Illumination Control

Light-X Generative 4D Video Rendering with Camera and Illumination Control

AK

31,126 görüntüleme • 6 ay önce

📢Animating the Uncaptured 📢 We animate 3D humanoid meshes using video diffusion priors given a text prompt. 🎥 🌍 Realistic motion generation for 3D characters - without motion capture! 🚀 Great work by Marc Benedí Angela Dai

📢Animating the Uncaptured 📢 We animate 3D humanoid meshes using video diffusion priors given a text prompt. 🎥 🌍 Realistic motion generation for 3D characters - without motion capture! 🚀 Great work by Marc Benedí Angela Dai

Matthias Niessner

11,696 görüntüleme • 1 yıl önce

and Wan2.2 again. MoCam - novel view synthesis. Lets you change the camera trajectory of a photo/video. - Video Re-camera - 3D Photo Conversion - Cinematic Control - Precise orbital/pan/zoom control - Corrects geometric artifacts

and Wan2.2 again. MoCam - novel view synthesis. Lets you change the camera trajectory of a photo/video. - Video Re-camera - 3D Photo Conversion - Cinematic Control - Precise orbital/pan/zoom control - Corrects geometric artifacts

Wildminder

13,720 görüntüleme • 1 ay önce

Bring your stories to life with a 3D camera. Start with a single frame and turn it into a 3D scene you can move through, shot by shot. Control the camera. Set the pace. Shape the story.

Bring your stories to life with a 3D camera. Start with a single frame and turn it into a 3D scene you can move through, shot by shot. Control the camera. Set the pace. Shape the story.

Moonvalley

18,713 görüntüleme • 10 ay önce

Nvidia just dropped GEN3C 3D-Informed World-Consistent Video Generation with Precise Camera Control

Nvidia just dropped GEN3C 3D-Informed World-Consistent Video Generation with Precise Camera Control

AK

58,146 görüntüleme • 1 yıl önce

📢📢📢 thrilled to announce "𝟒𝐃-𝐟𝐲: 𝐓𝐞𝐱𝐭-𝐭𝐨-𝟒𝐃 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐨𝐧 𝐔𝐬𝐢𝐧𝐠 𝐇𝐲𝐛𝐫𝐢𝐝 𝐒𝐜𝐨𝐫𝐞 𝐃𝐢𝐬𝐭𝐢𝐥𝐥𝐚𝐭𝐢𝐨𝐧 𝐒𝐚𝐦𝐩𝐥𝐢𝐧𝐠" Way to start Sherwin Bahmani 🎉 PhD co-advised with David Lindell at #UofT.

📢📢📢 thrilled to announce "𝟒𝐃-𝐟𝐲: 𝐓𝐞𝐱𝐭-𝐭𝐨-𝟒𝐃 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐨𝐧 𝐔𝐬𝐢𝐧𝐠 𝐇𝐲𝐛𝐫𝐢𝐝 𝐒𝐜𝐨𝐫𝐞 𝐃𝐢𝐬𝐭𝐢𝐥𝐥𝐚𝐭𝐢𝐨𝐧 𝐒𝐚𝐦𝐩𝐥𝐢𝐧𝐠" Way to start Sherwin Bahmani 🎉 PhD co-advised with David Lindell at #UofT.

Andrea Tagliasacchi @CVPR

74,631 görüntüleme • 2 yıl önce

One of the hardest things to achieve with AI is precise character motion. The new model by Kinetix, Kamo-1, is amazing at giving you far more control over your generations. It’s also the first 3D-conditioned model, so it understands the scene in 3D and gives you almost unlimited camera motion. Let me show you how to use it 👇

One of the hardest things to achieve with AI is precise character motion. The new model by Kinetix, Kamo-1, is amazing at giving you far more control over your generations. It’s also the first 3D-conditioned model, so it understands the scene in 3D and gives you almost unlimited camera motion. Let me show you how to use it 👇

Everett World

19,199 görüntüleme • 6 ay önce

"YoNoSplat: You Only Need One Model for Feedforward 3D Gaussian Splatting" TL;DR: a unified 3D Gaussian splatting model that reconstructs high-quality scene geometry and camera poses from unposed/uncalibrated images in a single forward pass.

"YoNoSplat: You Only Need One Model for Feedforward 3D Gaussian Splatting" TL;DR: a unified 3D Gaussian splatting model that reconstructs high-quality scene geometry and camera poses from unposed/uncalibrated images in a single forward pass.

Alexandre Morgand

14,839 görüntüleme • 3 ay önce

📢📢📢 𝐀𝐜𝐜𝐞𝐥𝐞𝐫𝐚𝐭𝐢𝐧𝐠 𝐍𝐞𝐮𝐫𝐚𝐥 𝐅𝐢𝐞𝐥𝐝 𝐓𝐫𝐚𝐢𝐧𝐢𝐧𝐠 𝐯𝐢𝐚 𝐒𝐨𝐟𝐭 𝐌𝐢𝐧𝐢𝐧𝐠 by Shakiba Kheradmand et al. TL;DR: importance sampling for accelerating your novel view synthesis workloads (...yes, it should also work for 3DGS)

📢📢📢 𝐀𝐜𝐜𝐞𝐥𝐞𝐫𝐚𝐭𝐢𝐧𝐠 𝐍𝐞𝐮𝐫𝐚𝐥 𝐅𝐢𝐞𝐥𝐝 𝐓𝐫𝐚𝐢𝐧𝐢𝐧𝐠 𝐯𝐢𝐚 𝐒𝐨𝐟𝐭 𝐌𝐢𝐧𝐢𝐧𝐠 by Shakiba Kheradmand et al. TL;DR: importance sampling for accelerating your novel view synthesis workloads (...yes, it should also work for 3DGS)

Andrea Tagliasacchi 🇨🇦

21,810 görüntüleme • 2 yıl önce

🚀Excited to introduce GEN3C #CVPR2025, a generative video model with an explicit 3D cache for precise camera control. 🎥It applies to multiple use cases, including single-view and sparse-view NVS🖼️ and challenging settings like monocular dynamic NVS and driving simulation🚗. Project page:

🚀Excited to introduce GEN3C #CVPR2025, a generative video model with an explicit 3D cache for precise camera control. 🎥It applies to multiple use cases, including single-view and sparse-view NVS🖼️ and challenging settings like monocular dynamic NVS and driving simulation🚗. Project page:

Xuanchi Ren

59,931 görüntüleme • 1 yıl önce

V3D Video Diffusion Models are Effective 3D Generators Automatic 3D generation has recently attracted widespread attention. Recent methods have greatly accelerated the generation speed, but usually produce less-detailed objects due to limited model capacity or 3D data. Motivated by recent advancements in video diffusion models, we introduce V3D, which leverages the world simulation capacity of pre-trained video diffusion models to facilitate 3D generation. To fully unleash the potential of video diffusion to perceive the 3D world, we further introduce geometrical consistency prior and extend the video diffusion model to a multi-view consistent 3D generator. Benefiting from this, the state-of-the-art video diffusion model could be fine-tuned to generate 360degree orbit frames surrounding an object given a single image. With our tailored reconstruction pipelines, we can generate high-quality meshes or 3D Gaussians within 3 minutes. Furthermore, our method can be extended to scene-level novel view synthesis, achieving precise control over the camera path with sparse input views. Extensive experiments demonstrate the superior performance of the proposed approach, especially in terms of generation quality and multi-view consistency

V3D Video Diffusion Models are Effective 3D Generators Automatic 3D generation has recently attracted widespread attention. Recent methods have greatly accelerated the generation speed, but usually produce less-detailed objects due to limited model capacity or 3D data. Motivated by recent advancements in video diffusion models, we introduce V3D, which leverages the world simulation capacity of pre-trained video diffusion models to facilitate 3D generation. To fully unleash the potential of video diffusion to perceive the 3D world, we further introduce geometrical consistency prior and extend the video diffusion model to a multi-view consistent 3D generator. Benefiting from this, the state-of-the-art video diffusion model could be fine-tuned to generate 360degree orbit frames surrounding an object given a single image. With our tailored reconstruction pipelines, we can generate high-quality meshes or 3D Gaussians within 3 minutes. Furthermore, our method can be extended to scene-level novel view synthesis, achieving precise control over the camera path with sparse input views. Extensive experiments demonstrate the superior performance of the proposed approach, especially in terms of generation quality and multi-view consistency

AK

31,997 görüntüleme • 2 yıl önce

📢 A Recipe for Generating 3D Worlds From a Single Image 📢 Our recipe explains how existing generative models can be adapted with minimal training effort to generate 3D worlds from a single input image.

📢 A Recipe for Generating 3D Worlds From a Single Image 📢 Our recipe explains how existing generative models can be adapted with minimal training effort to generate 3D worlds from a single input image.

Katja Schwarz

13,970 görüntüleme • 1 yıl önce

🎥 Take Full Control of the Camera! 🚨 I got early access to the new feature Camera Control with T2V 01 - Director Model by Hailuo AI (MiniMax) the ultimate tool for full camera control in video generation! ✨ What’s exciting? Direct with ease: Use natural language to guide your camera exactly how you want Smooth transitions: Combine multiple moves seamlessly for cinematic perfection Here are 10 sample prompts 🔖Bookmark for later!

🎥 Take Full Control of the Camera! 🚨 I got early access to the new feature Camera Control with T2V 01 - Director Model by Hailuo AI (MiniMax) the ultimate tool for full camera control in video generation! ✨ What’s exciting? Direct with ease: Use natural language to guide your camera exactly how you want Smooth transitions: Combine multiple moves seamlessly for cinematic perfection Here are 10 sample prompts 🔖Bookmark for later!

MayorkingAI

26,819 görüntüleme • 1 yıl önce

📢Introducing 360Anything, our method for lifting any perspective image or video to gravity-aligned 360° panoramas without using any camera or 3D information. This enables consistent novel view synthesis and 3D scene reconstruction. Project page: 🧵

📢Introducing 360Anything, our method for lifting any perspective image or video to gravity-aligned 360° panoramas without using any camera or 3D information. This enables consistent novel view synthesis and 3D scene reconstruction. Project page: 🧵

Ziyi Wu

46,859 görüntüleme • 4 ay önce

this is the world's first 3D AI Studio Rendora AI can generate fully editable 3D avatar, place it in any 3D environment, make it talk with gesture, control camera angle and.. edit on its own editing timeline step by step tutorial:

el.cine

53,833 görüntüleme • 11 ay önce