正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

SCoPE: We found a surprisingly simple way to encode the 3D camera directly into video diffusion — and it works remarkably well.

Ying Shan

2,588 subscribers

13,402 次观看 • 2 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

1 条评论

David Finsterwalder | eu/acc 的头像

David Finsterwalder | eu/acc2 个月前

Amazing work. Love it!

相关视频

Can we apply diffusion directly on MLP weights? Yes!!! We show generalization to new 3D shapes and 4D mesh animations: "HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusion" #ICCV2023! Project: Video:

Can we apply diffusion directly on MLP weights? Yes!!! We show generalization to new 3D shapes and 4D mesh animations: "HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusion" #ICCV2023! Project: Video:

Matthias Niessner

86,084 次观看 • 3 年前

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

Sherwin Bahmani

66,918 次观看 • 1 年前

The Pika ChatGPT Plugin makes AI video creation so simple. I used it to create this wrestling video directly in ChatGPT. Here’s the result and how it works 👇

The Pika ChatGPT Plugin makes AI video creation so simple. I used it to create this wrestling video directly in ChatGPT. Here’s the result and how it works 👇

Ai Arainz

90,501 次观看 • 7 天前

Introducing MirageLSD: The First Live-Stream Diffusion (LSD) AI Model Input any video stream, from a camera or video chat to a computer screen or game, and transform it into any world you desire, in real-time (<40ms latency). Here’s how it works (w/ demo you can use!):

Introducing MirageLSD: The First Live-Stream Diffusion (LSD) AI Model Input any video stream, from a camera or video chat to a computer screen or game, and transform it into any world you desire, in real-time (<40ms latency). Here’s how it works (w/ demo you can use!):

Decart

1,017,450 次观看 • 1 年前

Cosmos Policy turns a pretrained video diffusion model into a robot controller. Instead of redesigning the architecture, it injects robot state, actions, and values directly as latent frames inside the video model

Cosmos Policy turns a pretrained video diffusion model into a robot controller. Instead of redesigning the architecture, it injects robot state, actions, and values directly as latent frames inside the video model

Robots Digest 🤖

22,933 次观看 • 9 个月前

V3D Video Diffusion Models are Effective 3D Generators Automatic 3D generation has recently attracted widespread attention. Recent methods have greatly accelerated the generation speed, but usually produce less-detailed objects due to limited model capacity or 3D data. Motivated by recent advancements in video diffusion models, we introduce V3D, which leverages the world simulation capacity of pre-trained video diffusion models to facilitate 3D generation. To fully unleash the potential of video diffusion to perceive the 3D world, we further introduce geometrical consistency prior and extend the video diffusion model to a multi-view consistent 3D generator. Benefiting from this, the state-of-the-art video diffusion model could be fine-tuned to generate 360degree orbit frames surrounding an object given a single image. With our tailored reconstruction pipelines, we can generate high-quality meshes or 3D Gaussians within 3 minutes. Furthermore, our method can be extended to scene-level novel view synthesis, achieving precise control over the camera path with sparse input views. Extensive experiments demonstrate the superior performance of the proposed approach, especially in terms of generation quality and multi-view consistency

V3D Video Diffusion Models are Effective 3D Generators Automatic 3D generation has recently attracted widespread attention. Recent methods have greatly accelerated the generation speed, but usually produce less-detailed objects due to limited model capacity or 3D data. Motivated by recent advancements in video diffusion models, we introduce V3D, which leverages the world simulation capacity of pre-trained video diffusion models to facilitate 3D generation. To fully unleash the potential of video diffusion to perceive the 3D world, we further introduce geometrical consistency prior and extend the video diffusion model to a multi-view consistent 3D generator. Benefiting from this, the state-of-the-art video diffusion model could be fine-tuned to generate 360degree orbit frames surrounding an object given a single image. With our tailored reconstruction pipelines, we can generate high-quality meshes or 3D Gaussians within 3 minutes. Furthermore, our method can be extended to scene-level novel view synthesis, achieving precise control over the camera path with sparse input views. Extensive experiments demonstrate the superior performance of the proposed approach, especially in terms of generation quality and multi-view consistency

AK

31,997 次观看 • 2 年前

We took Hi3D's 3D maker platform for a spin to see if it could turn a simple text prompt into a fully assembled, print-ready Space Marine miniature. Here's how the whole process works: #sponsored #ad

We took Hi3D's 3D maker platform for a spin to see if it could turn a simple text prompt into a fully assembled, print-ready Space Marine miniature. Here's how the whole process works: #sponsored #ad

80 LEVEL

11,492 次观看 • 3 个月前

DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior paper page: present DreamCraft3D, a hierarchical 3D content generation method that produces high-fidelity and coherent 3D objects. We tackle the problem by leveraging a 2D reference image to guide the stages of geometry sculpting and texture boosting. A central focus of this work is to address the consistency issue that existing works encounter. To sculpt geometries that render coherently, we perform score distillation sampling via a view-dependent diffusion model. This 3D prior, alongside several training strategies, prioritizes the geometry consistency but compromises the texture fidelity. We further propose Bootstrapped Score Distillation to specifically boost the texture. We train a personalized diffusion model, Dreambooth, on the augmented renderings of the scene, imbuing it with 3D knowledge of the scene being optimized. The score distillation from this 3D-aware diffusion prior provides view-consistent guidance for the scene. Notably, through an alternating optimization of the diffusion prior and 3D scene representation, we achieve mutually reinforcing improvements: the optimized 3D scene aids in training the scene-specific diffusion model, which offers increasingly view-consistent guidance for 3D optimization. The optimization is thus bootstrapped and leads to substantial texture boosting. With tailored 3D priors throughout the hierarchical generation, DreamCraft3D generates coherent 3D objects with photorealistic renderings, advancing the state-of-the-art in 3D content generation.

DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior paper page: present DreamCraft3D, a hierarchical 3D content generation method that produces high-fidelity and coherent 3D objects. We tackle the problem by leveraging a 2D reference image to guide the stages of geometry sculpting and texture boosting. A central focus of this work is to address the consistency issue that existing works encounter. To sculpt geometries that render coherently, we perform score distillation sampling via a view-dependent diffusion model. This 3D prior, alongside several training strategies, prioritizes the geometry consistency but compromises the texture fidelity. We further propose Bootstrapped Score Distillation to specifically boost the texture. We train a personalized diffusion model, Dreambooth, on the augmented renderings of the scene, imbuing it with 3D knowledge of the scene being optimized. The score distillation from this 3D-aware diffusion prior provides view-consistent guidance for the scene. Notably, through an alternating optimization of the diffusion prior and 3D scene representation, we achieve mutually reinforcing improvements: the optimized 3D scene aids in training the scene-specific diffusion model, which offers increasingly view-consistent guidance for 3D optimization. The optimization is thus bootstrapped and leads to substantial texture boosting. With tailored 3D priors throughout the hierarchical generation, DreamCraft3D generates coherent 3D objects with photorealistic renderings, advancing the state-of-the-art in 3D content generation.

AK

161,530 次观看 • 3 年前

Synthesizing worlds with video diffusion models is often inconsistent — moving the camera back and forth leads to different scenes. We propose 🌐𝗪𝗼𝗿𝗹𝗱𝗠𝗲𝗺, a memory-based approach that ensures consistent world simulation without relying on explicit 3D reconstruction.

Synthesizing worlds with video diffusion models is often inconsistent — moving the camera back and forth leads to different scenes. We propose 🌐𝗪𝗼𝗿𝗹𝗱𝗠𝗲𝗺, a memory-based approach that ensures consistent world simulation without relying on explicit 3D reconstruction.

Xingang Pan

19,413 次观看 • 1 年前

Introducing powered by Over the Reality. FreeGaussian is a new way to transform your existing photos, videos, and 360 captures for free into high-quality 3D Gaussian splats, directly in your browser. Whether you already have: ↳ a 360 recording captured with any 360 camera ↳ a set of photos ↳ or even a simple video from a place you visited you can now turn it into a photorealistic 3D Gaussian splat. Just go to upload your content, and let the reconstruction happen. Once it’s ready, you can download your 3D Gaussian splat and point cloud, and use them across your own projects, pipelines, or applications.

Introducing powered by Over the Reality. FreeGaussian is a new way to transform your existing photos, videos, and 360 captures for free into high-quality 3D Gaussian splats, directly in your browser. Whether you already have: ↳ a 360 recording captured with any 360 camera ↳ a set of photos ↳ or even a simple video from a place you visited you can now turn it into a photorealistic 3D Gaussian splat. Just go to upload your content, and let the reconstruction happen. Once it’s ready, you can download your 3D Gaussian splat and point cloud, and use them across your own projects, pipelines, or applications.

Over the Reality 🌐

86,340 次观看 • 6 个月前

From a rough 3D blockout to the final cinematic shot. Seedance 2.5 turns simple shapes and motion into a whole new world while keeping the original camera direction

From a rough 3D blockout to the final cinematic shot. Seedance 2.5 turns simple shapes and motion into a whole new world while keeping the original camera direction

Patrick

67,466 次观看 • 2 个月前

📢📢📢 𝐀𝐂𝟑𝐃: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers TL;DR: for 3D camera control in generative video, it really helps knowing *which* part of your model you should mess with Internship by Sherwin Bahmani at Snap

📢📢📢 𝐀𝐂𝟑𝐃: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers TL;DR: for 3D camera control in generative video, it really helps knowing which part of your model you should mess with Internship by Sherwin Bahmani at Snap

Andrea Tagliasacchi

23,088 次观看 • 1 年前

I love this one scene in A Hard Day’s Night- for scarcely a minute, the dialogue stylization goes way up, it feels like a New Wave movie for a moment and the exchange does feel remarkably meaningful for a quick dialogue amongst passing strangers. John acts it surprisingly well.

I love this one scene in A Hard Day’s Night- for scarcely a minute, the dialogue stylization goes way up, it feels like a New Wave movie for a moment and the exchange does feel remarkably meaningful for a quick dialogue amongst passing strangers. John acts it surprisingly well.

Halestrom

80,517 次观看 • 8 个月前

AI just fired the video editor 3D Motion just dropped the fastest way to turn any image into a cinematic video. With depth mapping, smooth camera moves, and movie-like lighting, it makes static photos look like Hollywood shots. Tutorial below + Prompts:

AI just fired the video editor 3D Motion just dropped the fastest way to turn any image into a cinematic video. With depth mapping, smooth camera moves, and movie-like lighting, it makes static photos look like Hollywood shots. Tutorial below + Prompts:

Robin Delta

137,796 次观看 • 1 年前

DDG found a creative way to communicate with Halo and it actually works 😭😭

DDG found a creative way to communicate with Halo and it actually works 😭😭

ryan 🤿

795,497 次观看 • 1 年前

Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models paper page: Traditional 3D content creation tools empower users to bring their imagination to life by giving them direct control over a scene's geometry, appearance, motion, and camera path. Creating computer-generated videos, however, is a tedious manual process, which can be automated by emerging text-to-video diffusion models. Despite great promise, video diffusion models are difficult to control, hindering a user to apply their own creativity rather than amplifying it. To address this challenge, we present a novel approach that combines the controllability of dynamic 3D meshes with the expressivity and editability of emerging diffusion models. For this purpose, our approach takes an animated, low-fidelity rendered mesh as input and injects the ground truth correspondence information obtained from the dynamic mesh into various stages of a pre-trained text-to-image generation model to output high-quality and temporally consistent frames. We demonstrate our approach on various examples where motion can be obtained by animating rigged assets or changing the camera path.

Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models paper page: Traditional 3D content creation tools empower users to bring their imagination to life by giving them direct control over a scene's geometry, appearance, motion, and camera path. Creating computer-generated videos, however, is a tedious manual process, which can be automated by emerging text-to-video diffusion models. Despite great promise, video diffusion models are difficult to control, hindering a user to apply their own creativity rather than amplifying it. To address this challenge, we present a novel approach that combines the controllability of dynamic 3D meshes with the expressivity and editability of emerging diffusion models. For this purpose, our approach takes an animated, low-fidelity rendered mesh as input and injects the ground truth correspondence information obtained from the dynamic mesh into various stages of a pre-trained text-to-image generation model to output high-quality and temporally consistent frames. We demonstrate our approach on various examples where motion can be obtained by animating rigged assets or changing the camera path.

AK

117,091 次观看 • 2 年前

We built a 3D node just for Astra. Generate 3D faster in FLORA, iterate between 2D & 3D directly on the canvas, and export directly to Blender.

We built a 3D node just for Astra. Generate 3D faster in FLORA, iterate between 2D & 3D directly on the canvas, and export directly to Blender.

weber

13,074 次观看 • 1 个月前