正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

(1/3) Can we turn text-to-image models into photorealistic 3D generators? ViewDiff (#CVPR2024) produces realistic, multi-view consistent images of real-world 3D objects in authentic surroundings. Website Video How does it work?

Matthias Niessner

47,801 subscribers

34,753 次观看 • 2 年前 •via X (Twitter)

科学技术教育 #CVPR2024

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

this is the future of CGI this new OpenMagic AI can add image, video or 3D product into real life video seamlessly with one click and.. it can even turn product image into 3D model automatically 10 wild examples:

this is the future of CGI this new OpenMagic AI can add image, video or 3D product into real life video seamlessly with one click and.. it can even turn product image into 3D model automatically 10 wild examples:

el.cine

120,965 次观看 • 1 年前

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

MrNeRF

52,801 次观看 • 1 年前

CSM 3D AI: New image-to-3D AI program AI is reshaping the future of 3D design. This amazing tool transforms still images into 3D models, and the best part? You can animate them effortlessly using AI. Link in comment!

CSM 3D AI: New image-to-3D AI program AI is reshaping the future of 3D design. This amazing tool transforms still images into 3D models, and the best part? You can animate them effortlessly using AI. Link in comment!

el.cine

54,393 次观看 • 1 年前

🤩Video world models are cool, but it is cooler if they can simulate any 3D physical actions in real time! Introducing RealWonder⚡️: Now you can simulate 3D physical action (robot actions, 3D forces, force fields, etc.) consequences from a single image in real time! 🧵1/6

🤩Video world models are cool, but it is cooler if they can simulate any 3D physical actions in real time! Introducing RealWonder⚡️: Now you can simulate 3D physical action (robot actions, 3D forces, force fields, etc.) consequences from a single image in real time! 🧵1/6

Hong-Xing (Koven) Yu

29,804 次观看 • 4 个月前

Introducing Objaverse, a massive open dataset of text-paired 3D objects! Nearly 1 million annotated 3D objects to pave the way to build incredible large-scale 3D generative models: 🧵👇 🤗 Hugging Face: 📝ArXiv: #CVPR2023

Introducing Objaverse, a massive open dataset of text-paired 3D objects! Nearly 1 million annotated 3D objects to pave the way to build incredible large-scale 3D generative models: 🧵👇 🤗 Hugging Face: 📝ArXiv: #CVPR2023

Matt Deitke

1,600,901 次观看 • 3 年前

.Tencent Hy HY-World 2.0 has landed,fully OPEN SOURCE ! We kept hearing the same thing: world models look great, but you can't use the output. So we made one you can, real 3D assets you can edit, rearrange, and ship into production. Prompt to engine-ready 3D. For real this time： ✨Directly outputs editable, production-ready 3D assets ✨Seamlessly plugs into existing game pipelines ✨Multimodal input: text, image, videos 🪄Character mode: freely explore streets, buildings, and open worlds with realistic physics, collisions, and no time limits 🔗GitHub:

.Tencent Hy HY-World 2.0 has landed,fully OPEN SOURCE ! We kept hearing the same thing: world models look great, but you can't use the output. So we made one you can, real 3D assets you can edit, rearrange, and ship into production. Prompt to engine-ready 3D. For real this time： ✨Directly outputs editable, production-ready 3D assets ✨Seamlessly plugs into existing game pipelines ✨Multimodal input: text, image, videos 🪄Character mode: freely explore streets, buildings, and open worlds with realistic physics, collisions, and no time limits 🔗GitHub:

Tencent AI

116,648 次观看 • 2 个月前

the fact that i can take an image of a room and turn it into a 3d model in one shot is actually insane this took like 30 seconds from image to 3d model

the fact that i can take an image of a room and turn it into a 3d model in one shot is actually insane this took like 30 seconds from image to 3d model

Jan

131,884 次观看 • 7 个月前

🚀 Tencent 𝗛𝗬-𝗪𝗼𝗿𝗹𝗱 𝟮.𝟬 just dropped Text or image → 3D world (3DGS + Mesh). Fully automatic. Video world models let you 𝘄𝗮𝘁𝗰𝗵. We let you 𝗲𝘅𝗽𝗹𝗼𝗿𝗲. #worldmodel #HYWorld

🚀 Tencent 𝗛𝗬-𝗪𝗼𝗿𝗹𝗱 𝟮.𝟬 just dropped Text or image → 3D world (3DGS + Mesh). Fully automatic. Video world models let you 𝘄𝗮𝘁𝗰𝗵. We let you 𝗲𝘅𝗽𝗹𝗼𝗿𝗲. #worldmodel #HYWorld

Tengfei Wang

17,396 次观看 • 2 个月前

We are happy to announce that ComfyUI now natively supports toad Hunyuan3D 2.0 and Hunyuan3D 2.0 MV (Multi-View) model series! 3 workflows to get started: 🔹Hunyuan3D-2 mv: multi-view to 3D 🔹Hunyuan3D-2 mv Turbo: accelerated multi-view 🔹Hunyuan3D-2: single image to 3D

We are happy to announce that ComfyUI now natively supports toad Hunyuan3D 2.0 and Hunyuan3D 2.0 MV (Multi-View) model series! 3 workflows to get started: 🔹Hunyuan3D-2 mv: multi-view to 3D 🔹Hunyuan3D-2 mv Turbo: accelerated multi-view 🔹Hunyuan3D-2: single image to 3D

ComfyUI

66,892 次观看 • 1 年前

We introduce TurboEdit -- simple text-based image editing in 1/2 sec! We leverage few-step diffusion models, mapping real images into noise with an encoder. Please see our #ECCV2024 paper: Work w/ Zongze Wu, Nick Kolkin, Jon Brandt, Eli Shechtman 1/

We introduce TurboEdit -- simple text-based image editing in 1/2 sec! We leverage few-step diffusion models, mapping real images into noise with an encoder. Please see our #ECCV2024 paper: Work w/ Zongze Wu, Nick Kolkin, Jon Brandt, Eli Shechtman 1/

Richard Zhang

39,306 次观看 • 1 年前

ViewCrafter can generate high-quality 3D views from single or few images using a video diffusion model! It allows for precise camera control and is useful for real-time rendering and turning text into 3D scenes. Links ⬇️

ViewCrafter can generate high-quality 3D views from single or few images using a video diffusion model! It allows for precise camera control and is useful for real-time rendering and turning text into 3D scenes. Links ⬇️

Dreaming Tulpa 🥓👑

19,892 次观看 • 1 年前

3d world abilities of AI models keep getting better, this is @Kling_AI 1.5 It can move around a space in 3d and you can steer its camera movements All based on a single input image

3d world abilities of AI models keep getting better, this is @Kling_AI 1.5 It can move around a space in 3d and you can steer its camera movements All based on a single input image

@levelsio

112,201 次观看 • 1 年前

Hunyuan 3D-2.1 turns any flat image into studio-quality 3D models. And you can do it on this Hugging Face space for free.

Hunyuan 3D-2.1 turns any flat image into studio-quality 3D models. And you can do it on this Hugging Face space for free.

Rohan Paul

222,392 次观看 • 10 个月前

✨ Made a new mini feature on Photo AI: [ Grab from 3d model ] So the problem is we're at that stage in time (typical for AI) where image-to-3d models are not good enough but are fun to play with, but we know they'll be good enough in 1-2 years With [ Make 3d model ] you already can turn any Photo AI pic into a 3d model but it still looks hyper clunky and deformed, but it works! One cool idea I had to make that more useful and made now: Let people make a 3d model then change the view of the it with the 3d viewer, then press [ o ] and it grabs a frame of the 3d That image you can then [ Remix ] (img2img), and it becomes a real photo again and that in turn you can then turn into a video again with [ Make video ] So that essentially gives you a fully freeform camera position control to take photos with One thing I need to fix is the background/skybox, I kinda need to take the original photo and remove the person and just get the background for the 3d model viewer, in this case it should be white, but it's a start!

✨ Made a new mini feature on Photo AI: [ Grab from 3d model ] So the problem is we're at that stage in time (typical for AI) where image-to-3d models are not good enough but are fun to play with, but we know they'll be good enough in 1-2 years With [ Make 3d model ] you already can turn any Photo AI pic into a 3d model but it still looks hyper clunky and deformed, but it works! One cool idea I had to make that more useful and made now: Let people make a 3d model then change the view of the it with the 3d viewer, then press [ o ] and it grabs a frame of the 3d That image you can then [ Remix ] (img2img), and it becomes a real photo again and that in turn you can then turn into a video again with [ Make video ] So that essentially gives you a fully freeform camera position control to take photos with One thing I need to fix is the background/skybox, I kinda need to take the original photo and remove the person and just get the background for the 3d model viewer, in this case it should be white, but it's a start!

@levelsio

119,210 次观看 • 1 年前

(1/2) Check out 𝐆𝐚𝐮𝐬𝐬𝐢𝐚𝐧𝐀𝐯𝐚𝐭𝐚𝐫𝐬: Photorealistic Head Avatars with Rigged 3D Gaussians! We create photorealistic head avatars by animating 3D Gaussians on a parametric face model - edited and rendered in *real-time*!

(1/2) Check out 𝐆𝐚𝐮𝐬𝐬𝐢𝐚𝐧𝐀𝐯𝐚𝐭𝐚𝐫𝐬: Photorealistic Head Avatars with Rigged 3D Gaussians! We create photorealistic head avatars by animating 3D Gaussians on a parametric face model - edited and rendered in real-time!

Matthias Niessner

126,983 次观看 • 2 年前

📢📢📢 Today I'll be giving a talk at the #CVPR2024 workshop on "Learning 3D with Multi-View Supervision" See you at 16:20 in Summit 331 Get a preview of the upcoming "3D Gaussian Splatting as Markov Chain Monte Carlo"

📢📢📢 Today I'll be giving a talk at the #CVPR2024 workshop on "Learning 3D with Multi-View Supervision" See you at 16:20 in Summit 331 Get a preview of the upcoming "3D Gaussian Splatting as Markov Chain Monte Carlo"

Andrea Tagliasacchi @CVPR

47,167 次观看 • 2 年前

Realistic mini 3D models of people

Realistic mini 3D models of people

Interesting As Fuck

239,269 次观看 • 1 年前

Introducing 📦𝗔𝗿𝘁𝗶𝗟𝗮𝘁𝗲𝗻𝘁🔧 (SIGGRAPH Asia 2025) — a high-quality 3D diffusion model that explicitly models object articulation, paving the way for richer, more realistic assets in embodied AI and simulation: – Generates fully articulated 3D objects – Physically plausible joints & motion – High-fidelity 3D Gaussian appearance – Supports generation from a single real image arXiv: Project: Code (coming soon):

Introducing 📦𝗔𝗿𝘁𝗶𝗟𝗮𝘁𝗲𝗻𝘁🔧 (SIGGRAPH Asia 2025) — a high-quality 3D diffusion model that explicitly models object articulation, paving the way for richer, more realistic assets in embodied AI and simulation: – Generates fully articulated 3D objects – Physically plausible joints & motion – High-fidelity 3D Gaussian appearance – Supports generation from a single real image arXiv: Project: Code (coming soon):

Xingang Pan

11,502 次观看 • 7 个月前