Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

⚡️ Introducing Bolt3D ⚡️ Bolt3D generates interactive 3D scenes in less than 7 seconds on a single GPU from one or more images. It features a latent diffusion model that directly generates 3D Gaussians of seen and unseen regions, without any test time optimization. 🧵👇 (1/9)

Stan Szymanowicz

1,168 subscribers

126,142 просмотров • 1 год назад •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

Комментарии: 44

Фото профиля Stan Szymanowicz

Stan Szymanowicz1 год назад

Project page: Arxiv: A huge shout out to my absolutely incredible co-authors @jasonyzhang2 @_pratul_ @RuiqiGao @ArthurBrussee @holynski_ @rmbrualla @jon_barron @philipphenzler (2/9)

Фото профиля Stan Szymanowicz

Stan Szymanowicz1 год назад

Generating 3D content is challenging because (1) It is unclear how to represent and structure (possibly unbounded) 3D scene data for diffusion models. (2) 3D data is much more limited than 2D image and video data. (3/9)

Фото профиля Stan Szymanowicz

Stan Szymanowicz1 год назад

Bolt3D puts 3D data in 2D grids (remember Splatter Images?). First, we sample multi-view images and pointmaps with a latent diffusion model. Then, the properties of 3D Gaussians are estimated with a Gaussian head. We reconstruct a large dataset of 3D scenes for training. (4/9)

Фото профиля Stan Szymanowicz

Stan Szymanowicz1 год назад

The key to generating high-quality geometry with a latent diffusion is to train a VAE for geometry data (pointmaps). We found that, for geometry data, our transformer decoder is more appropriate than a convolutional decoder, and that RGB image VAEs fail catastrophically. (5/9)

Фото профиля Stan Szymanowicz

Stan Szymanowicz1 год назад

Bolt3D can accept variable number of input images. Our model adheres to conditioning when it is available and generates unobserved scene regions without any re-projection or in-painting mechanisms. (6/9)

Фото профиля Stan Szymanowicz

Stan Szymanowicz1 год назад

Bolt3D enables generative capabilities in feed-forward 3D reconstructors. Compared to SOTA optimization-based pipelines (first generate Novel Views, then distill 3D), Bolt3D reduces inference cost by 64-300x. (7/9)

Фото профиля Stan Szymanowicz

Stan Szymanowicz1 год назад

Check out the generated 3D scenes in our interactive viewer, powered by Brush, as well as more video results on the project website! (8/9)

Фото профиля Stan Szymanowicz

Stan Szymanowicz1 год назад

Finally, a special thank you to @poolio for his helpful suggestions, guidance and contributions! (9/9)

Фото профиля clem 🤗

clem 🤗1 год назад

would be so cool to have a to try!

Фото профиля Matthias Niessner

Matthias Niessner1 год назад

Great work!

Фото профиля Stan Szymanowicz

Stan Szymanowicz1 год назад

Thank you!

Фото профиля Hang Gao

Hang Gao1 год назад

Nice work!

Фото профиля Stan Szymanowicz

Stan Szymanowicz1 год назад

Thank you!

Фото профиля Jensen Zhou

Jensen Zhou1 год назад

@hangg70 Hi y'all, let me introduce you two.

Фото профиля Hang Gao

Hang Gao1 год назад

@StanSzymanowicz Amesome!

Фото профиля Stan Szymanowicz

Stan Szymanowicz1 год назад

@jensenzhoujh The two camera-conditioned-NVS experts!

Фото профиля Qianyi Wu

Qianyi Wu1 год назад

Impressive results!

Фото профиля Abdullah Hamdi

Abdullah Hamdi1 год назад

Amazing !

Фото профиля Samarth Sinha

Samarth Sinha1 год назад

Congrats Stan!!

Фото профиля Stan Szymanowicz

Stan Szymanowicz1 год назад

Thanks Sam!

Фото профиля Arjun Karpur

Arjun Karpur1 год назад

Congrats on the release, Stan!

Фото профиля Stan Szymanowicz

Stan Szymanowicz1 год назад

Thanks Arjun!

Фото профиля Markus Malmros

Markus Malmros1 год назад

Wow, looks really cool!

Фото профиля tipatat

tipatat1 год назад

Really impressive work! Would it be possible to improve the quality with a large data set of high quality 3DGS made of real world capture?

Фото профиля Nataniel Ruiz

Nataniel Ruiz1 год назад

amazing

Фото профиля Hirokatsu Kataoka | 片岡裕雄

Hirokatsu Kataoka | 片岡裕雄1 год назад

Amazing tech, congratulations Stan! 🎉

Фото профиля Stan Szymanowicz

Stan Szymanowicz1 год назад

Thanks Hirokatsu!

Фото профиля Bilawal Sidhu

Bilawal Sidhu1 год назад

Very cool results, congrats!

Фото профиля Haofei Xu

Haofei Xu1 год назад

Fantastic results! Congrats!

Фото профиля Stan Szymanowicz

Stan Szymanowicz1 год назад

Thanks Haofei!

Фото профиля Rundi Wu

Rundi Wu1 год назад

Great work and awesome results! Congrats Stan!

Фото профиля Stan Szymanowicz

Stan Szymanowicz1 год назад

Thanks Rundi!

Фото профиля Nikita Karaev

Nikita Karaev1 год назад

Super cool! Congrats Stan!

Фото профиля Stan Szymanowicz

Stan Szymanowicz1 год назад

Thank you Nikita!

Фото профиля BLd

BLd1 год назад

Congrats, results are outstanding!

Фото профиля :3

:31 год назад

In 2020 I made a framework that can generate cities at ~10 million vertices in 16.667ms, meaning that you can generate a whole game map 60 times per second while playing the game, and everything is fully interactive and hierarchically laid out:

Фото профиля Kosta Derpanis at #ECCV2026 🇸🇪

Kosta Derpanis at #ECCV2026 🇸🇪1 год назад

💪

Фото профиля TOTORITO

TOTORITO1 год назад

Waaawwww !!!

Фото профиля The Qubits Guy

The Qubits Guy1 год назад

When will @Apple license your tech for my next iPhone?

Фото профиля MOTA

MOTA1 год назад

release the kraken

Фото профиля MangoMagic™️

MangoMagic™️1 год назад

Fascinating tech or just a new way to procrastinate? #3DJustGotReal #WhereIsMyGPU #ProcrastinationStation

Фото профиля iKrasinski

iKrasinski1 год назад

Amazing. Waiting for the HF space to feel with my hands.

Фото профиля Simsalabim

Simsalabim1 год назад

Congrats, looks great! Any Plans on releasing the code? Would love to try it out!

Фото профиля Diego Bonati

Diego Bonati1 год назад

looks very interesting, I would like to generate stereoscopic images with this and also VR equirectangular format.

Похожие видео

Current 3D generative models are slow and low quality. We present GRM, a large-scale model that reconstructs 3D Gaussians in 0.1s and generates high-quality 3D assets from text or single images in a few seconds. Demo: 1/4

Current 3D generative models are slow and low quality. We present GRM, a large-scale model that reconstructs 3D Gaussians in 0.1s and generates high-quality 3D assets from text or single images in a few seconds. Demo: 1/4

Gordon Wetzstein

19,241 просмотров • 2 лет назад

🌟 Create anything in 3D! 🌟 Introducing CAT3D: a new method that generates high-fidelity 3D scenes from any number of real or generated images in one minute, powered by multi-view diffusion models. w/ lovely coauthors Aleksander Holynski, Ben Poole and an amazing team!

🌟 Create anything in 3D! 🌟 Introducing CAT3D: a new method that generates high-fidelity 3D scenes from any number of real or generated images in one minute, powered by multi-view diffusion models. w/ lovely coauthors Aleksander Holynski, Ben Poole and an amazing team!

Ruiqi Gao

153,337 просмотров • 2 лет назад

⚡️Generating 3DGS scenes in 5 seconds on a single GPU⚡️ #FlashWorld enables ⚡️*fast*⚡️ (10~100x faster than previous methods) and 🔥*high-quality*🔥 3D world generation, from a single image or text prompt. Code: Page:

⚡️Generating 3DGS scenes in 5 seconds on a single GPU⚡️ #FlashWorld enables ⚡️fast⚡️ (10~100x faster than previous methods) and 🔥high-quality🔥 3D world generation, from a single image or text prompt. Code: Page:

Tengfei Wang

119,877 просмотров • 1 год назад

Introducing RTFM (Real-Time Frame Model): a highly efficient World Model that generates video frames in real time as you interact with it, powered by a single H100 GPU. RTFM renders persistent and 3D consistent worlds, both real and imaginary. Try our demo of RTFM today!

Introducing RTFM (Real-Time Frame Model): a highly efficient World Model that generates video frames in real time as you interact with it, powered by a single H100 GPU. RTFM renders persistent and 3D consistent worlds, both real and imaginary. Try our demo of RTFM today!

World Labs

342,124 просмотров • 1 год назад

One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization paper page: Single image 3D reconstruction is an important but challenging task that requires extensive knowledge of our natural world. Many existing methods solve this problem by optimizing a neural radiance field under the guidance of 2D diffusion models but suffer from lengthy optimization time, 3D inconsistency results, and poor geometry. In this work, we propose a novel method that takes a single image of any object as input and generates a full 360-degree 3D textured mesh in a single feed-forward pass. Given a single image, we first use a view-conditioned 2D diffusion model, Zero123, to generate multi-view images for the input view, and then aim to lift them up to 3D space. Since traditional reconstruction methods struggle with inconsistent multi-view predictions, we build our 3D reconstruction module upon an SDF-based generalizable neural surface reconstruction method and propose several critical training strategies to enable the reconstruction of 360-degree meshes. Without costly optimizations, our method reconstructs 3D shapes in significantly less time than existing methods. Moreover, our method favors better geometry, generates more 3D consistent results, and adheres more closely to the input image. We evaluate our approach on both synthetic data and in-the-wild images and demonstrate its superiority in terms of both mesh quality and runtime. In addition, our approach can seamlessly support the text-to-3D task by integrating with off-the-shelf text-to-image diffusion models.

One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization paper page: Single image 3D reconstruction is an important but challenging task that requires extensive knowledge of our natural world. Many existing methods solve this problem by optimizing a neural radiance field under the guidance of 2D diffusion models but suffer from lengthy optimization time, 3D inconsistency results, and poor geometry. In this work, we propose a novel method that takes a single image of any object as input and generates a full 360-degree 3D textured mesh in a single feed-forward pass. Given a single image, we first use a view-conditioned 2D diffusion model, Zero123, to generate multi-view images for the input view, and then aim to lift them up to 3D space. Since traditional reconstruction methods struggle with inconsistent multi-view predictions, we build our 3D reconstruction module upon an SDF-based generalizable neural surface reconstruction method and propose several critical training strategies to enable the reconstruction of 360-degree meshes. Without costly optimizations, our method reconstructs 3D shapes in significantly less time than existing methods. Moreover, our method favors better geometry, generates more 3D consistent results, and adheres more closely to the input image. We evaluate our approach on both synthetic data and in-the-wild images and demonstrate its superiority in terms of both mesh quality and runtime. In addition, our approach can seamlessly support the text-to-3D task by integrating with off-the-shelf text-to-image diffusion models.

AK

179,061 просмотров • 3 лет назад

We’ve upgraded Stable Video Diffusion 4D to Stable Video 4D 2.0 (SV4D 2.0), improving the quality of 4D outputs generated from a single object-centric video. While 3D provides a static view of an object’s shape and size; 4D extends this by including time, showing how the object moves. This multi-view video diffusion model generates a 4D output in three steps: 1️⃣ Starts with an input video of a moving person or object 2️⃣ Generates novel views of the subject from unseen angles 3️⃣ Constructs a single dynamic 4D video output with spatial and temporal consistency You can learn more here: (1/4)

We’ve upgraded Stable Video Diffusion 4D to Stable Video 4D 2.0 (SV4D 2.0), improving the quality of 4D outputs generated from a single object-centric video. While 3D provides a static view of an object’s shape and size; 4D extends this by including time, showing how the object moves. This multi-view video diffusion model generates a 4D output in three steps: 1️⃣ Starts with an input video of a moving person or object 2️⃣ Generates novel views of the subject from unseen angles 3️⃣ Constructs a single dynamic 4D video output with spatial and temporal consistency You can learn more here: (1/4)

Stability AI

36,110 просмотров • 1 год назад

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

Sherwin Bahmani

66,918 просмотров • 1 год назад

Today, we are open-sourcing Hunyuan World 1.1 (WorldMirror), a universal feed-forward 3D reconstruction model. 🚀🚀🚀 While our previously released Hunyuan World 1.0 (open-sourced, lite version deployable on consumer GPUs) focused on generating 3D worlds from text or single-view images, Hunyuan World 1.1 significantly expands the input scope by unlocking video-to-3D and multi-view-to-3D world creation. Highlights: 🔹Any Input, Maximized Flexibility and Fidelity: Flexibly integrates diverse geometric priors (camera poses, intrinsics, depth maps) to resolve structural ambiguities and ensure geometrically consistent 3D outputs. 🔹Any Output, SOTA Results：This elegant architecture simultaneously generates multiple 3D representations: dense point clouds, multi-view depth maps, camera parameters, surface normals, and 3D Gaussian Splattings. 🔹Single-GPU & Fast Inference: As an all-in-one, feed-forward model, Hunyuan World 1.1 runs on a single GPU and delivers all 3D attributes in a single forward pass, within seconds. 🌐Project Page: 🔗Github： 🤗Hugging Face： ✨Demo: 📄Technical Report:

Today, we are open-sourcing Hunyuan World 1.1 (WorldMirror), a universal feed-forward 3D reconstruction model. 🚀🚀🚀 While our previously released Hunyuan World 1.0 (open-sourced, lite version deployable on consumer GPUs) focused on generating 3D worlds from text or single-view images, Hunyuan World 1.1 significantly expands the input scope by unlocking video-to-3D and multi-view-to-3D world creation. Highlights: 🔹Any Input, Maximized Flexibility and Fidelity: Flexibly integrates diverse geometric priors (camera poses, intrinsics, depth maps) to resolve structural ambiguities and ensure geometrically consistent 3D outputs. 🔹Any Output, SOTA Results：This elegant architecture simultaneously generates multiple 3D representations: dense point clouds, multi-view depth maps, camera parameters, surface normals, and 3D Gaussian Splattings. 🔹Single-GPU & Fast Inference: As an all-in-one, feed-forward model, Hunyuan World 1.1 runs on a single GPU and delivers all 3D attributes in a single forward pass, within seconds. 🌐Project Page: 🔗Github： 🤗Hugging Face： ✨Demo: 📄Technical Report:

Tencent Hy

168,984 просмотров • 1 год назад

We made an interactive client-server viewer for LagerNVS with Jonathon Luiten! You can now interactively explore scenes from just a photo capture - no optimization, no 3D Gaussians, just load your images, run the model on a cloud GPU and stream the renders to your local browser. Check out the video below for some spaces I recently captured in Oxford, London and beyond!

We made an interactive client-server viewer for LagerNVS with Jonathon Luiten! You can now interactively explore scenes from just a photo capture - no optimization, no 3D Gaussians, just load your images, run the model on a cloud GPU and stream the renders to your local browser. Check out the video below for some spaces I recently captured in Oxford, London and beyond!

Stan Szymanowicz

17,336 просмотров • 5 месяцев назад

We’re open-sourcing HY-World 2.0, a multimodal world model that generates, reconstructs, and simulates interactive *3D worlds* from text, images, and videos. Outputs can be integrated into game engines and embodied simulation pipelines. Key highlights: 🔹 One-click world generation Turn text or image into interactive 3D worlds automatically. 🔹 Pipeline-ready 3D outputs Editable 3D worlds for Unity and Unreal Engine, with standard 3D exports including mesh, 3DGS, and point clouds. 🔹 Unified world model system One model family for world generation and reconstruction across synthetic and real-world scenes. 🔹 Interactive character mode Explore generated 3D worlds in real time with physics-aware movement and collision support. ✨ Apply for access: 🔗 GitHub: 🤗 Hugging Face: 📄 Technical Report:

We’re open-sourcing HY-World 2.0, a multimodal world model that generates, reconstructs, and simulates interactive 3D worlds from text, images, and videos. Outputs can be integrated into game engines and embodied simulation pipelines. Key highlights: 🔹 One-click world generation Turn text or image into interactive 3D worlds automatically. 🔹 Pipeline-ready 3D outputs Editable 3D worlds for Unity and Unreal Engine, with standard 3D exports including mesh, 3DGS, and point clouds. 🔹 Unified world model system One model family for world generation and reconstruction across synthetic and real-world scenes. 🔹 Interactive character mode Explore generated 3D worlds in real time with physics-aware movement and collision support. ✨ Apply for access: 🔗 GitHub: 🤗 Hugging Face: 📄 Technical Report:

Tencent Hy

373,065 просмотров • 6 месяцев назад

#ICCV2025 🤩3D world generation is cool, but it is cooler to play with the worlds using 3D actions 👆💨, and see what happens! — Introducing *WonderPlay*: Now you can create dynamic 3D scenes that respond to your 3D actions from a single image! Web: 🧵1/7

#ICCV2025 🤩3D world generation is cool, but it is cooler to play with the worlds using 3D actions 👆💨, and see what happens! — Introducing WonderPlay: Now you can create dynamic 3D scenes that respond to your 3D actions from a single image! Web: 🧵1/7

Hong-Xing (Koven) Yu

58,065 просмотров • 1 год назад

China open-sourced a model that reconstructs any scene in 3D from a regular video, in real-time. one camera. no LiDAR. 10,000+ frames without falling apart. just walk around with your camera and watch the entire world get rebuilt in 3D at 20 fps. → runs at ~20 FPS on a single GPU → Stable over 10,000+ frames → Beats optimization-based methods on benchmarks → Works on drone footage, driving videos, indoor walkthroughs 100% open source.

China open-sourced a model that reconstructs any scene in 3D from a regular video, in real-time. one camera. no LiDAR. 10,000+ frames without falling apart. just walk around with your camera and watch the entire world get rebuilt in 3D at 20 fps. → runs at ~20 FPS on a single GPU → Stable over 10,000+ frames → Beats optimization-based methods on benchmarks → Works on drone footage, driving videos, indoor walkthroughs 100% open source.

Yasir Ai

254,982 просмотров • 3 месяцев назад

China open-sourced a model that reconstructs any scene in 3D from a regular video, in real-time. one camera. no LiDAR. 10,000+ frames without falling apart. just walk around with your camera and watch the entire world get rebuilt in 3D at 20 fps. → runs at ~20 FPS on a single GPU → Stable over 10,000+ frames → Beats optimization-based methods on benchmarks → Works on drone footage, driving videos, indoor walkthroughs 100% open source.

China open-sourced a model that reconstructs any scene in 3D from a regular video, in real-time. one camera. no LiDAR. 10,000+ frames without falling apart. just walk around with your camera and watch the entire world get rebuilt in 3D at 20 fps. → runs at ~20 FPS on a single GPU → Stable over 10,000+ frames → Beats optimization-based methods on benchmarks → Works on drone footage, driving videos, indoor walkthroughs 100% open source.

Superman

1,466,900 просмотров • 3 месяцев назад

2D is what the camera sees.👀 3D is what the world is. 🌍 Today, we’re proudly open-sourcing QuerySplat — an open-source feed-forward 3D model that turns a few unposed images into a navigable 3D scene in seconds (run on 4090🪶). Your input: a few unposed images Our output: a sharp, navigable 3D scene — generated in seconds! 🚀 🧠 organize scenes with 3D queries, not pixel-bound Gaussians 📷 no camera poses or manual calibration required ⚡ reconstruct new scenes in a single forward pass ✨ preserve sharp geometry and high-frequency appearance 🏆 achieve state-of-the-art results on DL3DV-Evaluation 🔓 code and model weights are now open-source From pixels to scenes, we are making 3D generation scalable. Try our free iOS app — it’s a lot of fun! 😆 🔗 Try it on the Web: 🔗 Paper: 🔗 Code & weights: 🔗 Project page: 🔗 #AIGC #3DGS #AI #Computervision #3D #artist #3dartist #3d

2D is what the camera sees.👀 3D is what the world is. 🌍 Today, we’re proudly open-sourcing QuerySplat — an open-source feed-forward 3D model that turns a few unposed images into a navigable 3D scene in seconds (run on 4090🪶). Your input: a few unposed images Our output: a sharp, navigable 3D scene — generated in seconds! 🚀 🧠 organize scenes with 3D queries, not pixel-bound Gaussians 📷 no camera poses or manual calibration required ⚡ reconstruct new scenes in a single forward pass ✨ preserve sharp geometry and high-frequency appearance 🏆 achieve state-of-the-art results on DL3DV-Evaluation 🔓 code and model weights are now open-source From pixels to scenes, we are making 3D generation scalable. Try our free iOS app — it’s a lot of fun! 😆 🔗 Try it on the Web: 🔗 Paper: 🔗 Code & weights: 🔗 Project page: 🔗 #AIGC #3DGS #AI #Computervision #3D #artist #3dartist #3d

InSpatio｜影溯

30,303 просмотров • 2 месяцев назад

📢GaussianGPT: autoregressive 3D Gaussian scene generation. We introduce a GPT-style model that directly generates 3D Gaussian scenes, token by token, in a series of small, discrete decision steps. Generation, completion, and large-scale outpainting in a single pipeline. Unlike diffusion-based approaches, GaussianGPT explicitly models the scene distribution at every step, allowing for quite flexible scene synthesis. 🌐 ▶️ Great work by Nicolas von Lützow, Barbara Roessle, Katharina Schmid

📢GaussianGPT: autoregressive 3D Gaussian scene generation. We introduce a GPT-style model that directly generates 3D Gaussian scenes, token by token, in a series of small, discrete decision steps. Generation, completion, and large-scale outpainting in a single pipeline. Unlike diffusion-based approaches, GaussianGPT explicitly models the scene distribution at every step, allowing for quite flexible scene synthesis. 🌐 ▶️ Great work by Nicolas von Lützow, Barbara Roessle, Katharina Schmid

Matthias Niessner

152,704 просмотров • 6 месяцев назад

🚨 A single image → a fully editable UE5 world. Introducing Lumera: a turnkey pipeline that reconstructs an engine-native 3D scene directly from ONE image — not just pixels, point clouds, or Gaussians. It gives you: → object-level 3D meshes → editable 3D layouts → engine-native lights → HDR environment lighting → a scene ready for Unreal Engine 5 In other words: Image → 3D world → UE5. We believe the future of image-to-3D is not “generate something that looks 3D.” It is generate the actual world project. 🌐 📄

🚨 A single image → a fully editable UE5 world. Introducing Lumera: a turnkey pipeline that reconstructs an engine-native 3D scene directly from ONE image — not just pixels, point clouds, or Gaussians. It gives you: → object-level 3D meshes → editable 3D layouts → engine-native lights → HDR environment lighting → a scene ready for Unreal Engine 5 In other words: Image → 3D world → UE5. We believe the future of image-to-3D is not “generate something that looks 3D.” It is generate the actual world project. 🌐 📄

Hao Zhao

52,340 просмотров • 1 месяц назад

DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior paper page: present DreamCraft3D, a hierarchical 3D content generation method that produces high-fidelity and coherent 3D objects. We tackle the problem by leveraging a 2D reference image to guide the stages of geometry sculpting and texture boosting. A central focus of this work is to address the consistency issue that existing works encounter. To sculpt geometries that render coherently, we perform score distillation sampling via a view-dependent diffusion model. This 3D prior, alongside several training strategies, prioritizes the geometry consistency but compromises the texture fidelity. We further propose Bootstrapped Score Distillation to specifically boost the texture. We train a personalized diffusion model, Dreambooth, on the augmented renderings of the scene, imbuing it with 3D knowledge of the scene being optimized. The score distillation from this 3D-aware diffusion prior provides view-consistent guidance for the scene. Notably, through an alternating optimization of the diffusion prior and 3D scene representation, we achieve mutually reinforcing improvements: the optimized 3D scene aids in training the scene-specific diffusion model, which offers increasingly view-consistent guidance for 3D optimization. The optimization is thus bootstrapped and leads to substantial texture boosting. With tailored 3D priors throughout the hierarchical generation, DreamCraft3D generates coherent 3D objects with photorealistic renderings, advancing the state-of-the-art in 3D content generation.

DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior paper page: present DreamCraft3D, a hierarchical 3D content generation method that produces high-fidelity and coherent 3D objects. We tackle the problem by leveraging a 2D reference image to guide the stages of geometry sculpting and texture boosting. A central focus of this work is to address the consistency issue that existing works encounter. To sculpt geometries that render coherently, we perform score distillation sampling via a view-dependent diffusion model. This 3D prior, alongside several training strategies, prioritizes the geometry consistency but compromises the texture fidelity. We further propose Bootstrapped Score Distillation to specifically boost the texture. We train a personalized diffusion model, Dreambooth, on the augmented renderings of the scene, imbuing it with 3D knowledge of the scene being optimized. The score distillation from this 3D-aware diffusion prior provides view-consistent guidance for the scene. Notably, through an alternating optimization of the diffusion prior and 3D scene representation, we achieve mutually reinforcing improvements: the optimized 3D scene aids in training the scene-specific diffusion model, which offers increasingly view-consistent guidance for 3D optimization. The optimization is thus bootstrapped and leads to substantial texture boosting. With tailored 3D priors throughout the hierarchical generation, DreamCraft3D generates coherent 3D objects with photorealistic renderings, advancing the state-of-the-art in 3D content generation.

AK

161,530 просмотров • 3 лет назад

converts single 2D images into interactable 3D models; with MonoArt. Interior designers will definitely appreciate this one. - Snap one photo of any furniture - Get a working 3D model in 20 seconds - jointly predicts geometry, part segments, and motion; - built on TRELLIS. Drawers actually slide, doors open.. Test room clearances in VR before buying a thing

converts single 2D images into interactable 3D models; with MonoArt. Interior designers will definitely appreciate this one. - Snap one photo of any furniture - Get a working 3D model in 20 seconds - jointly predicts geometry, part segments, and motion; - built on TRELLIS. Drawers actually slide, doors open.. Test room clearances in VR before buying a thing

Wildminder

25,185 просмотров • 7 месяцев назад