Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

DreamScene360 Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splatting The increasing demand for virtual reality applications has highlighted the significance of crafting immersive 3D assets. We present a text-to-3D 360^{circ} scene generation pipeline

AK

515,384 subscribers

24,910 Aufrufe • vor 2 Jahren •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von AK

AKvor 2 Jahren

that facilitates the creation of comprehensive 360^{circ} scenes for in-the-wild environments in a matter of minutes. Our approach utilizes the generative power of a 2D diffusion model and prompt self-refinement to create a high-quality and globally coherent panoramic image.

Profilbild von AK

AKvor 2 Jahren

This image acts as a preliminary "flat" (2D) scene representation. Subsequently, it is lifted into 3D Gaussians, employing splatting techniques to enable real-time exploration. To produce consistent 3D geometry, our pipeline constructs a spatially coherent structure by

Profilbild von AK

AKvor 2 Jahren

aligning the 2D monocular depth into a globally optimized point cloud. This point cloud serves as the initial state for the centroids of 3D Gaussians. In order to address invisible issues inherent in single-view inputs, we impose semantic and geometric constraints on

Profilbild von AK

AKvor 2 Jahren

both synthesized and input camera views as regularizations. These guide the optimization of Gaussians, aiding in the reconstruction of unseen regions. In summary, our method offers a

Profilbild von AK

AKvor 2 Jahren

globally consistent 3D scene within a 360^{circ} perspective, providing an enhanced immersive experience over existing techniques.

Profilbild von AK

AKvor 2 Jahren

paper page:

Profilbild von AK

AKvor 2 Jahren

daily papers:

Profilbild von Neural Hours

Neural Hoursvor 2 Jahren

🤯 Absolutely insane!

Profilbild von Beardyman

Beardymanvor 2 Jahren

@ShijieZhoucla Hi! I’d love to try Dream Scene!!

Profilbild von Wang Yun

Wang Yunvor 2 Jahren

Similar to blockade Labs

Ähnliche Videos

LayerPano3D Layered 3D Panorama for Hyper-Immersive Scene Generation discuss: 3D immersive scene generation is a challenging yet critical task in computer vision and graphics. A desired virtual 3D scene should 1) exhibit omnidirectional view consistency, and 2) allow for free exploration in complex scene hierarchies. Existing methods either rely on successive scene expansion via inpainting or employ panorama representation to represent large FOV scene environments. However, the generated scene suffers from semantic drift during expansion and is unable to handle occlusion among scene hierarchies. To tackle these challenges, we introduce LayerPano3D, a novel framework for full-view, explorable panoramic 3D scene generation from a single text prompt. Our key insight is to decompose a reference 2D panorama into multiple layers at different depth levels, where each layer reveals the unseen space from the reference views via diffusion prior. LayerPano3D comprises multiple dedicated designs: 1) we introduce a novel text-guided anchor view synthesis pipeline for high-quality, consistent panorama generation. 2) We pioneer the Layered 3D Panorama as underlying representation to manage complex scene hierarchies and lift it into 3D Gaussians to splat detailed 360-degree omnidirectional scenes with unconstrained viewing paths. Extensive experiments demonstrate that our framework generates state-of-the-art 3D panoramic scene in both full view consistency and immersive exploratory experience. We believe that LayerPano3D holds promise for advancing 3D panoramic scene creation with numerous applications.

LayerPano3D Layered 3D Panorama for Hyper-Immersive Scene Generation discuss: 3D immersive scene generation is a challenging yet critical task in computer vision and graphics. A desired virtual 3D scene should 1) exhibit omnidirectional view consistency, and 2) allow for free exploration in complex scene hierarchies. Existing methods either rely on successive scene expansion via inpainting or employ panorama representation to represent large FOV scene environments. However, the generated scene suffers from semantic drift during expansion and is unable to handle occlusion among scene hierarchies. To tackle these challenges, we introduce LayerPano3D, a novel framework for full-view, explorable panoramic 3D scene generation from a single text prompt. Our key insight is to decompose a reference 2D panorama into multiple layers at different depth levels, where each layer reveals the unseen space from the reference views via diffusion prior. LayerPano3D comprises multiple dedicated designs: 1) we introduce a novel text-guided anchor view synthesis pipeline for high-quality, consistent panorama generation. 2) We pioneer the Layered 3D Panorama as underlying representation to manage complex scene hierarchies and lift it into 3D Gaussians to splat detailed 360-degree omnidirectional scenes with unconstrained viewing paths. Extensive experiments demonstrate that our framework generates state-of-the-art 3D panoramic scene in both full view consistency and immersive exploratory experience. We believe that LayerPano3D holds promise for advancing 3D panoramic scene creation with numerous applications.

AK

41,891 Aufrufe • vor 2 Jahren

Gaussian Shell Maps are a new neural scene representation that connects fields and 3D Gaussians. This representation unlocks the full potential of 3D Gaussian splatting for generative AI applications, such as 3D avatar generation. 1/2

Gordon Wetzstein

52,555 Aufrufe • vor 2 Jahren

HoloDreamer can generate enclosed 3D scenes from text descriptions! It does so by first creating a high-quality equirectangular panorama and then rapidly reconstructing the 3D scene using 3D Gaussian Splatting. Links ⬇️

HoloDreamer can generate enclosed 3D scenes from text descriptions! It does so by first creating a high-quality equirectangular panorama and then rapidly reconstructing the 3D scene using 3D Gaussian Splatting. Links ⬇️

Dreaming Tulpa 🥓👑

59,422 Aufrufe • vor 2 Jahren

AnySplat Feed-forward 3D Gaussian Splatting from Unconstrained Views

AnySplat Feed-forward 3D Gaussian Splatting from Unconstrained Views

AK

18,803 Aufrufe • vor 1 Jahr

DreamBooth3D: Subject-Driven Text-to-3D Generation Personalized 3D models from just a few casual photos, with text-driven modifications abs: project page:

DreamBooth3D: Subject-Driven Text-to-3D Generation Personalized 3D models from just a few casual photos, with text-driven modifications abs: project page:

AK

107,503 Aufrufe • vor 3 Jahren

🚨 Hunyuan 3D 3.1 Pro and Rapid is here on fal! 🎯 Pro: High-fidelity Image-to-3D and Text-to-3D generation ⚡ Rapid: Speed-optimized 3D generation ✨ Smart Topology and Part generation for advanced 3D workflows

🚨 Hunyuan 3D 3.1 Pro and Rapid is here on fal! 🎯 Pro: High-fidelity Image-to-3D and Text-to-3D generation ⚡ Rapid: Speed-optimized 3D generation ✨ Smart Topology and Part generation for advanced 3D workflows

fal

26,138 Aufrufe • vor 9 Monaten

Is 3D scene generation much closer to being solved all of a sudden? It has been a few days since the release of OpenAI Sora. We run our COLMAP-Free 3D Gaussian Splatting on the released videos. Our method does not need to pre-process cameras and it seems we can directly just get 3D from the videos. Check out our results here. 🧵👇 (1/n

Is 3D scene generation much closer to being solved all of a sudden? It has been a few days since the release of OpenAI Sora. We run our COLMAP-Free 3D Gaussian Splatting on the released videos. Our method does not need to pre-process cameras and it seems we can directly just get 3D from the videos. Check out our results here. 🧵👇 (1/n

Xiaolong Wang

157,465 Aufrufe • vor 2 Jahren

AI Text to Animation working well! I created an entire pipeline around 3D asset generation, auto-rigging and then text to animation generation. (link in comments #gamedev #aigamedev #3dai

AI Text to Animation working well! I created an entire pipeline around 3D asset generation, auto-rigging and then text to animation generation. (link in comments #gamedev #aigamedev #3dai

DevDude

66,419 Aufrufe • vor 4 Monaten

CineMaster A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generation

CineMaster A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generation

AK

18,843 Aufrufe • vor 1 Jahr

DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior paper page: present DreamCraft3D, a hierarchical 3D content generation method that produces high-fidelity and coherent 3D objects. We tackle the problem by leveraging a 2D reference image to guide the stages of geometry sculpting and texture boosting. A central focus of this work is to address the consistency issue that existing works encounter. To sculpt geometries that render coherently, we perform score distillation sampling via a view-dependent diffusion model. This 3D prior, alongside several training strategies, prioritizes the geometry consistency but compromises the texture fidelity. We further propose Bootstrapped Score Distillation to specifically boost the texture. We train a personalized diffusion model, Dreambooth, on the augmented renderings of the scene, imbuing it with 3D knowledge of the scene being optimized. The score distillation from this 3D-aware diffusion prior provides view-consistent guidance for the scene. Notably, through an alternating optimization of the diffusion prior and 3D scene representation, we achieve mutually reinforcing improvements: the optimized 3D scene aids in training the scene-specific diffusion model, which offers increasingly view-consistent guidance for 3D optimization. The optimization is thus bootstrapped and leads to substantial texture boosting. With tailored 3D priors throughout the hierarchical generation, DreamCraft3D generates coherent 3D objects with photorealistic renderings, advancing the state-of-the-art in 3D content generation.

DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior paper page: present DreamCraft3D, a hierarchical 3D content generation method that produces high-fidelity and coherent 3D objects. We tackle the problem by leveraging a 2D reference image to guide the stages of geometry sculpting and texture boosting. A central focus of this work is to address the consistency issue that existing works encounter. To sculpt geometries that render coherently, we perform score distillation sampling via a view-dependent diffusion model. This 3D prior, alongside several training strategies, prioritizes the geometry consistency but compromises the texture fidelity. We further propose Bootstrapped Score Distillation to specifically boost the texture. We train a personalized diffusion model, Dreambooth, on the augmented renderings of the scene, imbuing it with 3D knowledge of the scene being optimized. The score distillation from this 3D-aware diffusion prior provides view-consistent guidance for the scene. Notably, through an alternating optimization of the diffusion prior and 3D scene representation, we achieve mutually reinforcing improvements: the optimized 3D scene aids in training the scene-specific diffusion model, which offers increasingly view-consistent guidance for 3D optimization. The optimization is thus bootstrapped and leads to substantial texture boosting. With tailored 3D priors throughout the hierarchical generation, DreamCraft3D generates coherent 3D objects with photorealistic renderings, advancing the state-of-the-art in 3D content generation.

AK

161,530 Aufrufe • vor 3 Jahren

We’re also publishing two research papers on the approaches we used for high quality 3D model generation & texture generation based on text prompts that are behind the Meta 3D Gen system. Meta 3D AssetGen ➡️ Meta 3D TextureGen ➡️

We’re also publishing two research papers on the approaches we used for high quality 3D model generation & texture generation based on text prompts that are behind the Meta 3D Gen system. Meta 3D AssetGen ➡️ Meta 3D TextureGen ➡️

AI at Meta

72,973 Aufrufe • vor 2 Jahren

📢GaussianGPT: autoregressive 3D Gaussian scene generation. We introduce a GPT-style model that directly generates 3D Gaussian scenes, token by token, in a series of small, discrete decision steps. Generation, completion, and large-scale outpainting in a single pipeline. Unlike diffusion-based approaches, GaussianGPT explicitly models the scene distribution at every step, allowing for quite flexible scene synthesis. 🌐 ▶️ Great work by Nicolas von Lützow, Barbara Roessle, Katharina Schmid

📢GaussianGPT: autoregressive 3D Gaussian scene generation. We introduce a GPT-style model that directly generates 3D Gaussian scenes, token by token, in a series of small, discrete decision steps. Generation, completion, and large-scale outpainting in a single pipeline. Unlike diffusion-based approaches, GaussianGPT explicitly models the scene distribution at every step, allowing for quite flexible scene synthesis. 🌐 ▶️ Great work by Nicolas von Lützow, Barbara Roessle, Katharina Schmid

Matthias Niessner

152,802 Aufrufe • vor 7 Monaten

3DTopia-XL High-Quality 3D PBR Asset Generation via Primitive Diffusion demo: model: 3DTopia-XL scales high-quality 3D asset generation using Diffusion Transformer (DiT) built upon an expressive and efficient 3D representation, PrimX. The denoising process takes 5 seconds to generate a 3D PBR asset from text/image input which is ready for the graphics pipeline to use.

3DTopia-XL High-Quality 3D PBR Asset Generation via Primitive Diffusion demo: model: 3DTopia-XL scales high-quality 3D asset generation using Diffusion Transformer (DiT) built upon an expressive and efficient 3D representation, PrimX. The denoising process takes 5 seconds to generate a 3D PBR asset from text/image input which is ready for the graphics pipeline to use.

AK

87,282 Aufrufe • vor 2 Jahren

3D AI is advancing quicker than we ever thought possible. Right now, we can use it to create individual 3D assets, but soon, building entire virtual worlds with just a text prompt could become a reality. This is from Meta 3D AI team.

3D AI is advancing quicker than we ever thought possible. Right now, we can use it to create individual 3D assets, but soon, building entire virtual worlds with just a text prompt could become a reality. This is from Meta 3D AI team.

el.cine

42,166 Aufrufe • vor 1 Jahr

Introducing AI 3D Generation ✨ 🔥 Text to 3D 🖼️ Image to 3D 🌳 Remix 📐 Editor integration & Exports Available now!

Introducing AI 3D Generation ✨ 🔥 Text to 3D 🖼️ Image to 3D 🌳 Remix 📐 Editor integration & Exports Available now!

Spline

705,817 Aufrufe • vor 2 Jahren

Microsoft presents World-R1 Reinforcing 3D Constraints for Text-to-Video Generation paper:

Microsoft presents World-R1 Reinforcing 3D Constraints for Text-to-Video Generation paper:

AK

24,625 Aufrufe • vor 6 Monaten