正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

📢WorldMesh is accepted to #ECCV2026, and we're releasing the code today! 🎉 Led by mschneider456: navigable, multi-room 3D scenes from a text prompt, with a mesh scaffold conditioning image diffusion for global consistency + photorealistic detail 👇

Angela Dai

10,024 subscribers

42,980 次观看 • 3 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

8 条评论

Angela Dai 的头像

Angela Dai3 个月前

Key idea: decouple structure from appearance. @mschneider456 builds a mesh scaffold (walls, floors, objects) from the prompt, then renders it to condition image synthesis, anchoring realistic appearance to explicit 3D geometry that stays consistent across rooms and close-ups

Angela Dai 的头像

Angela Dai3 个月前

Synthesized views are fused into 3D Gaussian splats, giving a navigable world with multi-room connectivity and robust 3D consistency. It scales linearly with the number of rooms.

clankr 的头像

clankr3 个月前

@mschneider456 A very clean implantation. Frankly, there seems to be a very significant leap in realism, which definitely has many applications in both model training and robotic simulation.

ValentynShum 的头像

ValentynShum3 个月前

@mschneider456 Wow

Emircan ERKUL 的头像

Emircan ERKUL3 个月前

@mschneider456 Cool idea! simplicity again wins! What left is extream light advanced AR glasses. RIP another 100 sector/product.

晚晚的头像

晚晚3 个月前

@mschneider456 这个多房间一致性看着挺难搞，代码开源太及时了

BlakeTheCoder 的头像

BlakeTheCoder3 个月前

@mschneider456 Really great work. I love the decoupling here. Structure before appearance finally solves the consistency problem end-to-end methods ignore.

Daniel 的头像

Daniel3 个月前

@MattNiessner @mschneider456 This is really cool!

相关视频

MVDream: Multi-view Diffusion for 3D Generation paper page: propose MVDream, a multi-view diffusion model that is able to generate geometrically consistent multi-view images from a given text prompt. By leveraging image diffusion models pre-trained on large-scale web datasets and a multi-view dataset rendered from 3D assets, the resulting multi-view diffusion model can achieve both the generalizability of 2D diffusion and the consistency of 3D data. Such a model can thus be applied as a multi-view prior for 3D generation via Score Distillation Sampling, where it greatly improves the stability of existing 2D-lifting methods by solving the 3D consistency problem. Finally, we show that the multi-view diffusion model can also be fine-tuned under a few shot setting for personalized 3D generation, i.e. DreamBooth3D application, where the consistency can be maintained after learning the subject identity.

MVDream: Multi-view Diffusion for 3D Generation paper page: propose MVDream, a multi-view diffusion model that is able to generate geometrically consistent multi-view images from a given text prompt. By leveraging image diffusion models pre-trained on large-scale web datasets and a multi-view dataset rendered from 3D assets, the resulting multi-view diffusion model can achieve both the generalizability of 2D diffusion and the consistency of 3D data. Such a model can thus be applied as a multi-view prior for 3D generation via Score Distillation Sampling, where it greatly improves the stability of existing 2D-lifting methods by solving the 3D consistency problem. Finally, we show that the multi-view diffusion model can also be fine-tuned under a few shot setting for personalized 3D generation, i.e. DreamBooth3D application, where the consistency can be maintained after learning the subject identity.

AK

294,519 次观看 • 3 年前

📢WorldAgents: 3D worlds only from 2D image models - without any training! We propose an agentic approach with a Director (VLM) to plan the scene, a Generator (Flux or NanoBanana) for new views, and a Verifier (VLM) for selection / 3D consistency. -> High-fidelity 3D worlds from a single text prompt. What's remarkable: our agents find consistent views from 2D image models to obtain 3D-consistent worlds; this shows that image models contain world priors - agents just need to find them! Great work by Ziya Erkoç Angela Dai

📢WorldAgents: 3D worlds only from 2D image models - without any training! We propose an agentic approach with a Director (VLM) to plan the scene, a Generator (Flux or NanoBanana) for new views, and a Verifier (VLM) for selection / 3D consistency. -> High-fidelity 3D worlds from a single text prompt. What's remarkable: our agents find consistent views from 2D image models to obtain 3D-consistent worlds; this shows that image models contain world priors - agents just need to find them! Great work by Ziya Erkoç Angela Dai

Matthias Niessner

19,093 次观看 • 6 个月前

Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery TL;DR: Skyfall-GS converts satellite images to explorable 3D urban scenes using diffusion models, with real-time rendering performance. Contributions: • We introduce Skyfall-GS, the first method to synthesize immersive, real-time, free-flight navigable 3D urban scenes solely from multi-view satellite imagery using generative refinement. • An open-domain refinement approach leverages pre-trained text-to-image diffusion models without domain-specific training. • A curriculum-learning-based iterative refinement strategy progressively enhances reconstruction quality from higher to lower viewpoints, significantly improving visual fidelity in occluded areas.

Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery TL;DR: Skyfall-GS converts satellite images to explorable 3D urban scenes using diffusion models, with real-time rendering performance. Contributions: • We introduce Skyfall-GS, the first method to synthesize immersive, real-time, free-flight navigable 3D urban scenes solely from multi-view satellite imagery using generative refinement. • An open-domain refinement approach leverages pre-trained text-to-image diffusion models without domain-specific training. • A curriculum-learning-based iterative refinement strategy progressively enhances reconstruction quality from higher to lower viewpoints, significantly improving visual fidelity in occluded areas.

MrNeRF

66,539 次观看 • 11 个月前

TurboEdit Instant text-based image editing discuss: We address the challenges of precise image inversion and disentangled image editing in the context of few-step diffusion models. We introduce an encoder based iterative inversion technique. The inversion network is conditioned on the input image and the reconstructed image from the previous step, allowing for correction of the next reconstruction towards the input image. We demonstrate that disentangled controls can be easily achieved in the few-step diffusion model by conditioning on an (automatically generated) detailed text prompt. To manipulate the inverted image, we freeze the noise maps and modify one attribute in the text prompt (either manually or via instruction based editing driven by an LLM), resulting in the generation of a new image similar to the input image with only one attribute changed. It can further control the editing strength and accept instructive text prompt. Our approach facilitates realistic text-guided image edits in real-time, requiring only 8 number of functional evaluations (NFEs) in inversion (one-time cost) and 4 NFEs per edit. Our method is not only fast, but also significantly outperforms state-of-the-art multi-step diffusion editing techniques.

TurboEdit Instant text-based image editing discuss: We address the challenges of precise image inversion and disentangled image editing in the context of few-step diffusion models. We introduce an encoder based iterative inversion technique. The inversion network is conditioned on the input image and the reconstructed image from the previous step, allowing for correction of the next reconstruction towards the input image. We demonstrate that disentangled controls can be easily achieved in the few-step diffusion model by conditioning on an (automatically generated) detailed text prompt. To manipulate the inverted image, we freeze the noise maps and modify one attribute in the text prompt (either manually or via instruction based editing driven by an LLM), resulting in the generation of a new image similar to the input image with only one attribute changed. It can further control the editing strength and accept instructive text prompt. Our approach facilitates realistic text-guided image edits in real-time, requiring only 8 number of functional evaluations (NFEs) in inversion (one-time cost) and 4 NFEs per edit. Our method is not only fast, but also significantly outperforms state-of-the-art multi-step diffusion editing techniques.

AK

16,062 次观看 • 2 年前

📢 Multi-View Image to 3D just got a major upgrade! Now your textures generate from multiple views too. So a few references of the same object bring the whole model back with much higher accuracy and consistency, from structure to texture.

📢 Multi-View Image to 3D just got a major upgrade! Now your textures generate from multiple views too. So a few references of the same object bring the whole model back with much higher accuracy and consistency, from structure to texture.

MeshyAI

131,590 次观看 • 2 个月前

Higgsfield Mod for Minecraft is live. > prompt any building or city, even the Statue of Liberty > create paintings with text-to-image > snap a view and restyle it with image-to-image > make videos from a prompt with text-to-video. > animate in-game photos with image-to-video

Higgsfield Mod for Minecraft is live. > prompt any building or city, even the Statue of Liberty > create paintings with text-to-image > snap a view and restyle it with image-to-image > make videos from a prompt with text-to-video. > animate in-game photos with image-to-video

Higgsfield AI 🧩

464,287 次观看 • 4 个月前

WorldExplorer: Towards Generating Fully Navigable 3D Scenes Contributions: • We introduce the first method for generating 3D scenes from text that supports high-quality view synthesis while enabling exploration across a wide range of camera poses. • We propose an iterative scene expansion strategy using video diffusion models, driven by trajectory sampling and adaptive collision detection. • We design a scene memory mechanism that conditions each video generation step on relevant past frames, improving view consistency and overall scene coherence.

WorldExplorer: Towards Generating Fully Navigable 3D Scenes Contributions: • We introduce the first method for generating 3D scenes from text that supports high-quality view synthesis while enabling exploration across a wide range of camera poses. • We propose an iterative scene expansion strategy using video diffusion models, driven by trajectory sampling and adaptive collision detection. • We design a scene memory mechanism that conditions each video generation step on relevant past frames, improving view consistency and overall scene coherence.

MrNeRF

23,814 次观看 • 1 年前

is it legal? NVIDIA and Microsoft just open-sourced the pieces of a full 3D studio for Opus 5.5 15 github repos, sorted by what you want to make: GENERATE 3D ASSETS 01. TRELLIS.2 by Microsoft: -> one image in, a fully textured 3D asset out, about 3 seconds on an H100 02. Hunyuan3D-2.1 by Tencent: -> production-ready 3D assets with PBR textures 03. stable-fast-3d by Stability AI: -> one photo to a clean UV-unwrapped mesh, built for speed 04. meshy-mcp-server by Meshy: -> text or image to 3D, then remesh, retexture, rig and animate from your agent BUILD SCENES AND WORLDS 05. mcp-for-blender: -> prompt-driven modeling, scene building and edits in a live Blender 06. HY-World-2.0 by Tencent: -> text or a single image to a navigable 3D world 07. supersplat by PlayCanvas: -> inspect, edit, optimize and publish 3D Gaussian splats in the browser 08. kit-usd-agents by NVIDIA: -> edit Omniverse USD scenes in plain English, plus MCP servers for Kit and Isaac Sim MAKE A GAME 09. unity-mcp: -> run the Unity Editor in plain English: scenes, C# scripts, assets, tests, builds 10. godot-ai: -> 46 tools for a live Godot editor: scenes, scripts, signals, UI, particles 11. unreal-engine-mcp: -> drive a live Unreal Editor: actors, world building, Blueprints 3D ON THE WEB 12. editor-mcp-server by PlayCanvas: -> your agent builds PlayCanvas scenes and checks they actually run 13. three.js: -> the JavaScript 3D library itself, 115k stars on GitHub 14. react-three-fiber: -> build three.js scenes as reusable React components 15. threejs-skills: -> 10 three.js skills for Claude Code, from shaders to post-processing a full 3D pipeline, and every piece of it lives on github

is it legal? NVIDIA and Microsoft just open-sourced the pieces of a full 3D studio for Opus 5.5 15 github repos, sorted by what you want to make: GENERATE 3D ASSETS 01. TRELLIS.2 by Microsoft: -> one image in, a fully textured 3D asset out, about 3 seconds on an H100 02. Hunyuan3D-2.1 by Tencent: -> production-ready 3D assets with PBR textures 03. stable-fast-3d by Stability AI: -> one photo to a clean UV-unwrapped mesh, built for speed 04. meshy-mcp-server by Meshy: -> text or image to 3D, then remesh, retexture, rig and animate from your agent BUILD SCENES AND WORLDS 05. mcp-for-blender: -> prompt-driven modeling, scene building and edits in a live Blender 06. HY-World-2.0 by Tencent: -> text or a single image to a navigable 3D world 07. supersplat by PlayCanvas: -> inspect, edit, optimize and publish 3D Gaussian splats in the browser 08. kit-usd-agents by NVIDIA: -> edit Omniverse USD scenes in plain English, plus MCP servers for Kit and Isaac Sim MAKE A GAME 09. unity-mcp: -> run the Unity Editor in plain English: scenes, C# scripts, assets, tests, builds 10. godot-ai: -> 46 tools for a live Godot editor: scenes, scripts, signals, UI, particles 11. unreal-engine-mcp: -> drive a live Unreal Editor: actors, world building, Blueprints 3D ON THE WEB 12. editor-mcp-server by PlayCanvas: -> your agent builds PlayCanvas scenes and checks they actually run 13. three.js: -> the JavaScript 3D library itself, 115k stars on GitHub 14. react-three-fiber: -> build three.js scenes as reusable React components 15. threejs-skills: -> 10 three.js skills for Claude Code, from shaders to post-processing a full 3D pipeline, and every piece of it lives on github

Mnimiy

106,339 次观看 • 1 天前

Stable Diffusion 3.5 from Stability AI is now LIVE on Civitai! Stable Diffusion 3.5 Large is a Multimodal Diffusion Transformer text-to-image model that features improved performance in image quality, typography, complex prompt understanding, and resource-efficiency. Generate with SD 3.5 Live on Civitai here👇🏽 All images used to create this video were generated with SD 3.5 by our community 🫶🏽

Stable Diffusion 3.5 from Stability AI is now LIVE on Civitai! Stable Diffusion 3.5 Large is a Multimodal Diffusion Transformer text-to-image model that features improved performance in image quality, typography, complex prompt understanding, and resource-efficiency. Generate with SD 3.5 Live on Civitai here👇🏽 All images used to create this video were generated with SD 3.5 by our community 🫶🏽

Civitai

31,503 次观看 • 1 年前

📢📢GenRecon: Bridging Generative Priors for Multi-View 3D Scene Reconstruction📢📢 Reconstructing high-fidelity 3D scenes from sparse RGB input is hard. It needs a strong 3D prior! We reformulate multi-view scene reconstruction as conditional 3D generation over overlapping spatial chunks, lifting posed image features into a generative shape prior via 3D conditioning. As an example prior, we build on Trellis2, and train it such that its reconstruction is pixel aligned and matches from all views. GenRecon achieves unprecedented reconstruction quality from any sparse RGB input sequence, even from a phone capture. The reconstruction also includes PBR materials which facilitates relighting and virtual object insertion. Amazing work by Katharina Schmid, Nicolas von Lützow, Jozef, Angela Dai

📢📢GenRecon: Bridging Generative Priors for Multi-View 3D Scene Reconstruction📢📢 Reconstructing high-fidelity 3D scenes from sparse RGB input is hard. It needs a strong 3D prior! We reformulate multi-view scene reconstruction as conditional 3D generation over overlapping spatial chunks, lifting posed image features into a generative shape prior via 3D conditioning. As an example prior, we build on Trellis2, and train it such that its reconstruction is pixel aligned and matches from all views. GenRecon achieves unprecedented reconstruction quality from any sparse RGB input sequence, even from a phone capture. The reconstruction also includes PBR materials which facilitates relighting and virtual object insertion. Amazing work by Katharina Schmid, Nicolas von Lützow, Jozef, Angela Dai

Matthias Niessner

18,351 次观看 • 4 个月前

Today, we launch Viggle V4 + Character Refine. - Strong character consistency with precise motion control - Upload 1 image → auto-generate multi-angle references - 1:1 motion transfer from reference videos - Multi-character swap in complex scenes - Generate in SECONDS at lower cost - Up to 60s videos Powered by our proprietary 3D foundation model, built for speed, reliability, and full creative control. Try it for free! 👇

Today, we launch Viggle V4 + Character Refine. - Strong character consistency with precise motion control - Upload 1 image → auto-generate multi-angle references - 1:1 motion transfer from reference videos - Multi-character swap in complex scenes - Generate in SECONDS at lower cost - Up to 60s videos Powered by our proprietary 3D foundation model, built for speed, reliability, and full creative control. Try it for free! 👇

ViggleAI

400,627 次观看 • 7 个月前

This is peak... Google just unveiled Genie 3 This AI generates photorealistic & 3D worlds from a text prompt and image... that you can explore in real-time Clearing a path towards AGI 10 wild examples + how to try below 1. Control a shiny marble

This is peak... Google just unveiled Genie 3 This AI generates photorealistic & 3D worlds from a text prompt and image... that you can explore in real-time Clearing a path towards AGI 10 wild examples + how to try below 1. Control a shiny marble

Linus ✦ Ekenstam

44,926 次观看 • 8 个月前

Marble is the first product from World Labs and is powered by our multimodal world model, which lets anyone create high-fidelity, persistent 3D worlds from just a single image, video, text prompt, or 3D layout. Read more at

Marble is the first product from World Labs and is powered by our multimodal world model, which lets anyone create high-fidelity, persistent 3D worlds from just a single image, video, text prompt, or 3D layout. Read more at

World Labs

117,588 次观看 • 11 个月前

DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior paper page: present DreamCraft3D, a hierarchical 3D content generation method that produces high-fidelity and coherent 3D objects. We tackle the problem by leveraging a 2D reference image to guide the stages of geometry sculpting and texture boosting. A central focus of this work is to address the consistency issue that existing works encounter. To sculpt geometries that render coherently, we perform score distillation sampling via a view-dependent diffusion model. This 3D prior, alongside several training strategies, prioritizes the geometry consistency but compromises the texture fidelity. We further propose Bootstrapped Score Distillation to specifically boost the texture. We train a personalized diffusion model, Dreambooth, on the augmented renderings of the scene, imbuing it with 3D knowledge of the scene being optimized. The score distillation from this 3D-aware diffusion prior provides view-consistent guidance for the scene. Notably, through an alternating optimization of the diffusion prior and 3D scene representation, we achieve mutually reinforcing improvements: the optimized 3D scene aids in training the scene-specific diffusion model, which offers increasingly view-consistent guidance for 3D optimization. The optimization is thus bootstrapped and leads to substantial texture boosting. With tailored 3D priors throughout the hierarchical generation, DreamCraft3D generates coherent 3D objects with photorealistic renderings, advancing the state-of-the-art in 3D content generation.

DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior paper page: present DreamCraft3D, a hierarchical 3D content generation method that produces high-fidelity and coherent 3D objects. We tackle the problem by leveraging a 2D reference image to guide the stages of geometry sculpting and texture boosting. A central focus of this work is to address the consistency issue that existing works encounter. To sculpt geometries that render coherently, we perform score distillation sampling via a view-dependent diffusion model. This 3D prior, alongside several training strategies, prioritizes the geometry consistency but compromises the texture fidelity. We further propose Bootstrapped Score Distillation to specifically boost the texture. We train a personalized diffusion model, Dreambooth, on the augmented renderings of the scene, imbuing it with 3D knowledge of the scene being optimized. The score distillation from this 3D-aware diffusion prior provides view-consistent guidance for the scene. Notably, through an alternating optimization of the diffusion prior and 3D scene representation, we achieve mutually reinforcing improvements: the optimized 3D scene aids in training the scene-specific diffusion model, which offers increasingly view-consistent guidance for 3D optimization. The optimization is thus bootstrapped and leads to substantial texture boosting. With tailored 3D priors throughout the hierarchical generation, DreamCraft3D generates coherent 3D objects with photorealistic renderings, advancing the state-of-the-art in 3D content generation.

AK

161,530 次观看 • 3 年前

Today, we are releasing Stable Video Diffusion, our first foundation model for generative AI video based on the image model, Stable Diffusion. As part of this research preview, the code, weights, and research paper are now available. Additionally, today you can sign up for our waitlist to access a new upcoming web experience featuring a Text-To-Video interface. To access the model & sign up for our waitlist, visit our website here:

Today, we are releasing Stable Video Diffusion, our first foundation model for generative AI video based on the image model, Stable Diffusion. As part of this research preview, the code, weights, and research paper are now available. Additionally, today you can sign up for our waitlist to access a new upcoming web experience featuring a Text-To-Video interface. To access the model & sign up for our waitlist, visit our website here:

Stability AI

1,024,795 次观看 • 2 年前

3DTopia-XL High-Quality 3D PBR Asset Generation via Primitive Diffusion demo: model: 3DTopia-XL scales high-quality 3D asset generation using Diffusion Transformer (DiT) built upon an expressive and efficient 3D representation, PrimX. The denoising process takes 5 seconds to generate a 3D PBR asset from text/image input which is ready for the graphics pipeline to use.

3DTopia-XL High-Quality 3D PBR Asset Generation via Primitive Diffusion demo: model: 3DTopia-XL scales high-quality 3D asset generation using Diffusion Transformer (DiT) built upon an expressive and efficient 3D representation, PrimX. The denoising process takes 5 seconds to generate a 3D PBR asset from text/image input which is ready for the graphics pipeline to use.

AK

87,282 次观看 • 2 年前

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.

AK

143,469 次观看 • 3 年前