正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

ControlMat: Controlled Generative Approach to Material Capture by Giuseppe Vecchio et al. From a single photo to clean, realistics, tileable PBR materials with this diffusion-based matching model.

Ben Ferns

3,145 subscribers

120,225 次观看 • 2 年前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

9 条评论

Two Minute Papers 的头像

Two Minute Papers2 年前

Congratulations! Sorry, I am on the phone - where is the supp. video this is from?

Ben Ferns 的头像

Ben Ferns2 年前

Hi! I'm just sharing the project (not one of the authors), but you can find that video on the supplemental page

Valentin Deschaintre 的头像

Valentin Deschaintre2 年前

@giuvecchio95 looks like your hard work is appreciated :)

Pete Thorne 的头像

Pete Thorne2 年前

Now this is more like it! Plenty of artists been asking for tools like this.

Pedro Silva 的头像

Pedro Silva2 年前

This is pretty cool

Bruno Galerne 的头像

Bruno Galerne2 年前

Impressive! @boubek

zak 的头像

zak2 年前

@kvollstaedt

Edzward 的头像

Edzward2 年前

Fake. Clearly achieved by the Dark Arts of Sorcery! 😳😳😳

Felix 'InterVR' Krell 的头像

Felix 'InterVR' Krell2 年前

@AnselmZielonka

相关视频

NeRF-Texture by Yihua Huang et al. The possibility of a NeRF-based 3D editing pipeline continues to expand with this model that allows sampling of scenes to re-map materials/volumetric-detail to existing shapes.

NeRF-Texture by Yihua Huang et al. The possibility of a NeRF-based 3D editing pipeline continues to expand with this model that allows sampling of scenes to re-map materials/volumetric-detail to existing shapes.

Ben Ferns

56,059 次观看 • 3 年前

DiffSplat Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation DiffSplat is a generative framework to synthesize 3D Gaussian Splats from text prompts & single-view images in ⚡️ 1~2 seconds. It is fine-tuned directly from a pretrained text-to-image diffusion model.

DiffSplat Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation DiffSplat is a generative framework to synthesize 3D Gaussian Splats from text prompts & single-view images in ⚡️ 1~2 seconds. It is fine-tuned directly from a pretrained text-to-image diffusion model.

AK

38,416 次观看 • 1 年前

ReCamMaster Camera-Controlled Generative Rendering from A Single Video

ReCamMaster Camera-Controlled Generative Rendering from A Single Video

AK

256,278 次观看 • 1 年前

📢 Automatic PBR : Image-to-material (I2M) 👉Textured Mesh in > Full PBR Mesh out Our latest foundation model for enriching existing 3D assets with the full array of PBR materials, whilst maintaining the same mesh geometry and UV unwrap. #I2M #PBR #CV #Marso #texturing #gamedev #computervision M-XR

📢 Automatic PBR : Image-to-material (I2M) 👉Textured Mesh in > Full PBR Mesh out Our latest foundation model for enriching existing 3D assets with the full array of PBR materials, whilst maintaining the same mesh geometry and UV unwrap. #I2M #PBR #CV #Marso #texturing #gamedev #computervision M-XR

Elliott Round

13,353 次观看 • 2 个月前

I'm thrilled to officially annouce🤩 📢 Automatic PBR : Image-to-material (I2M) 👉Textured Mesh in > Full PBR Mesh out Our latest foundation model for enriching existing 3D assets with the full array of PBR materials, whilst maintaining the same mesh geometry and UV unwrap. 🧵A thread : 1 of 7 👇 #I2M #PBR #CV #Marso #texturing #gamedev #computervision M-XR

I'm thrilled to officially annouce🤩 📢 Automatic PBR : Image-to-material (I2M) 👉Textured Mesh in > Full PBR Mesh out Our latest foundation model for enriching existing 3D assets with the full array of PBR materials, whilst maintaining the same mesh geometry and UV unwrap. 🧵A thread : 1 of 7 👇 #I2M #PBR #CV #Marso #texturing #gamedev #computervision M-XR

Elliott Round

36,221 次观看 • 2 个月前

I finally released my new video on YouTube about Diffusion Models / Score-Based Generative Models. Literally planned this for a year and put so much work in. I think this approach to diffusion models is so intuitive and highly recommend giving that a go! Video is 38min long, so you will need some time to watch that haha.

I finally released my new video on YouTube about Diffusion Models / Score-Based Generative Models. Literally planned this for a year and put so much work in. I think this approach to diffusion models is so intuitive and highly recommend giving that a go! Video is 38min long, so you will need some time to watch that haha.

dome | Outlier

54,560 次观看 • 1 年前

Trying to capture some more different and complex materials, metals look pretty good! Rendered in #b3d #pbr #photometry #solver

Trying to capture some more different and complex materials, metals look pretty good! Rendered in #b3d #pbr #photometry #solver

McGavish

13,315 次观看 • 2 年前

Capture realistic 3D assets with just your phone! 🤳 🪩Full PBR Textures, and multi-engine support, with many more material channels to come 📰 Join the waitlist: #3D #PBR #MaterialCapture #DigitalTwin #CGI #Photogrammetry

Capture realistic 3D assets with just your phone! 🤳 🪩Full PBR Textures, and multi-engine support, with many more material channels to come 📰 Join the waitlist: #3D #PBR #MaterialCapture #DigitalTwin #CGI #Photogrammetry

M-XR

31,983 次观看 • 2 年前

[SIGGRAPH ASIA '25] Detail-Enhanced Gaussian Splatting for Large-Scale Volumetric Capture Contributions: - A two-stage approach to performance capture, combining a scene-scale capture rig and a single-actor facial capture rig. - A novel high-quality scene-scale volumetric performance capture rig, incorporating both static and dynamic cameras to track the performance of multiple actors. - A reconstruction pipeline for dynamic performance capture, featuring stable calibration of moving cameras and 4DGS with improved dynamic range and color fidelity. - A detail enhancement Diffusion Model, which supports 4K, RGB, and Alpha, with improved temporal stability.

[SIGGRAPH ASIA '25] Detail-Enhanced Gaussian Splatting for Large-Scale Volumetric Capture Contributions: - A two-stage approach to performance capture, combining a scene-scale capture rig and a single-actor facial capture rig. - A novel high-quality scene-scale volumetric performance capture rig, incorporating both static and dynamic cameras to track the performance of multiple actors. - A reconstruction pipeline for dynamic performance capture, featuring stable calibration of moving cameras and 4DGS with improved dynamic range and color fidelity. - A detail enhancement Diffusion Model, which supports 4K, RGB, and Alpha, with improved temporal stability.

MrNeRF

42,382 次观看 • 8 个月前

$Example-based Motion Synthesis via Generative Motion Matching paper page: present GenMM, a generative model that "mines" as many diverse motions as possible from a single or few example sequences. In stark contrast to existing data-driven methods, which typically require long offline training time, are prone to visual artifacts, and tend to fail on large and complex skeletons, GenMM inherits the training-free nature and the superior quality of the well-known Motion Matching method. GenMM can synthesize a high-quality motion within a fraction of a second, even with highly complex and large skeletal structures. At the heart of our generative framework lies the generative motion matching module, which utilizes the bidirectional visual similarity as a generative cost function to motion matching, and operates in a multi-stage framework to progressively refine a random guess using exemplar motion matches. In addition to diverse motion generation, we show the versatility of our generative framework by extending it to a number of scenarios that are not possible with motion matching alone, including motion completion, key frame-guided generation, infinite looping, and motion reassembly.$

Example-based Motion Synthesis via Generative Motion Matching paper page: present GenMM, a generative model that "mines" as many diverse motions as possible from a single or few example sequences. In stark contrast to existing data-driven methods, which typically require long offline training time, are prone to visual artifacts, and tend to fail on large and complex skeletons, GenMM inherits the training-free nature and the superior quality of the well-known Motion Matching method. GenMM can synthesize a high-quality motion within a fraction of a second, even with highly complex and large skeletal structures. At the heart of our generative framework lies the generative motion matching module, which utilizes the bidirectional visual similarity as a generative cost function to motion matching, and operates in a multi-stage framework to progressively refine a random guess using exemplar motion matches. In addition to diverse motion generation, we show the versatility of our generative framework by extending it to a number of scenarios that are not possible with motion matching alone, including motion completion, key frame-guided generation, infinite looping, and motion reassembly.

AK

93,694 次观看 • 3 年前

MaterialFusion Enhancing Inverse Rendering with Material Diffusion Priors discuss: Recent works in inverse rendering have shown promise in using multi-view images of an object to recover shape, albedo, and materials. However, the recovered components often fail to render accurately under new lighting conditions due to the intrinsic challenge of disentangling albedo and material properties from input images. To address this challenge, we introduce MaterialFusion, an enhanced conventional 3D inverse rendering pipeline that incorporates a 2D prior on texture and material properties. We present StableMaterial, a 2D diffusion model prior that refines multi-lit data to estimate the most likely albedo and material from given input appearances. This model is trained on albedo, material, and relit image data derived from a curated dataset of approximately ~12K artist-designed synthetic Blender objects called BlenderVault. we incorporate this diffusion prior with an inverse rendering framework where we use score distillation sampling (SDS) to guide the optimization of the albedo and materials, improving relighting performance in comparison with previous work. We validate MaterialFusion's relighting performance on 4 datasets of synthetic and real objects under diverse illumination conditions, showing our diffusion-aided approach significantly improves the appearance of reconstructed objects under novel lighting conditions. We intend to publicly release our BlenderVault dataset to support further research in this field.

MaterialFusion Enhancing Inverse Rendering with Material Diffusion Priors discuss: Recent works in inverse rendering have shown promise in using multi-view images of an object to recover shape, albedo, and materials. However, the recovered components often fail to render accurately under new lighting conditions due to the intrinsic challenge of disentangling albedo and material properties from input images. To address this challenge, we introduce MaterialFusion, an enhanced conventional 3D inverse rendering pipeline that incorporates a 2D prior on texture and material properties. We present StableMaterial, a 2D diffusion model prior that refines multi-lit data to estimate the most likely albedo and material from given input appearances. This model is trained on albedo, material, and relit image data derived from a curated dataset of approximately ~12K artist-designed synthetic Blender objects called BlenderVault. we incorporate this diffusion prior with an inverse rendering framework where we use score distillation sampling (SDS) to guide the optimization of the albedo and materials, improving relighting performance in comparison with previous work. We validate MaterialFusion's relighting performance on 4 datasets of synthetic and real objects under diverse illumination conditions, showing our diffusion-aided approach significantly improves the appearance of reconstructed objects under novel lighting conditions. We intend to publicly release our BlenderVault dataset to support further research in this field.

AK

22,949 次观看 • 1 年前

Super clean and efficient meshes by an AI? YES! The typical 3D Generative AI solutions produce lots of artifacts and usually way to many polygons due to volumetric approaches. In comparison “MeshGPT creates triangle meshes by autoregressively sampling from a transformer model that has been trained to produce tokens from a learned geometric vocabulary. These tokens can then be decoded into the faces of a triangle mesh. This method generates clean, coherent, and compact meshes, characterized by sharp edges and high fidelity.” Surely it is limited by the trained vocabulary but various versions can be trained for specific sets to create generative model libraries for certain object groups. Very promising approach with the high quality.

Super clean and efficient meshes by an AI? YES! The typical 3D Generative AI solutions produce lots of artifacts and usually way to many polygons due to volumetric approaches. In comparison “MeshGPT creates triangle meshes by autoregressively sampling from a transformer model that has been trained to produce tokens from a learned geometric vocabulary. These tokens can then be decoded into the faces of a triangle mesh. This method generates clean, coherent, and compact meshes, characterized by sharp edges and high fidelity.” Surely it is limited by the trained vocabulary but various versions can be trained for specific sets to create generative model libraries for certain object groups. Very promising approach with the high quality.

René Schulte

20,772 次观看 • 2 年前

Oh wow! "[SIGGRAPH '24] DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models" Paper: Project: Code (MIT) :

Oh wow! "[SIGGRAPH '24] DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models" Paper: Project: Code (MIT) :

MrNeRF

59,766 次观看 • 2 年前

An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion discuss: We introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance, and patch structures within a 64x64 pixel image, effectively converting complex 3D shapes into a more manageable 2D format. By doing so, we address the challenges of both geometric and semantic irregularity inherent in polygonal meshes. This method allows us to use image generation models, such as Diffusion Transformers, directly for 3D shape generation. Evaluated on the ABO dataset, our generated shapes with patch structures achieve point cloud FID comparable to recent 3D generative models, while naturally supporting PBR material generation.

An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion discuss: We introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance, and patch structures within a 64x64 pixel image, effectively converting complex 3D shapes into a more manageable 2D format. By doing so, we address the challenges of both geometric and semantic irregularity inherent in polygonal meshes. This method allows us to use image generation models, such as Diffusion Transformers, directly for 3D shape generation. Evaluated on the ABO dataset, our generated shapes with patch structures achieve point cloud FID comparable to recent 3D generative models, while naturally supporting PBR material generation.

AK

66,435 次观看 • 1 年前

Today, we are releasing Stable Video Diffusion, our first foundation model for generative AI video based on the image model, Stable Diffusion. As part of this research preview, the code, weights, and research paper are now available. Additionally, today you can sign up for our waitlist to access a new upcoming web experience featuring a Text-To-Video interface. To access the model & sign up for our waitlist, visit our website here:

Today, we are releasing Stable Video Diffusion, our first foundation model for generative AI video based on the image model, Stable Diffusion. As part of this research preview, the code, weights, and research paper are now available. Additionally, today you can sign up for our waitlist to access a new upcoming web experience featuring a Text-To-Video interface. To access the model & sign up for our waitlist, visit our website here:

Stability AI

1,024,438 次观看 • 2 年前

Daily #Blender Tip: Learn how to create materials for games and films using PBR textures in Blender with this easy-to-follow tutorial video. PBR textures are a must-have for achieving realistic materials and this video will show you how to quickly set them up in Blender. #B3D

Daily #Blender Tip: Learn how to create materials for games and films using PBR textures in Blender with this easy-to-follow tutorial video. PBR textures are a must-have for achieving realistic materials and this video will show you how to quickly set them up in Blender. #B3D

Jan van den Hemel

40,821 次观看 • 3 年前

We are thrilled to release the next leap in art-grade 3D generative models. Our single-click model pipeline gives unprecedented mesh outputs, with mesh parts-based topology. It is available now for all Cube tiers to start for free. ☑️ Our multi-stage hierarchical AI models produce a fully assembled 3D mesh with adaptive poly-counts, providing the clean, separated topology you need. ✅ A parts-based approach enables high-resolution meshes. 🌐 Quad and Triangle mesh support. Access now:

We are thrilled to release the next leap in art-grade 3D generative models. Our single-click model pipeline gives unprecedented mesh outputs, with mesh parts-based topology. It is available now for all Cube tiers to start for free. ☑️ Our multi-stage hierarchical AI models produce a fully assembled 3D mesh with adaptive poly-counts, providing the clean, separated topology you need. ✅ A parts-based approach enables high-resolution meshes. 🌐 Quad and Triangle mesh support. Access now:

Common Sense Machines

420,323 次观看 • 1 年前

From Forger to Nomad the only thing Nomad was unable to import was the PBR Materials

From Forger to Nomad the only thing Nomad was unable to import was the PBR Materials

Juan Carlos Montes

16,654 次观看 • 2 年前

Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion abs: project page:

Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion abs: project page:

AK

262,963 次观看 • 3 年前