Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

ControlMat: Controlled Generative Approach to Material Capture by Giuseppe Vecchio et al. From a single photo to clean, realistics, tileable PBR materials with this diffusion-based matching model.

ben_ferns's profile picture

Ben Ferns

3,141 subscribers

120,218 Aufrufe • vor 2 Jahren •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

9 Kommentare

Profilbild von Two Minute Papers
Two Minute Papersvor 2 Jahren

Congratulations! Sorry, I am on the phone - where is the supp. video this is from?

Profilbild von Ben Ferns
Ben Fernsvor 2 Jahren

Hi! I'm just sharing the project (not one of the authors), but you can find that video on the supplemental page

Profilbild von Valentin Deschaintre
Valentin Deschaintrevor 2 Jahren

@giuvecchio95 looks like your hard work is appreciated :)

Profilbild von Pete Thorne
Pete Thornevor 2 Jahren

Now this is more like it! Plenty of artists been asking for tools like this.

Profilbild von Pedro Silva
Pedro Silvavor 2 Jahren

This is pretty cool

Profilbild von Bruno Galerne
Bruno Galernevor 2 Jahren

Impressive! @boubek

Profilbild von zak
zakvor 2 Jahren

@kvollstaedt

Profilbild von Edzward
Edzwardvor 2 Jahren

Fake. Clearly achieved by the Dark Arts of Sorcery! 😳😳😳

Profilbild von Felix 'InterVR' Krell
Felix 'InterVR' Krellvor 2 Jahren

@AnselmZielonka

Ähnliche Videos

NeRF-Texture by Yihua Huang et al. The possibility of a NeRF-based 3D editing pipeline continues to expand with this model that allows sampling of scenes to re-map materials/volumetric-detail to existing shapes.
0:56

NeRF-Texture by Yihua Huang et al. The possibility of a NeRF-based 3D editing pipeline continues to expand with this model that allows sampling of scenes to re-map materials/volumetric-detail to existing shapes.

Ben Ferns

56,059 Aufrufe • vor 3 Jahren

DiffSplat Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation DiffSplat is a generative framework to synthesize 3D Gaussian Splats from text prompts & single-view images in ⚡️ 1~2 seconds. It is fine-tuned directly from a pretrained text-to-image diffusion model.
0:24

DiffSplat Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation DiffSplat is a generative framework to synthesize 3D Gaussian Splats from text prompts & single-view images in ⚡️ 1~2 seconds. It is fine-tuned directly from a pretrained text-to-image diffusion model.

AK

38,416 Aufrufe • vor 1 Jahr

ReCamMaster Camera-Controlled Generative Rendering from A Single Video
0:43

ReCamMaster Camera-Controlled Generative Rendering from A Single Video

AK

256,278 Aufrufe • vor 1 Jahr

📢 Automatic PBR : Image-to-material (I2M) 👉Textured Mesh in > Full PBR Mesh out Our latest foundation model for enriching existing 3D assets with the full array of PBR materials, whilst maintaining the same mesh geometry and UV unwrap. #I2M #PBR #CV #Marso #texturing #gamedev #computervision M-XR
0:20

📢 Automatic PBR : Image-to-material (I2M) 👉Textured Mesh in > Full PBR Mesh out Our latest foundation model for enriching existing 3D assets with the full array of PBR materials, whilst maintaining the same mesh geometry and UV unwrap. #I2M #PBR #CV #Marso #texturing #gamedev #computervision M-XR

Elliott Round

13,353 Aufrufe • vor 2 Monaten

I'm thrilled to officially annouce🤩 📢 Automatic PBR : Image-to-material (I2M) 👉Textured Mesh in > Full PBR Mesh out Our latest foundation model for enriching existing 3D assets with the full array of PBR materials, whilst maintaining the same mesh geometry and UV unwrap. 🧵A thread : 1 of 7 👇 #I2M #PBR #CV #Marso #texturing #gamedev #computervision M-XR
1:15

I'm thrilled to officially annouce🤩 📢 Automatic PBR : Image-to-material (I2M) 👉Textured Mesh in > Full PBR Mesh out Our latest foundation model for enriching existing 3D assets with the full array of PBR materials, whilst maintaining the same mesh geometry and UV unwrap. 🧵A thread : 1 of 7 👇 #I2M #PBR #CV #Marso #texturing #gamedev #computervision M-XR

Elliott Round

36,221 Aufrufe • vor 1 Monat

I finally released my new video on YouTube about Diffusion Models / Score-Based Generative Models. Literally planned this for a year and put so much work in. I think this approach to diffusion models is so intuitive and highly recommend giving that a go! Video is 38min long, so you will need some time to watch that haha.
0:59

I finally released my new video on YouTube about Diffusion Models / Score-Based Generative Models. Literally planned this for a year and put so much work in. I think this approach to diffusion models is so intuitive and highly recommend giving that a go! Video is 38min long, so you will need some time to watch that haha.

dome | Outlier

54,560 Aufrufe • vor 1 Jahr

Trying to capture some more different and complex materials, metals look pretty good! Rendered in #b3d #pbr #photometry #solver
0:24

Trying to capture some more different and complex materials, metals look pretty good! Rendered in #b3d #pbr #photometry #solver

McGavish

13,315 Aufrufe • vor 2 Jahren

Capture realistic 3D assets with just your phone! 🤳 🪩Full PBR Textures, and multi-engine support, with many more material channels to come 📰 Join the waitlist: #3D #PBR #MaterialCapture #DigitalTwin #CGI #Photogrammetry
1:38

Capture realistic 3D assets with just your phone! 🤳 🪩Full PBR Textures, and multi-engine support, with many more material channels to come 📰 Join the waitlist: #3D #PBR #MaterialCapture #DigitalTwin #CGI #Photogrammetry

M-XR

31,983 Aufrufe • vor 2 Jahren

[SIGGRAPH ASIA '25] Detail-Enhanced Gaussian Splatting for Large-Scale Volumetric Capture Contributions: - A two-stage approach to performance capture, combining a scene-scale capture rig and a single-actor facial capture rig. - A novel high-quality scene-scale volumetric performance capture rig, incorporating both static and dynamic cameras to track the performance of multiple actors. - A reconstruction pipeline for dynamic performance capture, featuring stable calibration of moving cameras and 4DGS with improved dynamic range and color fidelity. - A detail enhancement Diffusion Model, which supports 4K, RGB, and Alpha, with improved temporal stability.
0:20

[SIGGRAPH ASIA '25] Detail-Enhanced Gaussian Splatting for Large-Scale Volumetric Capture Contributions: - A two-stage approach to performance capture, combining a scene-scale capture rig and a single-actor facial capture rig. - A novel high-quality scene-scale volumetric performance capture rig, incorporating both static and dynamic cameras to track the performance of multiple actors. - A reconstruction pipeline for dynamic performance capture, featuring stable calibration of moving cameras and 4DGS with improved dynamic range and color fidelity. - A detail enhancement Diffusion Model, which supports 4K, RGB, and Alpha, with improved temporal stability.

MrNeRF

42,382 Aufrufe • vor 7 Monaten

Example-based Motion Synthesis via Generative Motion Matching paper page: present GenMM, a generative model that "mines" as many diverse motions as possible from a single or few example sequences. In stark contrast to existing data-driven methods, which typically require long offline training time, are prone to visual artifacts, and tend to fail on large and complex skeletons, GenMM inherits the training-free nature and the superior quality of the well-known Motion Matching method. GenMM can synthesize a high-quality motion within a fraction of a second, even with highly complex and large skeletal structures. At the heart of our generative framework lies the generative motion matching module, which utilizes the bidirectional visual similarity as a generative cost function to motion matching, and operates in a multi-stage framework to progressively refine a random guess using exemplar motion matches. In addition to diverse motion generation, we show the versatility of our generative framework by extending it to a number of scenarios that are not possible with motion matching alone, including motion completion, key frame-guided generation, infinite looping, and motion reassembly.
5:56

Example-based Motion Synthesis via Generative Motion Matching paper page: present GenMM, a generative model that "mines" as many diverse motions as possible from a single or few example sequences. In stark contrast to existing data-driven methods, which typically require long offline training time, are prone to visual artifacts, and tend to fail on large and complex skeletons, GenMM inherits the training-free nature and the superior quality of the well-known Motion Matching method. GenMM can synthesize a high-quality motion within a fraction of a second, even with highly complex and large skeletal structures. At the heart of our generative framework lies the generative motion matching module, which utilizes the bidirectional visual similarity as a generative cost function to motion matching, and operates in a multi-stage framework to progressively refine a random guess using exemplar motion matches. In addition to diverse motion generation, we show the versatility of our generative framework by extending it to a number of scenarios that are not possible with motion matching alone, including motion completion, key frame-guided generation, infinite looping, and motion reassembly.

AK

93,694 Aufrufe • vor 3 Jahren

MaterialFusion Enhancing Inverse Rendering with Material Diffusion Priors discuss: Recent works in inverse rendering have shown promise in using multi-view images of an object to recover shape, albedo, and materials. However, the recovered components often fail to render accurately under new lighting conditions due to the intrinsic challenge of disentangling albedo and material properties from input images. To address this challenge, we introduce MaterialFusion, an enhanced conventional 3D inverse rendering pipeline that incorporates a 2D prior on texture and material properties. We present StableMaterial, a 2D diffusion model prior that refines multi-lit data to estimate the most likely albedo and material from given input appearances. This model is trained on albedo, material, and relit image data derived from a curated dataset of approximately ~12K artist-designed synthetic Blender objects called BlenderVault. we incorporate this diffusion prior with an inverse rendering framework where we use score distillation sampling (SDS) to guide the optimization of the albedo and materials, improving relighting performance in comparison with previous work. We validate MaterialFusion's relighting performance on 4 datasets of synthetic and real objects under diverse illumination conditions, showing our diffusion-aided approach significantly improves the appearance of reconstructed objects under novel lighting conditions. We intend to publicly release our BlenderVault dataset to support further research in this field.
0:43

MaterialFusion Enhancing Inverse Rendering with Material Diffusion Priors discuss: Recent works in inverse rendering have shown promise in using multi-view images of an object to recover shape, albedo, and materials. However, the recovered components often fail to render accurately under new lighting conditions due to the intrinsic challenge of disentangling albedo and material properties from input images. To address this challenge, we introduce MaterialFusion, an enhanced conventional 3D inverse rendering pipeline that incorporates a 2D prior on texture and material properties. We present StableMaterial, a 2D diffusion model prior that refines multi-lit data to estimate the most likely albedo and material from given input appearances. This model is trained on albedo, material, and relit image data derived from a curated dataset of approximately ~12K artist-designed synthetic Blender objects called BlenderVault. we incorporate this diffusion prior with an inverse rendering framework where we use score distillation sampling (SDS) to guide the optimization of the albedo and materials, improving relighting performance in comparison with previous work. We validate MaterialFusion's relighting performance on 4 datasets of synthetic and real objects under diverse illumination conditions, showing our diffusion-aided approach significantly improves the appearance of reconstructed objects under novel lighting conditions. We intend to publicly release our BlenderVault dataset to support further research in this field.

AK

22,949 Aufrufe • vor 1 Jahr

Super clean and efficient meshes by an AI? YES! The typical 3D Generative AI solutions produce lots of artifacts and usually way to many polygons due to volumetric approaches. In comparison “MeshGPT creates triangle meshes by autoregressively sampling from a transformer model that has been trained to produce tokens from a learned geometric vocabulary. These tokens can then be decoded into the faces of a triangle mesh. This method generates clean, coherent, and compact meshes, characterized by sharp edges and high fidelity.” Surely it is limited by the trained vocabulary but various versions can be trained for specific sets to create generative model libraries for certain object groups. Very promising approach with the high quality.
0:25

Super clean and efficient meshes by an AI? YES! The typical 3D Generative AI solutions produce lots of artifacts and usually way to many polygons due to volumetric approaches. In comparison “MeshGPT creates triangle meshes by autoregressively sampling from a transformer model that has been trained to produce tokens from a learned geometric vocabulary. These tokens can then be decoded into the faces of a triangle mesh. This method generates clean, coherent, and compact meshes, characterized by sharp edges and high fidelity.” Surely it is limited by the trained vocabulary but various versions can be trained for specific sets to create generative model libraries for certain object groups. Very promising approach with the high quality.

René Schulte

20,772 Aufrufe • vor 2 Jahren

Oh wow! "[SIGGRAPH '24] DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models" Paper: Project: Code (MIT) :
1:04

Oh wow! "[SIGGRAPH '24] DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models" Paper: Project: Code (MIT) :

MrNeRF

59,480 Aufrufe • vor 2 Jahren

An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion discuss: We introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance, and patch structures within a 64x64 pixel image, effectively converting complex 3D shapes into a more manageable 2D format. By doing so, we address the challenges of both geometric and semantic irregularity inherent in polygonal meshes. This method allows us to use image generation models, such as Diffusion Transformers, directly for 3D shape generation. Evaluated on the ABO dataset, our generated shapes with patch structures achieve point cloud FID comparable to recent 3D generative models, while naturally supporting PBR material generation.
0:34

An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion discuss: We introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance, and patch structures within a 64x64 pixel image, effectively converting complex 3D shapes into a more manageable 2D format. By doing so, we address the challenges of both geometric and semantic irregularity inherent in polygonal meshes. This method allows us to use image generation models, such as Diffusion Transformers, directly for 3D shape generation. Evaluated on the ABO dataset, our generated shapes with patch structures achieve point cloud FID comparable to recent 3D generative models, while naturally supporting PBR material generation.

AK

66,412 Aufrufe • vor 1 Jahr

Today, we are releasing Stable Video Diffusion, our first foundation model for generative AI video based on the image model, Stable Diffusion. As part of this research preview, the code, weights, and research paper are now available. Additionally, today you can sign up for our waitlist to access a new upcoming web experience featuring a Text-To-Video interface. To access the model & sign up for our waitlist, visit our website here:
0:20

Today, we are releasing Stable Video Diffusion, our first foundation model for generative AI video based on the image model, Stable Diffusion. As part of this research preview, the code, weights, and research paper are now available. Additionally, today you can sign up for our waitlist to access a new upcoming web experience featuring a Text-To-Video interface. To access the model & sign up for our waitlist, visit our website here:

Stability AI

1,024,438 Aufrufe • vor 2 Jahren

Daily #Blender Tip: Learn how to create materials for games and films using PBR textures in Blender with this easy-to-follow tutorial video. PBR textures are a must-have for achieving realistic materials and this video will show you how to quickly set them up in Blender. #B3D
0:46

Daily #Blender Tip: Learn how to create materials for games and films using PBR textures in Blender with this easy-to-follow tutorial video. PBR textures are a must-have for achieving realistic materials and this video will show you how to quickly set them up in Blender. #B3D

Jan van den Hemel

40,821 Aufrufe • vor 3 Jahren

We are thrilled to release the next leap in art-grade 3D generative models. Our single-click model pipeline gives unprecedented mesh outputs, with mesh parts-based topology. It is available now for all Cube tiers to start for free. ☑️ Our multi-stage hierarchical AI models produce a fully assembled 3D mesh with adaptive poly-counts, providing the clean, separated topology you need. ✅ A parts-based approach enables high-resolution meshes. 🌐 Quad and Triangle mesh support. Access now:
1:02

We are thrilled to release the next leap in art-grade 3D generative models. Our single-click model pipeline gives unprecedented mesh outputs, with mesh parts-based topology. It is available now for all Cube tiers to start for free. ☑️ Our multi-stage hierarchical AI models produce a fully assembled 3D mesh with adaptive poly-counts, providing the clean, separated topology you need. ✅ A parts-based approach enables high-resolution meshes. 🌐 Quad and Triangle mesh support. Access now:

Common Sense Machines

420,323 Aufrufe • vor 11 Monaten

From Forger to Nomad the only thing Nomad was unable to import was the PBR Materials
0:20

From Forger to Nomad the only thing Nomad was unable to import was the PBR Materials

Juan Carlos Montes

16,654 Aufrufe • vor 2 Jahren

Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion abs: project page:
1:22

Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion abs: project page:

AK

262,963 Aufrufe • vor 3 Jahren