Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Excited to share 🎨🖌️ 3D Paintbrush - a method for generating local stylized textures on meshes using text as input! Our method predicts a localization map & a highly detailed texture map which conforms to it (1/3)

Rana Hanocka

6,257 subscribers

49,056 views • 3 years ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

9 Comments

Rana Hanocka3 years ago

3D Paintbrush produces textures that effectively adhere to the localizations. This enables seamlessly compositing local textures without any unwanted fringes! (2/3)

Rana Hanocka3 years ago

Key to our method is cascaded score distillation (CSD): which simultaneously distills scores at multiple resolutions. Standard SDS only uses the first low-res stage. Incorporating the super-res cascaded stage from our CSD increases the resolution and detail! (3/3)

Rana Hanocka2 years ago

3D Paintbrush was led by 3DL PhD student @DecaturDale 🚀. Other co-authors are: 3DL postdoc @ItaiLang and Snap Researcher @AbermanKfir (/4)

Keenan Crane3 years ago

You’ve made Spot very happy. 🐮

Rana Hanocka3 years ago

Our whole lab is obsessed with spot! 🐄❤️ Thanks for making him! Bob also made an appearance in this paper 🐥🙂

Nitin Agarwal3 years ago

Really nice work! I wonder how far we are from conditioned (text/image) geometric editing as well.

Nikhila Ravi3 years ago

Wow!! So cool! 🤩

Daniel3 years ago

Awesome

Tariq Hussain3 years ago

wow amazing work

Related Videos

Nvidia presents Edify 3D! The method can generate high-quality 3D assets from text descriptions. It uses a diffusion model to create detailed quad-mesh topologies and high-resolution textures in under 2 minutes.

Nvidia presents Edify 3D! The method can generate high-quality 3D assets from text descriptions. It uses a diffusion model to create detailed quad-mesh topologies and high-resolution textures in under 2 minutes.

Dreaming Tulpa 🥓👑

39,523 views • 2 years ago

(1/4) Excited to share our #ICCV2023 paper Text2Room! We generate scene-scale textured 3D meshes from a given text prompt leveraging 2D text-to-image models such as StableDiffusion. Project: Code: Video:

(1/4) Excited to share our #ICCV2023 paper Text2Room! We generate scene-scale textured 3D meshes from a given text prompt leveraging 2D text-to-image models such as StableDiffusion. Project: Code: Video:

Matthias Niessner

74,907 views • 3 years ago

Excited to share our work on Neural Assets: a new method for enabling 3D asset-level control in image diffusion models – scalable & without any 3D inductive biases. Neural Assets goes beyond text or pixel-based control & provides an interface inspired by 3D graphics tools. 🧵

Excited to share our work on Neural Assets: a new method for enabling 3D asset-level control in image diffusion models – scalable & without any 3D inductive biases. Neural Assets goes beyond text or pixel-based control & provides an interface inspired by 3D graphics tools. 🧵

Thomas Kipf

98,036 views • 2 years ago

A new EEG microdisplay for #neurosurgery conforms to the brain and employs colorful light patterns to visualize brain activity in large animals, potentially providing a more reliable method for surgeons to map and avoid functional brain tissue. Science Translational Medicine

A new EEG microdisplay for #neurosurgery conforms to the brain and employs colorful light patterns to visualize brain activity in large animals, potentially providing a more reliable method for surgeons to map and avoid functional brain tissue. Science Translational Medicine

Science Magazine

68,616 views • 2 years ago

Check out our fun work on robust view synthesis! 🤩 Using a casual video as input, our method jointly 1⃣estimates accurate camera poses and 2⃣reconstructs multiple local radiance fields for a large scene. Project🕸️:

Check out our fun work on robust view synthesis! 🤩 Using a casual video as input, our method jointly 1⃣estimates accurate camera poses and 2⃣reconstructs multiple local radiance fields for a large scene. Project🕸️:

Jia-Bin Huang

35,538 views • 3 years ago

Thrilled to share our new work on Reconviagen✨! A key challenge in 3D creation is the alignment of generative 3D with observational input. Our method solves this by grounding the generative process in 3D reconstruction. Try it at: #Reconstruction #AIGC

Thrilled to share our new work on Reconviagen✨! A key challenge in 3D creation is the alignment of generative 3D with observational input. Our method solves this by grounding the generative process in 3D reconstruction. Try it at: #Reconstruction #AIGC

Chongjie Ye

11,624 views • 1 year ago

🖌️ Painting a flat grey model and not sure where to start? Kickstart your stylized texture painting in Toolbag with a baked AO map, block in values with overlays, then quickly block colors in using gradient remap layers 🎨 This lets you nail form first, and add color later.

🖌️ Painting a flat grey model and not sure where to start? Kickstart your stylized texture painting in Toolbag with a baked AO map, block in values with overlays, then quickly block colors in using gradient remap layers 🎨 This lets you nail form first, and add color later.

Marmoset

13,182 views • 8 months ago

There’s a method to it 🎨

There’s a method to it 🎨

UCF Football

10,635 views • 8 months ago

I’m trying to re-texture my avatar in #EmoteLab it’s very cute! > Texture Cache > CoffeeBean > Custom Textures - Go back to EmoteLab and click Reload Textures. Method 2: Easier Way to Edit Textures Directly - Go to Settings → Open Character Folder. - Find the version you are using, then open your character folder. - Click Regions. - You will find the textures used by the character there. You can edit these textures directly as well. 💡Tip: I recommend editing the WeightTransfer folder first, then Click Unload Current Model and Load the model again to refresh the textures. Finally, You will get your own EmoteLab texture!!

B U N R I👾 @Live2D

23,821 views • 8 months ago

Some of you may recall a Reddit post from few years ago, which demonstrated the potential for custom 3D enviroments during battles. Many moons later, discord user Shy presented a general method to convert map assets into battle backgrounds....

Some of you may recall a Reddit post from few years ago, which demonstrated the potential for custom 3D enviroments during battles. Many moons later, discord user Shy presented a general method to convert map assets into battle backgrounds....

AphexCubed

185,244 views • 8 months ago

🚨 TEXT TO 3D WORKS! 🚨 Excited to share a video on our text-to-3D (at runtime) AI engine powering the AI game Geppetto’s Workshop ! #AIart #gamedev #indiedev #GenerativeAI

🚨 TEXT TO 3D WORKS! 🚨 Excited to share a video on our text-to-3D (at runtime) AI engine powering the AI game Geppetto’s Workshop ! #AIart #gamedev #indiedev #GenerativeAI

Pillars Films

299,072 views • 3 years ago

Google presents VLOGGER Multimodal Diffusion for Embodied Avatar Synthesis We propose VLOGGER, a method for audio-driven human video generation from a single input image of a person, which builds on the success of recent generative diffusion models. Our method consists of

AK

66,375 views • 2 years ago

Using ImGui's font atlas to render text into an RGBA texture, then mapping it to a simple quad for in-game 3D text. Super useful for debugging and prototyping.

Using ImGui's font atlas to render text into an RGBA texture, then mapping it to a simple quad for in-game 3D text. Super useful for debugging and prototyping.

Ioannis Koukourakis

12,443 views • 10 months ago

Some of the raw map animation I made for a recent conference Ito were presenting at. The Miami city model was really detailed and it was fun finding ways to simulate disruptions on lines. #3d #map #animation

Some of the raw map animation I made for a recent conference Ito were presenting at. The Miami city model was really detailed and it was fun finding ways to simulate disruptions on lines. #3d #map #animation

Craig Taylor

30,648 views • 3 years ago

One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization paper page: Single image 3D reconstruction is an important but challenging task that requires extensive knowledge of our natural world. Many existing methods solve this problem by optimizing a neural radiance field under the guidance of 2D diffusion models but suffer from lengthy optimization time, 3D inconsistency results, and poor geometry. In this work, we propose a novel method that takes a single image of any object as input and generates a full 360-degree 3D textured mesh in a single feed-forward pass. Given a single image, we first use a view-conditioned 2D diffusion model, Zero123, to generate multi-view images for the input view, and then aim to lift them up to 3D space. Since traditional reconstruction methods struggle with inconsistent multi-view predictions, we build our 3D reconstruction module upon an SDF-based generalizable neural surface reconstruction method and propose several critical training strategies to enable the reconstruction of 360-degree meshes. Without costly optimizations, our method reconstructs 3D shapes in significantly less time than existing methods. Moreover, our method favors better geometry, generates more 3D consistent results, and adheres more closely to the input image. We evaluate our approach on both synthetic data and in-the-wild images and demonstrate its superiority in terms of both mesh quality and runtime. In addition, our approach can seamlessly support the text-to-3D task by integrating with off-the-shelf text-to-image diffusion models.

One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization paper page: Single image 3D reconstruction is an important but challenging task that requires extensive knowledge of our natural world. Many existing methods solve this problem by optimizing a neural radiance field under the guidance of 2D diffusion models but suffer from lengthy optimization time, 3D inconsistency results, and poor geometry. In this work, we propose a novel method that takes a single image of any object as input and generates a full 360-degree 3D textured mesh in a single feed-forward pass. Given a single image, we first use a view-conditioned 2D diffusion model, Zero123, to generate multi-view images for the input view, and then aim to lift them up to 3D space. Since traditional reconstruction methods struggle with inconsistent multi-view predictions, we build our 3D reconstruction module upon an SDF-based generalizable neural surface reconstruction method and propose several critical training strategies to enable the reconstruction of 360-degree meshes. Without costly optimizations, our method reconstructs 3D shapes in significantly less time than existing methods. Moreover, our method favors better geometry, generates more 3D consistent results, and adheres more closely to the input image. We evaluate our approach on both synthetic data and in-the-wild images and demonstrate its superiority in terms of both mesh quality and runtime. In addition, our approach can seamlessly support the text-to-3D task by integrating with off-the-shelf text-to-image diffusion models.

AK

179,061 views • 3 years ago

Virtual textures in the browser. The whole of World of Warcraft map on a single texture. WebGL 2.0 implementation of virtual textures in meep writeup:

Virtual textures in the browser. The whole of World of Warcraft map on a single texture. WebGL 2.0 implementation of virtual textures in meep writeup:

Alex Goldring

146,489 views • 10 months ago

🚀High-Res 3D Text-to-Texture🚀 🎨#MVPaint delivers high-res 3D textures with minimal UV wrapping dependency and high versatility using synchronized multi-view diffusion 📑 Paper: 🏠 Project Page: 🧑‍💻 Code:

🚀High-Res 3D Text-to-Texture🚀 🎨#MVPaint delivers high-res 3D textures with minimal UV wrapping dependency and high versatility using synchronized multi-view diffusion 📑 Paper: 🏠 Project Page: 🧑‍💻 Code:

Ziwei Liu

28,184 views • 2 years ago

I'm excited to share our new work Diffusion as Shader (DaS), a versatile controllable video generation method for various tasks: object manipulation, camera control, mesh-to-video, and motion transfer. Project page: Github:

I'm excited to share our new work Diffusion as Shader (DaS), a versatile controllable video generation method for various tasks: object manipulation, camera control, mesh-to-video, and motion transfer. Project page: Github:

Yuan Liu

33,490 views • 1 year ago