Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

🚀 Introducing DiffusionRenderer, a neural rendering engine powered by video diffusion models. 🎥 Estimates high-quality geometry and materials from videos, synthesizes photorealistic light transport, enables relighting and material editing with realistic shadows and reflections

Zian Wang

1,227 subscribers

125,484 views • 1 year ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Oh wow! "[SIGGRAPH '24] DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models" Paper: Project: Code (MIT) :

Oh wow! "[SIGGRAPH '24] DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models" Paper: Project: Code (MIT) :

MrNeRF

59,766 views • 2 years ago

"ROSE: Remove Objects with Side Effects in Videos" TL;DR: Diffusion transformer; five common cases: shadows, reflections, light, translucency and mirror as video side effects to remove

"ROSE: Remove Objects with Side Effects in Videos" TL;DR: Diffusion transformer; five common cases: shadows, reflections, light, translucency and mirror as video side effects to remove

Alexandre Morgand

85,858 views • 10 months ago

Introducing FLARE #CVPR2026 2025 FLARE is a feed-forward model that simultaneously estimates high-quality camera poses, 3D geometry, and appearance from sparse uncalibrated images. 1/4

Introducing FLARE #CVPR2026 2025 FLARE is a feed-forward model that simultaneously estimates high-quality camera poses, 3D geometry, and appearance from sparse uncalibrated images. 1/4

Gordon Wetzstein

29,518 views • 1 year ago

Light it your way — instantly. Introducing Relighting on Akool ✨ Turn any video into a professionally lit scene with just a few clicks. No studio. No expensive equipment. No reshoots. With Akool Relighting, you can: • Adjust lighting direction and mood • Enhance subject clarity and depth • Transform environments instantly • Create cinematic-quality visuals in seconds From bright daylight to dramatic studio lighting, all powered by AI. Create stunning visuals without the setup. That’s the power of Akool Relighting.

Light it your way — instantly. Introducing Relighting on Akool ✨ Turn any video into a professionally lit scene with just a few clicks. No studio. No expensive equipment. No reshoots. With Akool Relighting, you can: • Adjust lighting direction and mood • Enhance subject clarity and depth • Transform environments instantly • Create cinematic-quality visuals in seconds From bright daylight to dramatic studio lighting, all powered by AI. Create stunning visuals without the setup. That’s the power of Akool Relighting.

Akool Inc

4,193,352 views • 3 months ago

RTFM can be seen as a learned renderer: it is an autoregressive diffusion transformer trained end-to-end on large-scale video data, and it learns to model 3D geometry, reflections, shadows and more just by observing them in its training set.

RTFM can be seen as a learned renderer: it is an autoregressive diffusion transformer trained end-to-end on large-scale video data, and it learns to model 3D geometry, reflections, shadows and more just by observing them in its training set.

World Labs

15,870 views • 9 months ago

⚡️ NeuralAI Presents: World Engine Teaser 🌏 THE GAME ENGINE REIMAGINED Generate full scale maps, scenes, and entire worlds in collaboration mode with the World Engine, the ultimate AI-powered game engine! Real-time Unleashed Crisp Visuals at AAA Quality Real-time Asset Generation AI Agent Integration and Tokenisation ...And much more! Powered by $NEURAL Ready to create your world? 🚀 🧑‍🚀

⚡️ NeuralAI Presents: World Engine Teaser 🌏 THE GAME ENGINE REIMAGINED Generate full scale maps, scenes, and entire worlds in collaboration mode with the World Engine, the ultimate AI-powered game engine! Real-time Unleashed Crisp Visuals at AAA Quality Real-time Asset Generation AI Agent Integration and Tokenisation ...And much more! Powered by $NEURAL Ready to create your world? 🚀 🧑‍🚀

NeuralAI

99,117 views • 1 year ago

MaterialFusion Enhancing Inverse Rendering with Material Diffusion Priors discuss: Recent works in inverse rendering have shown promise in using multi-view images of an object to recover shape, albedo, and materials. However, the recovered components often fail to render accurately under new lighting conditions due to the intrinsic challenge of disentangling albedo and material properties from input images. To address this challenge, we introduce MaterialFusion, an enhanced conventional 3D inverse rendering pipeline that incorporates a 2D prior on texture and material properties. We present StableMaterial, a 2D diffusion model prior that refines multi-lit data to estimate the most likely albedo and material from given input appearances. This model is trained on albedo, material, and relit image data derived from a curated dataset of approximately ~12K artist-designed synthetic Blender objects called BlenderVault. we incorporate this diffusion prior with an inverse rendering framework where we use score distillation sampling (SDS) to guide the optimization of the albedo and materials, improving relighting performance in comparison with previous work. We validate MaterialFusion's relighting performance on 4 datasets of synthetic and real objects under diverse illumination conditions, showing our diffusion-aided approach significantly improves the appearance of reconstructed objects under novel lighting conditions. We intend to publicly release our BlenderVault dataset to support further research in this field.

MaterialFusion Enhancing Inverse Rendering with Material Diffusion Priors discuss: Recent works in inverse rendering have shown promise in using multi-view images of an object to recover shape, albedo, and materials. However, the recovered components often fail to render accurately under new lighting conditions due to the intrinsic challenge of disentangling albedo and material properties from input images. To address this challenge, we introduce MaterialFusion, an enhanced conventional 3D inverse rendering pipeline that incorporates a 2D prior on texture and material properties. We present StableMaterial, a 2D diffusion model prior that refines multi-lit data to estimate the most likely albedo and material from given input appearances. This model is trained on albedo, material, and relit image data derived from a curated dataset of approximately ~12K artist-designed synthetic Blender objects called BlenderVault. we incorporate this diffusion prior with an inverse rendering framework where we use score distillation sampling (SDS) to guide the optimization of the albedo and materials, improving relighting performance in comparison with previous work. We validate MaterialFusion's relighting performance on 4 datasets of synthetic and real objects under diverse illumination conditions, showing our diffusion-aided approach significantly improves the appearance of reconstructed objects under novel lighting conditions. We intend to publicly release our BlenderVault dataset to support further research in this field.

AK

22,959 views • 1 year ago

Introducing The Matrix --- a foundation world model for generating infinite-length, hyper-realistic videos with real-time, frame-level control: - Infinite-length video generation - 720p high-quality rendering - Real-time, frame-level control at 16 FPS - Generalization to real-world video control 🔗Blog: 📄Paper: 💻Code & Playable Demo: Coming soon! Key Innovation: A brand new technique called the shift-window denoise process model, enabling auto-regressive generation for diffusion and consistency models in real-time. Special thanks to project leader Ruili Feng and the entire Matrix team for their dedication and hard work over the year-long project.

Hongyang Zhang

178,322 views • 1 year ago

3D Gaussian Splats are cool, but they're static (Part 25). DG-Mesh can reconstruct high-quality and time-consistent 3D meshes from a single video and track mesh vertices over time, which enables texture editing on dynamic objects.

3D Gaussian Splats are cool, but they're static (Part 25). DG-Mesh can reconstruct high-quality and time-consistent 3D meshes from a single video and track mesh vertices over time, which enables texture editing on dynamic objects.

Dreaming Tulpa 🥓👑

79,033 views • 2 years ago

Exciting milestones in our generative AI research: Emu Video, which lets you create high quality videos from a text prompt, and Emu Edit, which enables detailed image editing based on your instructions. These new models are built on Emu, our foundation model for image generation and technology from them will underpin new creative features across our apps next year. Try it out: Emu Video: Emu Edit:

Exciting milestones in our generative AI research: Emu Video, which lets you create high quality videos from a text prompt, and Emu Edit, which enables detailed image editing based on your instructions. These new models are built on Emu, our foundation model for image generation and technology from them will underpin new creative features across our apps next year. Try it out: Emu Video: Emu Edit:

Boz

110,730 views • 2 years ago

🌟 Create anything in 3D! 🌟 Introducing CAT3D: a new method that generates high-fidelity 3D scenes from any number of real or generated images in one minute, powered by multi-view diffusion models. w/ lovely coauthors Aleksander Holynski, Ben Poole and an amazing team!

🌟 Create anything in 3D! 🌟 Introducing CAT3D: a new method that generates high-fidelity 3D scenes from any number of real or generated images in one minute, powered by multi-view diffusion models. w/ lovely coauthors Aleksander Holynski, Ben Poole and an amazing team!

Ruiqi Gao

153,196 views • 2 years ago

Introducing Dream Machine - a next generation video model for creating high quality, realistic shots from text instructions and images using AI. It’s available to everyone today! Try for free here #LumaDreamMachine

Introducing Dream Machine - a next generation video model for creating high quality, realistic shots from text instructions and images using AI. It’s available to everyone today! Try for free here #LumaDreamMachine

Luma

2,477,527 views • 2 years ago

Video Agent from HeyGen is live on Replicate! Turn a text prompt into a complete, polished video with AI-generated script, avatar presenter, voiceover, visuals, and editing. Plus, create realistic talking avatar videos with HeyGen's new Avatar IV engine. Video Translate has new "speed" and "precision" modes for more accurate translation lipsync.

Replicate

15,240 views • 4 months ago

Seed-Music A Unified Framework for High Quality and Controlled Music Generation discuss: We introduce Seed-Music, a suite of music generation systems capable of producing high-quality music with fine-grained style control. Our unified framework leverages both auto-regressive language modeling and diffusion approaches to support two key music creation workflows: controlled music generation and post-production editing. For controlled music generation, our system enables vocal music generation with performance controls from multi-modal inputs, including style descriptions, audio references, musical scores, and voice prompts. For post-production editing, it offers interactive tools for editing lyrics and vocal melodies directly in the generated audio.

Seed-Music A Unified Framework for High Quality and Controlled Music Generation discuss: We introduce Seed-Music, a suite of music generation systems capable of producing high-quality music with fine-grained style control. Our unified framework leverages both auto-regressive language modeling and diffusion approaches to support two key music creation workflows: controlled music generation and post-production editing. For controlled music generation, our system enables vocal music generation with performance controls from multi-modal inputs, including style descriptions, audio references, musical scores, and voice prompts. For post-production editing, it offers interactive tools for editing lyrics and vocal melodies directly in the generated audio.

AK

32,967 views • 1 year ago

$Spatial AI is increasingly important, and the newest papers from #NVIDIAResearch, 3DGRT and 3DGUT, represent significant advancements in enabling researchers and developers to explore and innovate with 3D Gaussian Splatting techniques. 💎 3DGRT (Gaussian Ray Tracing) ➡️ - Enables secondary lighting effects such as refractions, shadows, depth of field, and mirrors. - Provides a foundation for further research in ray tracing-based rendering of Gaussian representations. - Achieves real-time rendering performance on modern GPUs. ✨ 3DGUT (Gaussain Unscented Transform) ➡️ - Support for distorted camera models and time-dependent effects like rolling shutter. - Enables phenomena such as reflections and refractions while retaining real-time efficiency. 📥 Code:$

Spatial AI is increasingly important, and the newest papers from #NVIDIAResearch, 3DGRT and 3DGUT, represent significant advancements in enabling researchers and developers to explore and innovate with 3D Gaussian Splatting techniques. 💎 3DGRT (Gaussian Ray Tracing) ➡️ - Enables secondary lighting effects such as refractions, shadows, depth of field, and mirrors. - Provides a foundation for further research in ray tracing-based rendering of Gaussian representations. - Achieves real-time rendering performance on modern GPUs. ✨ 3DGUT (Gaussain Unscented Transform) ➡️ - Support for distorted camera models and time-dependent effects like rolling shutter. - Enables phenomena such as reflections and refractions while retaining real-time efficiency. 📥 Code:

NVIDIA AI Developer

27,760 views • 1 year ago

Introducing varg – Claude Code for video ads AI that saves $1M in ad testing budgets, made for bulk video ad generation It generates 1000s realistic videos with consistent characters and variations of hooks, voices, music and localizations. Powered by varg/sdk.

Introducing varg – Claude Code for video ads AI that saves $1M in ad testing budgets, made for bulk video ad generation It generates 1000s realistic videos with consistent characters and variations of hooks, voices, music and localizations. Powered by varg/sdk.

Alex Varga /✦

95,191 views • 4 months ago

📢📢📢Data release: high-res, multi-view, OLAT face recordings 📢📢📢 We captured individuals in our custom light stage with 16 high-end, global shutter cameras (72 fps) and 40 LED modules, totaling 2.8M precisely calibrated frames. We us the data for BecomingLit (#NeurIPS2025): intrinsically decomposed Gaussian avatars, enabling photorealistic and real-time relighting via hybrid neural shading. Code & Data: Great work by Jonathan Schmidt, Simon Giebenhain

📢📢📢Data release: high-res, multi-view, OLAT face recordings 📢📢📢 We captured individuals in our custom light stage with 16 high-end, global shutter cameras (72 fps) and 40 LED modules, totaling 2.8M precisely calibrated frames. We us the data for BecomingLit (#NeurIPS2025): intrinsically decomposed Gaussian avatars, enabling photorealistic and real-time relighting via hybrid neural shading. Code & Data: Great work by Jonathan Schmidt, Simon Giebenhain

Matthias Niessner

19,442 views • 6 months ago

📢 LinPrim: Linear Primitives for Differentiable Volumetric Rendering 📢 We use octahedra or tetrahedra as explicit as volumetric building blocks for gradient-based novel view synthesis - as an alternative to 3D Gaussians with discrete, bounded geometry. We show how it can be used to reconstruct photorealistic scenes, and introduce a corresponding differentiable CUDA rasterizer that enables real-time rendering. On real-world scenes, LinPrim achieves comparable image quality with fewer primitives, adding a practical polyhedral option to the 3D scene representation toolbox and expanding the known design space. 🌍 🎥 Amazing work by Nicolas von Lützow!

📢 LinPrim: Linear Primitives for Differentiable Volumetric Rendering 📢 We use octahedra or tetrahedra as explicit as volumetric building blocks for gradient-based novel view synthesis - as an alternative to 3D Gaussians with discrete, bounded geometry. We show how it can be used to reconstruct photorealistic scenes, and introduce a corresponding differentiable CUDA rasterizer that enables real-time rendering. On real-world scenes, LinPrim achieves comparable image quality with fewer primitives, adding a practical polyhedral option to the 3D scene representation toolbox and expanding the known design space. 🌍 🎥 Amazing work by Nicolas von Lützow!

Matthias Niessner

11,413 views • 1 year ago

We are thrilled to announce "Layered Diffusion Brushes": a real-time training-free image editor powered by diffusion models. 🎨✨ This is new work from my PhD student Peyman Gholami Peyman Gholami. Explore the interactive demo and check out more videos at:

We are thrilled to announce "Layered Diffusion Brushes": a real-time training-free image editor powered by diffusion models. 🎨✨ This is new work from my PhD student Peyman Gholami Peyman Gholami. Explore the interactive demo and check out more videos at:

Robert Xiao

14,334 views • 2 years ago

Introducing Ray2, a new frontier in video generative models. Scaled to 10x compute, #Ray2 creates realistic videos with natural and coherent motion, unlocking new freedoms of creative expression and visual storytelling. Available now. Learn more

Introducing Ray2, a new frontier in video generative models. Scaled to 10x compute, #Ray2 creates realistic videos with natural and coherent motion, unlocking new freedoms of creative expression and visual storytelling. Available now. Learn more

Luma

4,156,173 views • 1 year ago