Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

NVIDIA just showed some new relighting capabilities using gaussians, though this is pure ray tracing (no splatting). TRON extends their ray tracing library (3DGRT) to get an editable 3D material scaffold.

Radiance Fields

8,334 subscribers

28,421 views • 17 days ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

The biggest announcement from NVIDIA GTC was just released! It's the open sourcing of TWO Radiance Field methods, 3D Gaussian Ray Tracing and 3D Gaussian Unscented Transform from NVIDIA AI Developer. Not only has the code been released, but they come with an Apache 2.0 License, meaning you can use these methods commercially. This is the same license that Nerfstudio carries and is a big win for the community! 3DGRT introduces a pure ray tracing implementation, while 3DGUT brings the effects of ray tracing, such as secondary lighting effects to rasterization. This is a big deal! Article👇 Code👇

The biggest announcement from NVIDIA GTC was just released! It's the open sourcing of TWO Radiance Field methods, 3D Gaussian Ray Tracing and 3D Gaussian Unscented Transform from NVIDIA AI Developer. Not only has the code been released, but they come with an Apache 2.0 License, meaning you can use these methods commercially. This is the same license that Nerfstudio carries and is a big win for the community! 3DGRT introduces a pure ray tracing implementation, while 3DGUT brings the effects of ray tracing, such as secondary lighting effects to rasterization. This is a big deal! Article👇 Code👇

Radiance Fields

29,470 views • 1 year ago

⚡ Ray Tracing + 3D Gaussians = New Possibilities! Gaussian splatting is limited by rasterization—our #SIGGRAPHAsia2024 paper shows how to ray trace instead, enabling reflections, shadows, fisheye cameras, and more. The most important (and hardest!) part is making it fast. (1/N)

⚡ Ray Tracing + 3D Gaussians = New Possibilities! Gaussian splatting is limited by rasterization—our #SIGGRAPHAsia2024 paper shows how to ray trace instead, enabling reflections, shadows, fisheye cameras, and more. The most important (and hardest!) part is making it fast. (1/N)

Ashkan Mirzaei

98,213 views • 1 year ago

$Spatial AI is increasingly important, and the newest papers from #NVIDIAResearch, 3DGRT and 3DGUT, represent significant advancements in enabling researchers and developers to explore and innovate with 3D Gaussian Splatting techniques. 💎 3DGRT (Gaussian Ray Tracing) ➡️ - Enables secondary lighting effects such as refractions, shadows, depth of field, and mirrors. - Provides a foundation for further research in ray tracing-based rendering of Gaussian representations. - Achieves real-time rendering performance on modern GPUs. ✨ 3DGUT (Gaussain Unscented Transform) ➡️ - Support for distorted camera models and time-dependent effects like rolling shutter. - Enables phenomena such as reflections and refractions while retaining real-time efficiency. 📥 Code:$

Spatial AI is increasingly important, and the newest papers from #NVIDIAResearch, 3DGRT and 3DGUT, represent significant advancements in enabling researchers and developers to explore and innovate with 3D Gaussian Splatting techniques. 💎 3DGRT (Gaussian Ray Tracing) ➡️ - Enables secondary lighting effects such as refractions, shadows, depth of field, and mirrors. - Provides a foundation for further research in ray tracing-based rendering of Gaussian representations. - Achieves real-time rendering performance on modern GPUs. ✨ 3DGUT (Gaussain Unscented Transform) ➡️ - Support for distorted camera models and time-dependent effects like rolling shutter. - Enables phenomena such as reflections and refractions while retaining real-time efficiency. 📥 Code:

NVIDIA AI Developer

27,760 views • 1 year ago

Rise & shine for a new look at Nova Prospekt in Half-Life 2 RTX 👨‍🔬 Half-Life 2 RTX: An RTX Remix Project is in development by Orbifold Studios using NVIDIA RTX Remix to bring full ray tracing, DLSS 3.5 & remastered assets to this classic. Learn More:

Rise & shine for a new look at Nova Prospekt in Half-Life 2 RTX 👨‍🔬 Half-Life 2 RTX: An RTX Remix Project is in development by Orbifold Studios using NVIDIA RTX Remix to bring full ray tracing, DLSS 3.5 & remastered assets to this classic. Learn More:

NVIDIA GeForce

115,545 views • 1 year ago

Check out the latest Half-Life 2 RTX Ravenholm trailer. Half-Life 2 RTX: An RTX Remix Project is in development by Orbifold Studios using NVIDIA RTX Remix to bring full ray tracing, DLSS 3.5 + remastered assets. Learn more →

Check out the latest Half-Life 2 RTX Ravenholm trailer. Half-Life 2 RTX: An RTX Remix Project is in development by Orbifold Studios using NVIDIA RTX Remix to bring full ray tracing, DLSS 3.5 + remastered assets. Learn more →

NVIDIA GeForce

237,100 views • 2 years ago

An FPS from 2006 made guns feel more powerful than shooters with 4K graphics, ray tracing, and 200GB installs. That game was BLACK. No battle pass. No operator skins. No weapon charms. Just pure “this gun might bring the building down” energy.

An FPS from 2006 made guns feel more powerful than shooters with 4K graphics, ray tracing, and 200GB installs. That game was BLACK. No battle pass. No operator skins. No weapon charms. Just pure “this gun might bring the building down” energy.

Video Game Nostalgia

159,590 views • 1 month ago

In need of a new PC? You still have time to win this CyberPowerPC powered by an NVIDIA GeForce RTX 4080. Boost your gaming and creative workflow with advanced features like Ray Tracing, DLSS3, and GPU-accelerated encoders. Enter here 👉

In need of a new PC? You still have time to win this CyberPowerPC powered by an NVIDIA GeForce RTX 4080. Boost your gaming and creative workflow with advanced features like Ray Tracing, DLSS3, and GPU-accelerated encoders. Enter here 👉

PNY Pro

28,692 views • 3 years ago

Brute force Path tracing on RTX 5090. As an experiment this is using Ultra-performance (480p) upscaled to 1440p. No denoiser only DLSS-SR. Doing 120 paths per pixel at 50FPS. Here is the video. Thanks to discord user Perhapsy for doing the testing. This is not even efficient use of such high ray count

Brute force Path tracing on RTX 5090. As an experiment this is using Ultra-performance (480p) upscaled to 1440p. No denoiser only DLSS-SR. Doing 120 paths per pixel at 50FPS. Here is the video. Thanks to discord user Perhapsy for doing the testing. This is not even efficient use of such high ray count

Dark Souls Lighting Engine

28,807 views • 2 months ago

I created a 3D Gaussian Splat of my kitchen using 1/3rd the images it used to take me by using NVIDIA AI Developer's 3DGRUT! I can now use 180 degree fisheye images and ray tracing to make detailed splats. The only reason the scene isn't sharper is because my input images weren't super sharp - when I took the images back in October, I was still learning to use the lens. I plan to make a "first reactions/overview video". Tutorial after that. For reference, this took 206 images and the ultrawide on my iPhone took 608 images to capture. #3D #AEC #Computervision

I created a 3D Gaussian Splat of my kitchen using 1/3rd the images it used to take me by using NVIDIA AI Developer's 3DGRUT! I can now use 180 degree fisheye images and ray tracing to make detailed splats. The only reason the scene isn't sharper is because my input images weren't super sharp - when I took the images back in October, I was still learning to use the lens. I plan to make a "first reactions/overview video". Tutorial after that. For reference, this took 206 images and the ultrawide on my iPhone took 608 images to capture. #3D #AEC #Computervision

Jonathan Stephens

30,663 views • 1 year ago

The greatest Gaussian Splatting VR demo ever? Captured live at NVIDIA GTC — Mariya Sha experiencing 4D Gaussian Splatting in VR for the very first time. You can tell just from her reaction that this was something else! The scene was optimized using QUEEN, NVIDIA AI Developer's advanced research project for fast radiance field rendering. It’s an incredible look at what happens when high fidelity static and dynamic gaussian splats meets VR. Watch the full video on YouTube: Robert Scoble Did you get to do this one? #VR #3D #VFX

The greatest Gaussian Splatting VR demo ever? Captured live at NVIDIA GTC — Mariya Sha experiencing 4D Gaussian Splatting in VR for the very first time. You can tell just from her reaction that this was something else! The scene was optimized using QUEEN, NVIDIA AI Developer's advanced research project for fast radiance field rendering. It’s an incredible look at what happens when high fidelity static and dynamic gaussian splats meets VR. Watch the full video on YouTube: Robert Scoble Did you get to do this one? #VR #3D #VFX

Jonathan Stephens

28,114 views • 1 year ago

I tested out Cyberpunk 2077 Path Tracing turned on with RX 9070 XT now that FSR Redstone is out. By using the new FSR Redstone and FSR ML Based Frame-Gen, the game reaches up to 130FPS in 1440p with FSR set to Performance Mode. The performance gains are not that much higher compared to the previous versions of FSR but the image quality got a huge upgrade, especially when using Frame-Gen. This ML based Frame Gen is much better than the previous version as it gets rid of most artifacts and ghosting caused by the previous version. If for some reason you always wanted to play Cyberpunk 2077 with Path Tracing turned on on an AMD GPU, now you can enjoy a playable experience with 100+ FPS on an RX 9070 XT. I still don't recommend it though as the frame pacing and input lag still feels off for me, I'd just rather play this without Frame Gen and Path Tracing. I'm going to test out Black Myth: Wukong next

I tested out Cyberpunk 2077 Path Tracing turned on with RX 9070 XT now that FSR Redstone is out. By using the new FSR Redstone and FSR ML Based Frame-Gen, the game reaches up to 130FPS in 1440p with FSR set to Performance Mode. The performance gains are not that much higher compared to the previous versions of FSR but the image quality got a huge upgrade, especially when using Frame-Gen. This ML based Frame Gen is much better than the previous version as it gets rid of most artifacts and ghosting caused by the previous version. If for some reason you always wanted to play Cyberpunk 2077 with Path Tracing turned on on an AMD GPU, now you can enjoy a playable experience with 100+ FPS on an RX 9070 XT. I still don't recommend it though as the frame pacing and input lag still feels off for me, I'd just rather play this without Frame Gen and Path Tracing. I'm going to test out Black Myth: Wukong next

NikTek

242,749 views • 6 months ago

The new Indiana jones game is best looking game I’ve ever played The full path tracing mode is truly the first time I’ve been awestruck with ray tracing. Every room GI, every material’s speculars, every object on a shelf, every leaf and branch just looks… right! To me, it’s the first time I’ve felt like a game has taken a “next-gen” leap in visuals over the past decade or two honestly. Unlike hellblade II, this game is also really fun and charming, with incredibly well written and acted cutscenes, fantastic sound, and great gameplay. Way better than it has any right to be. Easily my GOTY

The new Indiana jones game is best looking game I’ve ever played The full path tracing mode is truly the first time I’ve been awestruck with ray tracing. Every room GI, every material’s speculars, every object on a shelf, every leaf and branch just looks… right! To me, it’s the first time I’ve felt like a game has taken a “next-gen” leap in visuals over the past decade or two honestly. Unlike hellblade II, this game is also really fun and charming, with incredibly well written and acted cutscenes, fantastic sound, and great gameplay. Way better than it has any right to be. Easily my GOTY

Azad Balabanian

34,477 views • 1 year ago

New Mark Cerny PS5 Pro Technical Seminar. Some interesting info. GPU is a hybrid of RDNA tech Base tech - RDNA2.X (RDNA2 + 3) Ray Tracing - RDNA4 Machine Learning - Custom Not fully RDNA3 to reduce workload for ports/code. Doesn't have RDNA3 shader executables and double floating point calculations, though it does have other RDNA3 and custom features such as extra vertex and primitive processing, more shader instructions for ML etc. Not having double FP is why Tflop count is 16.7 Tflop and not 33.5 Tflop, but Tflop count difference is "meaningless", esp to real world performance, and they didn't need 32bit calculation bump. Though interestingly PS5 Pro still retains 8 operations per lane for 67 Tflops of 16bit performance. Custom ML hardware has 300 TOPS of performance. PS5 Pro also has over 1GB of extra GDDR6 ram for games vs PS5, thanks to extra DDR5 system ram that many OS operations have been moved on to. Ray Tracing is roughly 2-3x faster than equivalent PS5 performance. ML hardware are PlayStation custom additions to RDNA that they themselves designed in collaboration with AMD. #PS5Pro #PS5 Full video below

New Mark Cerny PS5 Pro Technical Seminar. Some interesting info. GPU is a hybrid of RDNA tech Base tech - RDNA2.X (RDNA2 + 3) Ray Tracing - RDNA4 Machine Learning - Custom Not fully RDNA3 to reduce workload for ports/code. Doesn't have RDNA3 shader executables and double floating point calculations, though it does have other RDNA3 and custom features such as extra vertex and primitive processing, more shader instructions for ML etc. Not having double FP is why Tflop count is 16.7 Tflop and not 33.5 Tflop, but Tflop count difference is "meaningless", esp to real world performance, and they didn't need 32bit calculation bump. Though interestingly PS5 Pro still retains 8 operations per lane for 67 Tflops of 16bit performance. Custom ML hardware has 300 TOPS of performance. PS5 Pro also has over 1GB of extra GDDR6 ram for games vs PS5, thanks to extra DDR5 system ram that many OS operations have been moved on to. Ray Tracing is roughly 2-3x faster than equivalent PS5 performance. ML hardware are PlayStation custom additions to RDNA that they themselves designed in collaboration with AMD. #PS5Pro #PS5 Full video below

NIB

52,900 views • 1 year ago

Watch Dogs: Legion to me felt like going backwards. This is what killed the whole Watch Dogs franchise, even though Ubisoft implemented Ray Tracing with this game, everytime I play it I feel like it lacks any form of soul with no clear art direction. Watch Dogs 2 on the other hand which came out in 2016 feels much more alive and doesn't look like it has aged much at all. Could Ubisoft just improve on this instead of evolving backwards? They cooked with Watch Dogs 2 and then forgot their recipe with Watch Dogs: Legion, after that they decided to give up on working on this series that always had a huge potential.

Watch Dogs: Legion to me felt like going backwards. This is what killed the whole Watch Dogs franchise, even though Ubisoft implemented Ray Tracing with this game, everytime I play it I feel like it lacks any form of soul with no clear art direction. Watch Dogs 2 on the other hand which came out in 2016 feels much more alive and doesn't look like it has aged much at all. Could Ubisoft just improve on this instead of evolving backwards? They cooked with Watch Dogs 2 and then forgot their recipe with Watch Dogs: Legion, after that they decided to give up on working on this series that always had a huge potential.

NikTek

284,171 views • 5 months ago

3D scanning and rendering is moving so fast - got my splats up and running and I'm mind blown getting ~100fps for this complex 3D scene ⬇️ 🤯 1. WAY faster than NeRF: For comparison, NeRFs would takes around 10 seconds per frame (!) Instead I'm zipping around with FPV controls without breaking a sweat - though I do crash a few times towards the end of the video lol 2. Old Meets New: Gaussian Splatting is cool in that it fuses classical graphics and deep learning techniques. Like NeRFs, this is still a radiance field - just without the slower (ne)ural rendering part. 3. Explicit Representation: Instead you represent a 3D scene as a collection of ellipsoidal "splats" called gaussians. Each gaussian has a position, size, and color. Rendering in real-time is done by projecting into the image plane and alpha blending. 4. Photorealistic Effects: Gaussian splatting use spherical harmonics to represent the view-dependent effects and lighting - allowing surfaces to change color when viewed from different angles, enabling greater photorealism. It doesn't use a neural network, but the training loop is similar to deep learning. 5. Enables Direct Editing: But it's not just speed - with Gaussian Splatting you also get 3D editing support! So you can select, move, and delete stuff, even relight stuff. This type of editing has been more tedious to do with NeRFs and their implicit black box representations. 📲 More tests cooking! Much more to unpack here including simpler explanations. If you enjoyed this post, you might enjoy my feed: Bilawal Sidhu

3D scanning and rendering is moving so fast - got my splats up and running and I'm mind blown getting ~100fps for this complex 3D scene ⬇️ 🤯 1. WAY faster than NeRF: For comparison, NeRFs would takes around 10 seconds per frame (!) Instead I'm zipping around with FPV controls without breaking a sweat - though I do crash a few times towards the end of the video lol 2. Old Meets New: Gaussian Splatting is cool in that it fuses classical graphics and deep learning techniques. Like NeRFs, this is still a radiance field - just without the slower (ne)ural rendering part. 3. Explicit Representation: Instead you represent a 3D scene as a collection of ellipsoidal "splats" called gaussians. Each gaussian has a position, size, and color. Rendering in real-time is done by projecting into the image plane and alpha blending. 4. Photorealistic Effects: Gaussian splatting use spherical harmonics to represent the view-dependent effects and lighting - allowing surfaces to change color when viewed from different angles, enabling greater photorealism. It doesn't use a neural network, but the training loop is similar to deep learning. 5. Enables Direct Editing: But it's not just speed - with Gaussian Splatting you also get 3D editing support! So you can select, move, and delete stuff, even relight stuff. This type of editing has been more tedious to do with NeRFs and their implicit black box representations. 📲 More tests cooking! Much more to unpack here including simpler explanations. If you enjoyed this post, you might enjoy my feed: Bilawal Sidhu

Bilawal Sidhu

337,090 views • 2 years ago

Pushed another 2021 Unreal Engine experiment through LTX in ComfyUI! Not perfect, but look at those free waves! 🌊 Back in 2021, I was exploring the first version of Metahumans and ray tracing, trying to figure out how to generate water displacement. It’s crazy to see this extra layer of detail now on top of it. Although not all shots resulted in good waterflow direction . Combined a line-art + depth pre-processor to generate the shot guide. Also used these line-art generations in Nano Banana 2 to get the new first and last frames. Some shots are still tricky to get right if you want to keep control over the shot, especially the last shot with nothing in the end frame.

Pushed another 2021 Unreal Engine experiment through LTX in ComfyUI! Not perfect, but look at those free waves! 🌊 Back in 2021, I was exploring the first version of Metahumans and ray tracing, trying to figure out how to generate water displacement. It’s crazy to see this extra layer of detail now on top of it. Although not all shots resulted in good waterflow direction . Combined a line-art + depth pre-processor to generate the shot guide. Also used these line-art generations in Nano Banana 2 to get the new first and last frames. Some shots are still tricky to get right if you want to keep control over the shot, especially the last shot with nothing in the end frame.

Jerome | InsaneUnreal

20,275 views • 4 months ago

Here's everything we know about Nvidia's "greatest leap in graphics since real-time ray tracing" You can see Digital Foundry's jaw drop in this reaction after they just saw DLSS 5.0: - Will ship in Fall of 2026! - Demo ran 4k on 2 5090's but is already running on single GPU in their labs. - Will run and ship on single GPUs. - Doesn't change the scene or geo, just relights it super accurately. - Nvidia say they look at this as trying to anticipate the original, fully realized vision of the game. - Works on environments, not just characters. - Todd Howard has signed off on it, saying it's how he wanted Starfield to be seen. - Developer can tune DLSS 5 lighting. - You can apply to games as primitive as Minecraft, it's not just for high end games. - Developers are already on board. - Should scale with output resolutions. "I don't think I've seen a demo as astonishing as this..."

Here's everything we know about Nvidia's "greatest leap in graphics since real-time ray tracing" You can see Digital Foundry's jaw drop in this reaction after they just saw DLSS 5.0: - Will ship in Fall of 2026! - Demo ran 4k on 2 5090's but is already running on single GPU in their labs. - Will run and ship on single GPUs. - Doesn't change the scene or geo, just relights it super accurately. - Nvidia say they look at this as trying to anticipate the original, fully realized vision of the game. - Works on environments, not just characters. - Todd Howard has signed off on it, saying it's how he wanted Starfield to be seen. - Developer can tune DLSS 5 lighting. - You can apply to games as primitive as Minecraft, it's not just for high end games. - Developers are already on board. - Should scale with output resolutions. "I don't think I've seen a demo as astonishing as this..."

Grummz

305,555 views • 4 months ago

This has to be fake. It has to be either an actor or some weird schismatic sect not under the Catholic Church. Either way, you see how they always attack the Catholic church? No way a real Catholic Church let this random OF chick in dressed like this. I'm tired of society and the degeneracy. This is what you get when you platform ex-OF chicks like Nala Ray and bring in a new media attention avenue for the worst of society to exploit for views and attention.

Sensitive content

This has to be fake. It has to be either an actor or some weird schismatic sect not under the Catholic Church. Either way, you see how they always attack the Catholic church? No way a real Catholic Church let this random OF chick in dressed like this. I'm tired of society and the degeneracy. This is what you get when you platform ex-OF chicks like Nala Ray and bring in a new media attention avenue for the worst of society to exploit for views and attention.

Ryan Lee

19,501 views • 2 months ago

MaterialFusion Enhancing Inverse Rendering with Material Diffusion Priors discuss: Recent works in inverse rendering have shown promise in using multi-view images of an object to recover shape, albedo, and materials. However, the recovered components often fail to render accurately under new lighting conditions due to the intrinsic challenge of disentangling albedo and material properties from input images. To address this challenge, we introduce MaterialFusion, an enhanced conventional 3D inverse rendering pipeline that incorporates a 2D prior on texture and material properties. We present StableMaterial, a 2D diffusion model prior that refines multi-lit data to estimate the most likely albedo and material from given input appearances. This model is trained on albedo, material, and relit image data derived from a curated dataset of approximately ~12K artist-designed synthetic Blender objects called BlenderVault. we incorporate this diffusion prior with an inverse rendering framework where we use score distillation sampling (SDS) to guide the optimization of the albedo and materials, improving relighting performance in comparison with previous work. We validate MaterialFusion's relighting performance on 4 datasets of synthetic and real objects under diverse illumination conditions, showing our diffusion-aided approach significantly improves the appearance of reconstructed objects under novel lighting conditions. We intend to publicly release our BlenderVault dataset to support further research in this field.

MaterialFusion Enhancing Inverse Rendering with Material Diffusion Priors discuss: Recent works in inverse rendering have shown promise in using multi-view images of an object to recover shape, albedo, and materials. However, the recovered components often fail to render accurately under new lighting conditions due to the intrinsic challenge of disentangling albedo and material properties from input images. To address this challenge, we introduce MaterialFusion, an enhanced conventional 3D inverse rendering pipeline that incorporates a 2D prior on texture and material properties. We present StableMaterial, a 2D diffusion model prior that refines multi-lit data to estimate the most likely albedo and material from given input appearances. This model is trained on albedo, material, and relit image data derived from a curated dataset of approximately ~12K artist-designed synthetic Blender objects called BlenderVault. we incorporate this diffusion prior with an inverse rendering framework where we use score distillation sampling (SDS) to guide the optimization of the albedo and materials, improving relighting performance in comparison with previous work. We validate MaterialFusion's relighting performance on 4 datasets of synthetic and real objects under diverse illumination conditions, showing our diffusion-aided approach significantly improves the appearance of reconstructed objects under novel lighting conditions. We intend to publicly release our BlenderVault dataset to support further research in this field.

AK

22,959 views • 1 year ago