Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Learn how the Radiance Field method, Gaussian Splatting gets compressed with @wimost and Florian Barthel's Self Organizing Gaussians. Self Organizing Gaussians is also what nerfstudio uses for gsplat. Full episode is now live here:

View Dependent

1,019 subscribers

16,630 views • 1 year ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

📢 SHeaP: Self-Supervised Head Predictor Learned via 2D Gaussians 📢 Given a single input image, we predict accurate 3D head geometry, pose, and expression. Previous works (e.g. DECA, EMOCA) use differentiable mesh rasterization to learn a self-supervised head geometry predictor via a photometric reconstruction loss. We borrow these ideas, but our key insight is to replace the mesh rendering with 2D Gaussian Splatting. This leads to much higher accuracy of the underlying predicted geometry and thus more gradient signal during training. 🌍 🎥 Great work by Liam Schoneveld Davide Davoli Jiapeng Tang

📢 SHeaP: Self-Supervised Head Predictor Learned via 2D Gaussians 📢 Given a single input image, we predict accurate 3D head geometry, pose, and expression. Previous works (e.g. DECA, EMOCA) use differentiable mesh rasterization to learn a self-supervised head geometry predictor via a photometric reconstruction loss. We borrow these ideas, but our key insight is to replace the mesh rendering with 2D Gaussian Splatting. This leads to much higher accuracy of the underlying predicted geometry and thus more gradient signal during training. 🌍 🎥 Great work by Liam Schoneveld Davide Davoli Jiapeng Tang

Matthias Niessner

28,559 views • 1 year ago

Nvidia announces GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning paper page: Gaussian splatting has emerged as a powerful 3D representation that harnesses the advantages of both explicit (mesh) and implicit (NeRF) 3D representations. In this paper, we seek to leverage Gaussian splatting to generate realistic animatable avatars from textual descriptions, addressing the limitations (e.g., flexibility and efficiency) imposed by mesh or NeRF-based representations. However, a naive application of Gaussian splatting cannot generate high-quality animatable avatars and suffers from learning instability; it also cannot capture fine avatar geometries and often leads to degenerate body parts. To tackle these problems, we first propose a primitive-based 3D Gaussian representation where Gaussians are defined inside pose-driven primitives to facilitate animation. Second, to stabilize and amortize the learning of millions of Gaussians, we propose to use neural implicit fields to predict the Gaussian attributes (e.g., colors). Finally, to capture fine avatar geometries and extract detailed meshes, we propose a novel SDF-based implicit mesh learning approach for 3D Gaussians that regularizes the underlying geometries and extracts highly detailed textured meshes. Our proposed method, GAvatar, enables the large-scale generation of diverse animatable avatars using only text prompts. GAvatar significantly surpasses existing methods in terms of both appearance and geometry quality, and achieves extremely fast rendering (100 fps) at 1K resolution.

AK

140,992 views • 2 years ago

The greatest Gaussian Splatting VR demo ever? Captured live at NVIDIA GTC — Mariya Sha experiencing 4D Gaussian Splatting in VR for the very first time. You can tell just from her reaction that this was something else! The scene was optimized using QUEEN, NVIDIA AI Developer's advanced research project for fast radiance field rendering. It’s an incredible look at what happens when high fidelity static and dynamic gaussian splats meets VR. Watch the full video on YouTube: Robert Scoble Did you get to do this one? #VR #3D #VFX

The greatest Gaussian Splatting VR demo ever? Captured live at NVIDIA GTC — Mariya Sha experiencing 4D Gaussian Splatting in VR for the very first time. You can tell just from her reaction that this was something else! The scene was optimized using QUEEN, NVIDIA AI Developer's advanced research project for fast radiance field rendering. It’s an incredible look at what happens when high fidelity static and dynamic gaussian splats meets VR. Watch the full video on YouTube: Robert Scoble Did you get to do this one? #VR #3D #VFX

Jonathan Stephens

28,114 views • 1 year ago

She’s 93. She bought a Tesla Model Y with Full Self-Driving because she no longer felt comfortable driving herself. Now it takes her to church and she uses Grok for navigation. This is what the technology is for.

She’s 93. She bought a Tesla Model Y with Full Self-Driving because she no longer felt comfortable driving herself. Now it takes her to church and she uses Grok for navigation. This is what the technology is for.

Sovey

6,268,170 views • 3 months ago

GS^3: Efficient Relighting with Triple Gaussian Splatting Abstract: We present a spatial and angular Gaussian based representation and a triple splatting process, for real-time, high-quality novel lighting-and-view synthesis from multi-view point-lit input images. To describe complex ap pearance, we employ a Lambertian plus a mixture of angular Gaussians as an effective reflectance function for each spatial Gaussian. To generate self-shadow, we splat all spatial Gaussians towards the light source to obtain shadow values, which are further refined by a small multi-layer perceptron. To compensate for other effects like global illumination, another network is trained to compute and add a per-spatial-Gaussian RGB tuple. The effectiveness of our representation is demonstrated on 30 samples with a wide variation in geometry (from solid to fluffy) and appearance (from translucent to anisotropic), as well as using different forms of input data, including rendered images of synthetic/reconstructed objects, photographs captured with a handheld camera and a flash, or from a professional lightstage. We achieve a training time of 40-70 minutes and a rendering speed of 90 fps on a single commodity GPU. Our results compare favorably with state-of-the-art techniques in terms of quality/performance.

GS^3: Efficient Relighting with Triple Gaussian Splatting Abstract: We present a spatial and angular Gaussian based representation and a triple splatting process, for real-time, high-quality novel lighting-and-view synthesis from multi-view point-lit input images. To describe complex ap pearance, we employ a Lambertian plus a mixture of angular Gaussians as an effective reflectance function for each spatial Gaussian. To generate self-shadow, we splat all spatial Gaussians towards the light source to obtain shadow values, which are further refined by a small multi-layer perceptron. To compensate for other effects like global illumination, another network is trained to compute and add a per-spatial-Gaussian RGB tuple. The effectiveness of our representation is demonstrated on 30 samples with a wide variation in geometry (from solid to fluffy) and appearance (from translucent to anisotropic), as well as using different forms of input data, including rendered images of synthetic/reconstructed objects, photographs captured with a handheld camera and a flash, or from a professional lightstage. We achieve a training time of 40-70 minutes and a rendering speed of 90 fps on a single commodity GPU. Our results compare favorably with state-of-the-art techniques in terms of quality/performance.

MrNeRF

17,786 views • 1 year ago

New Episode of Multichain Nation is live 🌟 For Episode 18, we caught up with Taejeong Kang who placed 1st in the Product track at the XRPL Korea hackathon 🥇 Watch the full episode to learn about Kang's product xyz and how it uses Axelar Network 👇

New Episode of Multichain Nation is live 🌟 For Episode 18, we caught up with Taejeong Kang who placed 1st in the Product track at the XRPL Korea hackathon 🥇 Watch the full episode to learn about Kang's product xyz and how it uses Axelar Network 👇

Multichain Nation

26,841 views • 1 year ago

Everybody is stepping into the LOD game, and so is LichtFeld Studio. Preview of 260M Gaussians streaming into the viewer live. I use the RAD format which is processed on GPU within LichtFeld. It can be also simply dumped straight into the spark.js web viewer, albeit it will die at that amount of Gaussians. What other solutions don't tell you is that they need hours to preprocess a 3DGS ply to make it streamable. This was just a ply exported to RAD by LichtFeld Studio's convert tool (took 5 min at that size) and it is immediately ready to stream. In the comments there is a smaller dataset with 103M Gaussians that streams on startup into the viewer. Both datasets were created by Andrii Shramko. Let's see how far I can push this (need bigger datasets)

Everybody is stepping into the LOD game, and so is LichtFeld Studio. Preview of 260M Gaussians streaming into the viewer live. I use the RAD format which is processed on GPU within LichtFeld. It can be also simply dumped straight into the spark.js web viewer, albeit it will die at that amount of Gaussians. What other solutions don't tell you is that they need hours to preprocess a 3DGS ply to make it streamable. This was just a ply exported to RAD by LichtFeld Studio's convert tool (took 5 min at that size) and it is immediately ready to stream. In the comments there is a smaller dataset with 103M Gaussians that streams on startup into the viewer. Both datasets were created by Andrii Shramko. Let's see how far I can push this (need bigger datasets)

MrNeRF

23,651 views • 1 month ago

🚀 Major upgrades just landed in SuperSplat, the free and open source platform for 3D Gaussian splatting. Here is a 24-MILLION-Gaussian scan streaming live to a web browser. Near instant load time. Solid 60 fps. How? A new compute-based WebGPU renderer + automatic LOD streaming. 🧵

🚀 Major upgrades just landed in SuperSplat, the free and open source platform for 3D Gaussian splatting. Here is a 24-MILLION-Gaussian scan streaming live to a web browser. Near instant load time. Solid 60 fps. How? A new compute-based WebGPU renderer + automatic LOD streaming. 🧵

PlayCanvas

52,817 views • 2 months ago

Here it is! Learn how to build real-time games with...Laravel! Watch the first episode right here, then head over to Laracasts for the full course.

Here it is! Learn how to build real-time games with...Laravel! Watch the first episode right here, then head over to Laracasts for the full course.

Laracasts

28,901 views • 2 years ago

3D scanning and rendering is moving so fast - got my splats up and running and I'm mind blown getting ~100fps for this complex 3D scene ⬇️ 🤯 1. WAY faster than NeRF: For comparison, NeRFs would takes around 10 seconds per frame (!) Instead I'm zipping around with FPV controls without breaking a sweat - though I do crash a few times towards the end of the video lol 2. Old Meets New: Gaussian Splatting is cool in that it fuses classical graphics and deep learning techniques. Like NeRFs, this is still a radiance field - just without the slower (ne)ural rendering part. 3. Explicit Representation: Instead you represent a 3D scene as a collection of ellipsoidal "splats" called gaussians. Each gaussian has a position, size, and color. Rendering in real-time is done by projecting into the image plane and alpha blending. 4. Photorealistic Effects: Gaussian splatting use spherical harmonics to represent the view-dependent effects and lighting - allowing surfaces to change color when viewed from different angles, enabling greater photorealism. It doesn't use a neural network, but the training loop is similar to deep learning. 5. Enables Direct Editing: But it's not just speed - with Gaussian Splatting you also get 3D editing support! So you can select, move, and delete stuff, even relight stuff. This type of editing has been more tedious to do with NeRFs and their implicit black box representations. 📲 More tests cooking! Much more to unpack here including simpler explanations. If you enjoyed this post, you might enjoy my feed: Bilawal Sidhu

3D scanning and rendering is moving so fast - got my splats up and running and I'm mind blown getting ~100fps for this complex 3D scene ⬇️ 🤯 1. WAY faster than NeRF: For comparison, NeRFs would takes around 10 seconds per frame (!) Instead I'm zipping around with FPV controls without breaking a sweat - though I do crash a few times towards the end of the video lol 2. Old Meets New: Gaussian Splatting is cool in that it fuses classical graphics and deep learning techniques. Like NeRFs, this is still a radiance field - just without the slower (ne)ural rendering part. 3. Explicit Representation: Instead you represent a 3D scene as a collection of ellipsoidal "splats" called gaussians. Each gaussian has a position, size, and color. Rendering in real-time is done by projecting into the image plane and alpha blending. 4. Photorealistic Effects: Gaussian splatting use spherical harmonics to represent the view-dependent effects and lighting - allowing surfaces to change color when viewed from different angles, enabling greater photorealism. It doesn't use a neural network, but the training loop is similar to deep learning. 5. Enables Direct Editing: But it's not just speed - with Gaussian Splatting you also get 3D editing support! So you can select, move, and delete stuff, even relight stuff. This type of editing has been more tedious to do with NeRFs and their implicit black box representations. 📲 More tests cooking! Much more to unpack here including simpler explanations. If you enjoyed this post, you might enjoy my feed: Bilawal Sidhu

Bilawal Sidhu

337,090 views • 2 years ago

🦵Self-Tying The Legs🦵 Looking to explore self-tying? With very few knots in spots that are difficult to reach, practicing leg ties is a great way to get a feel for rope and begin to learn without a model. Watch the full three-part series here! ➡️

Sensitive content

🦵Self-Tying The Legs🦵 Looking to explore self-tying? With very few knots in spots that are difficult to reach, practicing leg ties is a great way to get a feel for rope and begin to learn without a model. Watch the full three-part series here! ➡️

Shibari Study

21,547 views • 2 years ago

The biggest announcement from NVIDIA GTC was just released! It's the open sourcing of TWO Radiance Field methods, 3D Gaussian Ray Tracing and 3D Gaussian Unscented Transform from NVIDIA AI Developer. Not only has the code been released, but they come with an Apache 2.0 License, meaning you can use these methods commercially. This is the same license that Nerfstudio carries and is a big win for the community! 3DGRT introduces a pure ray tracing implementation, while 3DGUT brings the effects of ray tracing, such as secondary lighting effects to rasterization. This is a big deal! Article👇 Code👇

The biggest announcement from NVIDIA GTC was just released! It's the open sourcing of TWO Radiance Field methods, 3D Gaussian Ray Tracing and 3D Gaussian Unscented Transform from NVIDIA AI Developer. Not only has the code been released, but they come with an Apache 2.0 License, meaning you can use these methods commercially. This is the same license that Nerfstudio carries and is a big win for the community! 3DGRT introduces a pure ray tracing implementation, while 3DGUT brings the effects of ray tracing, such as secondary lighting effects to rasterization. This is a big deal! Article👇 Code👇

Radiance Fields

29,483 views • 1 year ago

Ok, last leak for now, here is ROCKETESS' full intro! I wonder how she gets on with Bandit and Wizard... 👀

Ok, last leak for now, here is ROCKETESS' full intro! I wonder how she gets on with Bandit and Wizard... 👀

Seeker

20,124 views • 1 month ago

Drivable 3D Gaussian Avatars paper page: present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats. Current photorealistic drivable avatars require either accurate 3D registrations during training, dense input images during testing, or both. The ones based on neural radiance fields also tend to be prohibitively slow for telepresence applications. This work uses the recently presented 3D Gaussian Splatting (3DGS) technique to render realistic humans at real-time framerates, using dense calibrated multi-view videos as input. To deform those primitives, we depart from the commonly used point deformation method of linear blend skinning (LBS) and use a classic volumetric deformation method: cage deformations. Given their smaller size, we drive these deformations with joint angles and keypoints, which are more suitable for communication applications. Our experiments on nine subjects with varied body shapes, clothes, and motions obtain higher-quality results than state-of-the-art methods when using the same training and test data.

Drivable 3D Gaussian Avatars paper page: present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats. Current photorealistic drivable avatars require either accurate 3D registrations during training, dense input images during testing, or both. The ones based on neural radiance fields also tend to be prohibitively slow for telepresence applications. This work uses the recently presented 3D Gaussian Splatting (3DGS) technique to render realistic humans at real-time framerates, using dense calibrated multi-view videos as input. To deform those primitives, we depart from the commonly used point deformation method of linear blend skinning (LBS) and use a classic volumetric deformation method: cage deformations. Given their smaller size, we drive these deformations with joint angles and keypoints, which are more suitable for communication applications. Our experiments on nine subjects with varied body shapes, clothes, and motions obtain higher-quality results than state-of-the-art methods when using the same training and test data.

AK

327,105 views • 2 years ago

Tomorrow, my video comparing the TOP 5 Tools for Gaussian Splatting will go live. You will learn: - What is the best tool out there (free and paid) - How to clean your splat - How to compress your file by up to 99% - The core differences between profiles - How to create an immersive gallery to share your creation Drop a comment if you are interested and I will send you the link 👇

Tomorrow, my video comparing the TOP 5 Tools for Gaussian Splatting will go live. You will learn: - What is the best tool out there (free and paid) - How to clean your splat - How to compress your file by up to 99% - The core differences between profiles - How to create an immersive gallery to share your creation Drop a comment if you are interested and I will send you the link 👇

Gabriele Romagnoli

15,059 views • 1 year ago

Rupert Sheldrake: digital doesn't seem able to access morphic resonance field of memory [Akash, or cosmic consciousness], which is naturally built into self organizing systems. Possibly, analog or quantum computers might be able to create their own morphic memory field. Me: If megalithic structures are analog technologies designed with high dielectric materials, do they hold charge potential for analog intelligence, and if so, does that suggest Ancient stone temples access memory the way hueman temples do? And an even greater curiosity: what sort of memory, then, does Earth's origin Magneto-Dielectric load stone hold. And possible considerations: in the age of strong telepathy, all the various self-organizing systems of Life are able to embed in each other's morphic fields to communicate. Which may be how the plants used to be able to tell the Native Americans what their names and medicinal purposes were. As well as the stories of talking animals.

Rupert Sheldrake: digital doesn't seem able to access morphic resonance field of memory [Akash, or cosmic consciousness], which is naturally built into self organizing systems. Possibly, analog or quantum computers might be able to create their own morphic memory field. Me: If megalithic structures are analog technologies designed with high dielectric materials, do they hold charge potential for analog intelligence, and if so, does that suggest Ancient stone temples access memory the way hueman temples do? And an even greater curiosity: what sort of memory, then, does Earth's origin Magneto-Dielectric load stone hold. And possible considerations: in the age of strong telepathy, all the various self-organizing systems of Life are able to embed in each other's morphic fields to communicate. Which may be how the plants used to be able to tell the Native Americans what their names and medicinal purposes were. As well as the stories of talking animals.

Cymatic Joule

13,702 views • 6 months ago

The latest episode of the Permissionless Podcast with f(gautham)💤 is LIVE! Learn how Polynomial went from idea to doing over $5B in volume, how they're building the derivatives superchain, and more. Full episode and YouTube link are available below.

The latest episode of the Permissionless Podcast with f(gautham)💤 is LIVE! Learn how Polynomial went from idea to doing over $5B in volume, how they're building the derivatives superchain, and more. Full episode and YouTube link are available below.

Guardian

19,622 views • 1 year ago