Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Open Sourcing Forge: 3D Gaussian splat rendering for web developers! 3DGS has become a dominant paradigm for differentiable rendering, combining high visual quality and real-time rendering. However, support for splatting on the web still lags behind its adoption in AI.

spark

5,812 subscribers

120,180 просмотров • 1 год назад •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

Комментарии: 14

Фото профиля spark

spark1 год назад

At @theworldlabs, we’re building generative 3D foundation models to unlock spatial intelligence. Splats are an important representation for 3D AI, so to enable our vision we built a fast, dynamic web renderer. We believe this tech can be useful to everyone working with 3DGS.

Фото профиля spark

spark1 год назад

Today we’re open sourcing Forge under the MIT license for anyone to use or contribute to, aiming to expand what’s possible with 3DGS with a flexible architecture that enables fully dynamic & programmable Gaussian splatting.

Фото профиля spark

spark1 год назад

Forge integrates seamlessly with @threejs, runs in the browser via WebGL2 (desktop, mobile, and VR), can load most major splat file formats, and requires minimal code to get up and running. It supports multiple splat objects, multiple cameras, and real-time animation/editing.

Фото профиля spark

spark1 год назад

Underlying this is a design that runs user-programmable functions for every splat on the GPU (like shaders in modern 3D graphics) and can be used to create complex animations and effects. We want to help people build new interactive experiences never before seen with 3DGS!

Фото профиля spark

spark1 год назад

You can find full source code, examples, documentation, and more at We can’t wait to see what you’ll build!

Фото профиля MrNeRF

MrNeRF1 год назад

@forgegfx @BenMildenhall Awesome!

Фото профиля Georgia Gkioxari

Georgia Gkioxari1 год назад

@forgegfx Nice!! Looks 🔥🔥

Фото профиля Yanming Di

Yanming Di1 год назад

@forgegfx @grok what is splat, when can CS people stop inventing new terms ...

Фото профиля shank ⚡️

shank ⚡️1 год назад

@forgegfx Cool Will try

Фото профиля Julien Blanchon 🇺🇦

Julien Blanchon 🇺🇦1 год назад

@forgegfx Possible to have an integration with threlte ?

Фото профиля Ian Curtis

Ian Curtis1 год назад

@forgegfx 🤩🔥💯

Фото профиля Saitoshi

Saitoshi1 год назад

Great to see Forge open-sourced, @sparkjsdev! Easy integration with Three.js scenes will make it even more accessible for developers. This will be a game changer!

Фото профиля XR# —WebXR with .NET, C# & XAML

XR# —WebXR with .NET, C# & XAML1 год назад

@forgegfx Awesome! 👏

Фото профиля Patricia

Patricia1 год назад

@forgegfx Awesome!

Похожие видео

Google presents RadSplat Radiance Field-Informed Gaussian Splatting for Robust Real-Time Rendering with 900+ FPS Recent advances in view synthesis and real-time rendering have achieved photorealistic quality at impressive rendering speeds. While Radiance Field-based

Google presents RadSplat Radiance Field-Informed Gaussian Splatting for Robust Real-Time Rendering with 900+ FPS Recent advances in view synthesis and real-time rendering have achieved photorealistic quality at impressive rendering speeds. While Radiance Field-based

AK

139,790 просмотров • 2 лет назад

'3D Gaussian Splatting for Real-Time RF Rendering' rose to the 2nd challenge I presented. The magnifying glass isn't still crystal clear, but the result is compelling. Look at the small details/hairs on the succulent plant. Rendered in real-time. #gaussiansplatting #3D #AI

'3D Gaussian Splatting for Real-Time RF Rendering' rose to the 2nd challenge I presented. The magnifying glass isn't still crystal clear, but the result is compelling. Look at the small details/hairs on the succulent plant. Rendered in real-time. #gaussiansplatting #3D #AI

Hugues Bruyère

298,382 просмотров • 3 лет назад

$🚀 PlayCanvas Engine 2.21.0 is out! Your free and open source web graphics engine 🔮 3D Gaussian splat refraction 🌊 Water rendering 🌥️ Volumetric fog 〰️ Styled and dynamic fat line rendering Get started today: npm install playcanvas [1 / 2]$

🚀 PlayCanvas Engine 2.21.0 is out! Your free and open source web graphics engine 🔮 3D Gaussian splat refraction 🌊 Water rendering 🌥️ Volumetric fog 〰️ Styled and dynamic fat line rendering Get started today: npm install playcanvas [1 / 2]

PlayCanvas

11,511 просмотров • 3 месяцев назад

Painting with 3D Gaussian Splat Brushes Contributions: • Designing a set of interactive tools and brush parameters for artistic brush creation and control using 3DGS content. • Computing oriented 3D Gaussian splat brushes for stamp-based painting on 3D surfaces represented as meshes or 3DGS scenes. • Deforming the splats in a brush stamp to ensure a smooth appearance of the painted 3D stroke. • Producing seamless brush strokes despite overlapping brush stamps, using diffusion inpainting. • Efficient modeling and rendering of brush strokes to facilitate 3DGS painting in real time.

Painting with 3D Gaussian Splat Brushes Contributions: • Designing a set of interactive tools and brush parameters for artistic brush creation and control using 3DGS content. • Computing oriented 3D Gaussian splat brushes for stamp-based painting on 3D surfaces represented as meshes or 3DGS scenes. • Deforming the splats in a brush stamp to ensure a smooth appearance of the painted 3D stroke. • Producing seamless brush strokes despite overlapping brush stamps, using diffusion inpainting. • Efficient modeling and rendering of brush strokes to facilitate 3DGS painting in real time.

MrNeRF

23,060 просмотров • 1 год назад

Happy to introduce Habitat-GS, a non-intrusive extension of Habitat-Sim that brings dynamic Gaussian Splatting for photorealistic rendering and comes with hundreds of high-quality 3DGS scene assets, aiming to empowering navigation research. Code:

Happy to introduce Habitat-GS, a non-intrusive extension of Habitat-Sim that brings dynamic Gaussian Splatting for photorealistic rendering and comes with hundreds of high-quality 3DGS scene assets, aiming to empowering navigation research. Code:

Sida Peng

12,300 просмотров • 5 месяцев назад

📢Happy to present Convex Splatting, a novel way for 3D reconstruction based on 3D smooth convexes. For the first time, a splatting-based method reaches the quality of NeRF sota methods but with real-time rendering and few primitives!! I expect this to replace Gaussian splatting for 3D in the coming months. CODE RELEASED TODAY! joint work with collaborators from Université de Liège Visual Geometry Group (VGG) , KAUST Computer Vision Lab (IVUL) a thread 🧵 1/n

📢Happy to present Convex Splatting, a novel way for 3D reconstruction based on 3D smooth convexes. For the first time, a splatting-based method reaches the quality of NeRF sota methods but with real-time rendering and few primitives!! I expect this to replace Gaussian splatting for 3D in the coming months. CODE RELEASED TODAY! joint work with collaborators from Université de Liège Visual Geometry Group (VGG) , KAUST Computer Vision Lab (IVUL) a thread 🧵 1/n

Abdullah Hamdi

71,661 просмотров • 1 год назад

Happy to announce the results of our latest research, which takes 3D Gaussian Splatting to the next level: "A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets," which has been accepted at #SIGGRAPH2024!🎉 Find it here:

Happy to announce the results of our latest research, which takes 3D Gaussian Splatting to the next level: "A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets," which has been accepted at #SIGGRAPH2024!🎉 Find it here:

Bernhard Kerbl

356,584 просмотров • 2 лет назад

The PlayCanvas team has solved collision for 3D Gaussian splats. Install splat-transform via NPM to get a CLI tool + library that can output high quality voxel-based collision. Here you can see a splat navigated in first person mode with voxel rendering toggled on/off. 🧵

The PlayCanvas team has solved collision for 3D Gaussian splats. Install splat-transform via NPM to get a CLI tool + library that can output high quality voxel-based collision. Here you can see a splat navigated in first person mode with voxel rendering toggled on/off. 🧵

Will Eastcott

30,303 просмотров • 6 месяцев назад

GPS-Gaussian+: Generalizable Pixel-wise 3D Gaussian Splatting for Real-Time Human-Scene Rendering from Sparse Views TL;DR: Are we witnessing the first steps towards 3DGS live streaming? Contributions: • We introduce a generalizable 3D Gaussian Splatting methodology that employs pixel-wise Gaussian parameter maps defined on 2D source image planes to formulate 3D Gaussians in a feed-forward manner. • We propose a fully differentiable framework composed of an iterative depth estimation module and a Gaussian parameter regression module. The intermediate depth prediction bridges the two components and allows them to benefit from joint training. • We introduce a regularization term and an epipolar attention mechanism to preserve geometry consistency between the two source views when using only rendering loss. Our method generalizes well to unseen characters even in complicated scenes. • We develop a real-time FVV system that achieves high-resolution rendering of characters in the scene without any geometry supervision.

GPS-Gaussian+: Generalizable Pixel-wise 3D Gaussian Splatting for Real-Time Human-Scene Rendering from Sparse Views TL;DR: Are we witnessing the first steps towards 3DGS live streaming? Contributions: • We introduce a generalizable 3D Gaussian Splatting methodology that employs pixel-wise Gaussian parameter maps defined on 2D source image planes to formulate 3D Gaussians in a feed-forward manner. • We propose a fully differentiable framework composed of an iterative depth estimation module and a Gaussian parameter regression module. The intermediate depth prediction bridges the two components and allows them to benefit from joint training. • We introduce a regularization term and an epipolar attention mechanism to preserve geometry consistency between the two source views when using only rendering loss. Our method generalizes well to unseen characters even in complicated scenes. • We develop a real-time FVV system that achieves high-resolution rendering of characters in the scene without any geometry supervision.

MrNeRF

25,893 просмотров • 1 год назад

3D Gaussian Splatting inside Godot🎮 Instead of forcing splats into a mesh pipeline, this project adds a separate rendering pass🤩 and blends it after mesh rendering. And yeah, it’s fully open source🥳

3D Gaussian Splatting inside Godot🎮 Instead of forcing splats into a mesh pipeline, this project adds a separate rendering pass🤩 and blends it after mesh rendering. And yeah, it’s fully open source🥳

KIRI Engine - 3D Scanner App

19,314 просмотров • 7 месяцев назад

$Spatial AI is increasingly important, and the newest papers from #NVIDIAResearch, 3DGRT and 3DGUT, represent significant advancements in enabling researchers and developers to explore and innovate with 3D Gaussian Splatting techniques. 💎 3DGRT (Gaussian Ray Tracing) ➡️ - Enables secondary lighting effects such as refractions, shadows, depth of field, and mirrors. - Provides a foundation for further research in ray tracing-based rendering of Gaussian representations. - Achieves real-time rendering performance on modern GPUs. ✨ 3DGUT (Gaussain Unscented Transform) ➡️ - Support for distorted camera models and time-dependent effects like rolling shutter. - Enables phenomena such as reflections and refractions while retaining real-time efficiency. 📥 Code:$

Spatial AI is increasingly important, and the newest papers from #NVIDIAResearch, 3DGRT and 3DGUT, represent significant advancements in enabling researchers and developers to explore and innovate with 3D Gaussian Splatting techniques. 💎 3DGRT (Gaussian Ray Tracing) ➡️ - Enables secondary lighting effects such as refractions, shadows, depth of field, and mirrors. - Provides a foundation for further research in ray tracing-based rendering of Gaussian representations. - Achieves real-time rendering performance on modern GPUs. ✨ 3DGUT (Gaussain Unscented Transform) ➡️ - Support for distorted camera models and time-dependent effects like rolling shutter. - Enables phenomena such as reflections and refractions while retaining real-time efficiency. 📥 Code:

NVIDIA AI Developer

27,782 просмотров • 1 год назад

Off-axis projection experiment no. 2 mediapipe for face tracking, sparkjs for gaussian splat rendering via threejs

Off-axis projection experiment no. 2 mediapipe for face tracking, sparkjs for gaussian splat rendering via threejs

Ian Curtis

51,892 просмотров • 9 месяцев назад

City-on-Web: Real-time Neural Rendering of Large-scale Scenes on the Web paper page: enables real-time neural rendering of large-scale scenes over web using laptop GPUs. The result is tested on the NVIDIA RTX 3060 Laptop GPU with 1920x1080 resolution

City-on-Web: Real-time Neural Rendering of Large-scale Scenes on the Web paper page: enables real-time neural rendering of large-scale scenes over web using laptop GPUs. The result is tested on the NVIDIA RTX 3060 Laptop GPU with 1920x1080 resolution

AK

79,524 просмотров • 2 лет назад

$FAU Erlangen-Nürnberg presents TRIPS Trilinear Point Splatting for Real-Time Radiance Field Rendering paper page: Point-based radiance field rendering has demonstrated impressive results for novel view synthesis, offering a compelling blend of rendering quality and computational efficiency. However, also latest approaches in this domain are not without their shortcomings. 3D Gaussian Splatting [Kerbl and Kopanas et al. 2023] struggles when tasked with rendering highly detailed scenes, due to blurring and cloudy artifacts. On the other hand, ADOP [R\"uckert et al. 2022] can accommodate crisper images, but the neural reconstruction network decreases performance, it grapples with temporal instability and it is unable to effectively address large gaps in the point cloud. In this paper, we present TRIPS (Trilinear Point Splatting), an approach that combines ideas from both Gaussian Splatting and ADOP. The fundamental concept behind our novel technique involves rasterizing points into a screen-space image pyramid, with the selection of the pyramid layer determined by the projected point size. This approach allows rendering arbitrarily large points using a single trilinear write. A lightweight neural network is then used to reconstruct a hole-free image including detail beyond splat resolution. Importantly, our render pipeline is entirely differentiable, allowing for automatic optimization of both point sizes and positions. Our evaluation demonstrate that TRIPS surpasses existing state-of-the-art methods in terms of rendering quality while maintaining a real-time frame rate of 60 frames per second on readily available hardware. This performance extends to challenging scenarios, such as scenes featuring intricate geometry, expansive landscapes, and auto-exposed footage.$

FAU Erlangen-Nürnberg presents TRIPS Trilinear Point Splatting for Real-Time Radiance Field Rendering paper page: Point-based radiance field rendering has demonstrated impressive results for novel view synthesis, offering a compelling blend of rendering quality and computational efficiency. However, also latest approaches in this domain are not without their shortcomings. 3D Gaussian Splatting [Kerbl and Kopanas et al. 2023] struggles when tasked with rendering highly detailed scenes, due to blurring and cloudy artifacts. On the other hand, ADOP [R\"uckert et al. 2022] can accommodate crisper images, but the neural reconstruction network decreases performance, it grapples with temporal instability and it is unable to effectively address large gaps in the point cloud. In this paper, we present TRIPS (Trilinear Point Splatting), an approach that combines ideas from both Gaussian Splatting and ADOP. The fundamental concept behind our novel technique involves rasterizing points into a screen-space image pyramid, with the selection of the pyramid layer determined by the projected point size. This approach allows rendering arbitrarily large points using a single trilinear write. A lightweight neural network is then used to reconstruct a hole-free image including detail beyond splat resolution. Importantly, our render pipeline is entirely differentiable, allowing for automatic optimization of both point sizes and positions. Our evaluation demonstrate that TRIPS surpasses existing state-of-the-art methods in terms of rendering quality while maintaining a real-time frame rate of 60 frames per second on readily available hardware. This performance extends to challenging scenarios, such as scenes featuring intricate geometry, expansive landscapes, and auto-exposed footage.

AK

45,489 просмотров • 2 лет назад

Spark 2.0 Developer Preview is now available: introducing Level-of-Detail (LoD) rendering and streaming for world-scale Gaussian Splats on the web. Learn more ↓

Spark 2.0 Developer Preview is now available: introducing Level-of-Detail (LoD) rendering and streaming for world-scale Gaussian Splats on the web. Learn more ↓

spark

233,091 просмотров • 8 месяцев назад

LangSplatV2: High-dimensional 3D Language Gaussian Splatting with 450+ FPS Contributions: • LangSplatV2 achieves real-time performance with 476.2 FPS for high-dimensional feature splatting and 384.6 FPS for 3D open-vocabulary text querying. • Delivers a 42× speedup and 47× boost in performance compared to LangSplat. • Improves query accuracy while drastically reducing inference time. • Replaces the heavyweight decoder in LangSplat with a sparse coefficient field, removing the main performance bottleneck. • Introduces a CUDA-optimized sparse coefficient splatting method, enabling fast and high-quality rendering of high-dimensional features. • Enables scalable 3D language interaction in complex scenes, opening up real-time applications previously not possible with LangSplat.

LangSplatV2: High-dimensional 3D Language Gaussian Splatting with 450+ FPS Contributions: • LangSplatV2 achieves real-time performance with 476.2 FPS for high-dimensional feature splatting and 384.6 FPS for 3D open-vocabulary text querying. • Delivers a 42× speedup and 47× boost in performance compared to LangSplat. • Improves query accuracy while drastically reducing inference time. • Replaces the heavyweight decoder in LangSplat with a sparse coefficient field, removing the main performance bottleneck. • Introduces a CUDA-optimized sparse coefficient splatting method, enabling fast and high-quality rendering of high-dimensional features. • Enables scalable 3D language interaction in complex scenes, opening up real-time applications previously not possible with LangSplat.

MrNeRF

14,365 просмотров • 1 год назад