Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

open_semantic_slam ICRA2025: OpenGS-SLAM: Open-Set Dense Semantic SLAM with 3D Gaussian Splatting for Object-Level Scene Understanding

rsasaki0109

6,931 subscribers

27,137 views • 1 year ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

3 Comments

Anto Patrex1 year ago

Is this realtime?

Rainmaker1 year ago

Can reinforcement learning handle stock market swings? In my latest free Substack, find out how SARSA reinforcement learning algorithm can help create adaptive strategies and improve performance.

Albert1 year ago

how about in dynamic environments and under heavy occlusions?"

Related Videos

LEGO-SLAM: Language-Embedded Gaussian Optimization SLAM LEGO-SLAM running at 15 FPS on a ScanNet scene with language-based loop closing for drift correction. LEGO-SLAM is a 3DGS-based SLAM framework that supports open-vocabulary semantic querying and rendering. It tracks via G-ICP and efficiently builds a map by embedding Gaussians with scene-adaptive 16D language features. Map management is achieved through Language Pruning and Language-Based Loop Detection. The generated map enables open-vocabulary 3D Object Localization.

LEGO-SLAM: Language-Embedded Gaussian Optimization SLAM LEGO-SLAM running at 15 FPS on a ScanNet scene with language-based loop closing for drift correction. LEGO-SLAM is a 3DGS-based SLAM framework that supports open-vocabulary semantic querying and rendering. It tracks via G-ICP and efficiently builds a map by embedding Gaussians with scene-adaptive 16D language features. Map management is achieved through Language Pruning and Language-Based Loop Detection. The generated map enables open-vocabulary 3D Object Localization.

Ryohei Sasaki@engineer

15,060 views • 4 months ago

"Triangle Splatting SLAM" TL;DR: the first dense RGB-D SLAM system built on differentiable triangle splatting, enabling real-time tracking, mesh reconstruction, collision checking, and scene editing from a unified representation.

"Triangle Splatting SLAM" TL;DR: the first dense RGB-D SLAM system built on differentiable triangle splatting, enabling real-time tracking, mesh reconstruction, collision checking, and scene editing from a unified representation.

Alexandre Morgand

11,958 views • 2 months ago

[SIGGRAPH '24] RTG-SLAM: Real-time 3D Reconstruction at Scale Using Gaussian Splatting Paper (pdf link): Project: Code: Method ⬇️ 1 I 2

[SIGGRAPH '24] RTG-SLAM: Real-time 3D Reconstruction at Scale Using Gaussian Splatting Paper (pdf link): Project: Code: Method ⬇️ 1 I 2

MrNeRF

35,675 views • 2 years ago

Code released: "GaussianObject: Just Taking Four Images to Get A High-Quality 3D Object with Gaussian Splatting"

Code released: "GaussianObject: Just Taking Four Images to Get A High-Quality 3D Object with Gaussian Splatting"

MrNeRF

22,274 views • 2 years ago

SuperGSeg: Open-Vocabulary 3D Segmentation with Structured Super-Gaussians Contributions: • We propose SuperGSeg: a 3D segmentation method with neural Gaussians, designed to learn hierarchical instance segmentation features from 2D foundation models. • We introduce the concept of Super-Gaussian, a novel representation that integrates hierarchical instance segmentation features, enabling the embedding of high-dimensional language features. This approach addresses previously unfeasible challenges in representing complex scenes with rich semantic details. • Extensive experiments on the LERF-OVS and ScanNet datasets demonstrate the effectiveness of the proposed method, achieving significant improvements in open-vocabulary 3D object-level and scene-level semantic segmentation. It shows particular strength in capturing fine-grained scene details and dense pixel semantic segmentation tasks for the first time.

SuperGSeg: Open-Vocabulary 3D Segmentation with Structured Super-Gaussians Contributions: • We propose SuperGSeg: a 3D segmentation method with neural Gaussians, designed to learn hierarchical instance segmentation features from 2D foundation models. • We introduce the concept of Super-Gaussian, a novel representation that integrates hierarchical instance segmentation features, enabling the embedding of high-dimensional language features. This approach addresses previously unfeasible challenges in representing complex scenes with rich semantic details. • Extensive experiments on the LERF-OVS and ScanNet datasets demonstrate the effectiveness of the proposed method, achieving significant improvements in open-vocabulary 3D object-level and scene-level semantic segmentation. It shows particular strength in capturing fine-grained scene details and dense pixel semantic segmentation tasks for the first time.

MrNeRF

13,594 views • 1 year ago

"YoNoSplat: You Only Need One Model for Feedforward 3D Gaussian Splatting" TL;DR: a unified 3D Gaussian splatting model that reconstructs high-quality scene geometry and camera poses from unposed/uncalibrated images in a single forward pass.

"YoNoSplat: You Only Need One Model for Feedforward 3D Gaussian Splatting" TL;DR: a unified 3D Gaussian splatting model that reconstructs high-quality scene geometry and camera poses from unposed/uncalibrated images in a single forward pass.

Alexandre Morgand

15,035 views • 5 months ago

Goodbye SOGS, hello SOG! 👋 PlayCanvas open sources Spatially Ordered Gaussians - a new super-compressed format for 3D Gaussian Splatting.

Goodbye SOGS, hello SOG! 👋 PlayCanvas open sources Spatially Ordered Gaussians - a new super-compressed format for 3D Gaussian Splatting.

PlayCanvas

48,308 views • 10 months ago

Gaussian Shell Maps are a new neural scene representation that connects fields and 3D Gaussians. This representation unlocks the full potential of 3D Gaussian splatting for generative AI applications, such as 3D avatar generation. 1/2

Gordon Wetzstein

52,480 views • 2 years ago

$DreamScene360 Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splatting The increasing demand for virtual reality applications has highlighted the significance of crafting immersive 3D assets. We present a text-to-3D 360^{circ} scene generation pipeline$

DreamScene360 Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splatting The increasing demand for virtual reality applications has highlighted the significance of crafting immersive 3D assets. We present a text-to-3D 360^{circ} scene generation pipeline

AK

24,910 views • 2 years ago

Fisheye Gaussian Splatting in PlayCanvas with 180° FOV in the browser. Step inside the scene… this is immersion on another level 🔥

Fisheye Gaussian Splatting in PlayCanvas with 180° FOV in the browser. Step inside the scene… this is immersion on another level 🔥

Martin Valigursky

10,715 views • 4 months ago

GauS-SLAM: Dense RGB-D SLAM with Gaussian Surfels • We propose a 2D Gaussian-based incremental reconstruction strategy and a Surface-aware Depth Rendering mechanism. This approach effectively mitigates geometry distortions and improves tracking accuracy. • Our dense SLAM system features a front-end/back-end architecture and incorporates a local map design, ensuring tracking accuracy and efficiency. • We conduct extensive experiments demonstrating the superiority of our approach in both tracking accuracy and reconstruction quality compared to SOTA methods.

GauS-SLAM: Dense RGB-D SLAM with Gaussian Surfels • We propose a 2D Gaussian-based incremental reconstruction strategy and a Surface-aware Depth Rendering mechanism. This approach effectively mitigates geometry distortions and improves tracking accuracy. • Our dense SLAM system features a front-end/back-end architecture and incorporates a local map design, ensuring tracking accuracy and efficiency. • We conduct extensive experiments demonstrating the superiority of our approach in both tracking accuracy and reconstruction quality compared to SOTA methods.

MrNeRF

11,004 views • 1 year ago

Happy to announce the results of our latest research, which takes 3D Gaussian Splatting to the next level: "A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets," which has been accepted at #SIGGRAPH2024!🎉 Find it here:

Happy to announce the results of our latest research, which takes 3D Gaussian Splatting to the next level: "A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets," which has been accepted at #SIGGRAPH2024!🎉 Find it here:

Bernhard Kerbl

356,419 views • 2 years ago

Big day for open source 3D capture! 🌐 nerfstudio hits 1.0 and gets a *commercial-ready* Gaussian Splatting implementation Google AI open sourced Zip-NeRF — the highest quality 3D reconstruction pipeline, setting the stage for a future SMERF release

Big day for open source 3D capture! 🌐 nerfstudio hits 1.0 and gets a commercial-ready Gaussian Splatting implementation Google AI open sourced Zip-NeRF — the highest quality 3D reconstruction pipeline, setting the stage for a future SMERF release

Bilawal Sidhu

85,222 views • 2 years ago

remind-reid-tracker REMIND — RE-Identification with Memory for INDoor Navigation REMIND addresses a core challenge in visual tracking: re-identifying objects that disappear and reappear, look similar to one another, or are observed from changing viewpoints. Rather than relying on position or motion cues, REMIND builds appearance-based identity models per object using DINOv3 patch features, decomposed into: Global and part-level descriptors — K-means and attention-guided semantic parts extracted per detection Relational context (neighbor sets) — structural scene layout encoded as co-occurrence graphs of neighboring objects Known-set distance disambiguation — geometry-aware resolution of visually ambiguous groups Adaptive memory — per-object appearance, part, and background models that update over time The association pipeline runs a per-frame sequence of visual evidence building, context activation, global Hungarian assignment, and post-assignment guards, producing explicit uncertainty signals (ambiguous, provisional) alongside confident identity decisions. Evaluation outputs span case, object, frame, class, scene, and batch levels, with full internal telemetry for diagnostic analysis.

remind-reid-tracker REMIND — RE-Identification with Memory for INDoor Navigation REMIND addresses a core challenge in visual tracking: re-identifying objects that disappear and reappear, look similar to one another, or are observed from changing viewpoints. Rather than relying on position or motion cues, REMIND builds appearance-based identity models per object using DINOv3 patch features, decomposed into: Global and part-level descriptors — K-means and attention-guided semantic parts extracted per detection Relational context (neighbor sets) — structural scene layout encoded as co-occurrence graphs of neighboring objects Known-set distance disambiguation — geometry-aware resolution of visually ambiguous groups Adaptive memory — per-object appearance, part, and background models that update over time The association pipeline runs a per-frame sequence of visual evidence building, context activation, global Hungarian assignment, and post-assignment guards, producing explicit uncertainty signals (ambiguous, provisional) alongside confident identity decisions. Evaluation outputs span case, object, frame, class, scene, and batch levels, with full internal telemetry for diagnostic analysis.

Ryohei Sasaki@engineer

18,478 views • 12 days ago

Coding some shaders for 3D Gaussian Splatting 😀 Input is a 40 second video processed with Luma Check the reflections on the bag! Demo:

Coding some shaders for 3D Gaussian Splatting 😀 Input is a 40 second video processed with Luma Check the reflections on the bag! Demo:

Alexandre Devaux

44,165 views • 2 years ago

📢 I’m testing ViggleAI new AI tool, PINOC, and it’s seriously impressive. You can create 3D Gaussian splatting models from just one image and animate them with a video! It’s the first AI I’ve tried that can animate Gaussian splatting models. I also talked to someone from Viggle, and they might add the option to import your own Gaussian splatting models and animate them. If that happens, I’ll probably make a video tutorial showing how to boost the detail level of a Gaussian splatting model by 4x to create highly realistic animated characters.

📢 I’m testing ViggleAI new AI tool, PINOC, and it’s seriously impressive. You can create 3D Gaussian splatting models from just one image and animate them with a video! It’s the first AI I’ve tried that can animate Gaussian splatting models. I also talked to someone from Viggle, and they might add the option to import your own Gaussian splatting models and animate them. If that happens, I’ll probably make a video tutorial showing how to boost the detail level of a Gaussian splatting model by 4x to create highly realistic animated characters.

Alex

29,776 views • 3 months ago

InstantSplat++ is now open source. It is a lightweight library that connects foundation models (VGGT, MASt3R, MAP-Anything, etc.) with the Gaussian splatting family. Given uncalibrated images, it optimizes a 3D scene in a few seconds. Try the demo and code here:

InstantSplat++ is now open source. It is a lightweight library that connects foundation models (VGGT, MASt3R, MAP-Anything, etc.) with the Gaussian splatting family. Given uncalibrated images, it optimizes a 3D scene in a few seconds. Try the demo and code here:

Zhiwen(Aaron) Fan

31,864 views • 5 months ago

⚙️ 404—GEN has released the WORLD'S LARGEST open-source 3D model dataset ⚙️ • 21.5M+ AI-generated 3d models • Open-source and attribution-ready • Larger than all existing 3D datasets combined This is an unmatched scale for Gaussian Splatting research. More details below 🧵 ⬇️

⚙️ 404—GEN has released the WORLD'S LARGEST open-source 3D model dataset ⚙️ • 21.5M+ AI-generated 3d models • Open-source and attribution-ready • Larger than all existing 3D datasets combined This is an unmatched scale for Gaussian Splatting research. More details below 🧵 ⬇️

404

44,735 views • 1 year ago

As Hamburg is covered in a thick layer of snow, Jörg Kahlhöfer took the opportunity to capture this street scene with 3D Gaussian Splatting. Watch him try out different lighting setups:

As Hamburg is covered in a thick layer of snow, Jörg Kahlhöfer took the opportunity to capture this street scene with 3D Gaussian Splatting. Watch him try out different lighting setups:

80 LEVEL

48,450 views • 7 months ago

Experimenting with Gaussian splatting as a scene representation for monocular autonomous driving. visualization using Rerun running through a custom Apple ML-Sharp based pipeline..

Experimenting with Gaussian splatting as a scene representation for monocular autonomous driving. visualization using Rerun running through a custom Apple ML-Sharp based pipeline..

Jaskirat Singh

18,719 views • 7 months ago