Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

NeuralGS Bridging Neural Fields and 3D Gaussian Splatting for Compact 3D Representations

AK

423,551 subscribers

68,496 views • 1 year ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

F3D-Gaus: Feed-forward 3D-aware Generation on ImageNet with Cycle-Consistent Gaussian Splatting Contributions: • We pioneer 3D-aware generation using generalizable feed-forward Gaussian Splatting representation, achieving significant efficiency and favorable rendering quality on monocular datasets. • We significantly advance the capability of pixel-aligned Gaussian Splatting representations by designing a self-supervised cycle training strategy specifically tailored for monocular datasets. • We further mitigate the artifacts of 3D-aware representations caused by large viewpoint shifts by introducing geometry-aware video priors.

F3D-Gaus: Feed-forward 3D-aware Generation on ImageNet with Cycle-Consistent Gaussian Splatting Contributions: • We pioneer 3D-aware generation using generalizable feed-forward Gaussian Splatting representation, achieving significant efficiency and favorable rendering quality on monocular datasets. • We significantly advance the capability of pixel-aligned Gaussian Splatting representations by designing a self-supervised cycle training strategy specifically tailored for monocular datasets. • We further mitigate the artifacts of 3D-aware representations caused by large viewpoint shifts by introducing geometry-aware video priors.

MrNeRF

14,229 views • 1 year ago

3D Gaussian Splatting with Unity VFX Graph

3D Gaussian Splatting with Unity VFX Graph

Keijiro Takahashi

135,007 views • 2 years ago

Gaussian Painters imported into #b3d as ellipsoids (3D Gaussian Splatting plugin for Blender - work in progress)

Gaussian Painters imported into #b3d as ellipsoids (3D Gaussian Splatting plugin for Blender - work in progress)

Alex Carlier

156,152 views • 2 years ago

I implemented 3D Gaussian Splatting in WebGL. Check it out here:

I implemented 3D Gaussian Splatting in WebGL. Check it out here:

Kevin Kwok

168,875 views • 2 years ago

Testing InstantSplat with a few frames from Oppenheimer, super fast 3D Gaussian Splatting

Testing InstantSplat with a few frames from Oppenheimer, super fast 3D Gaussian Splatting

Radamés Ajna

48,871 views • 2 years ago

We will be in ACM SIGGRAPH 2023 with "3D Gaussian Splatting for Real-Time Radiance Field Rendering", have you ever seen radiance fields with 100+ FPS and MipNeRF360 quality? Check out our website here:

We will be in ACM SIGGRAPH 2023 with "3D Gaussian Splatting for Real-Time Radiance Field Rendering", have you ever seen radiance fields with 100+ FPS and MipNeRF360 quality? Check out our website here:

George Kopanas

135,782 views • 3 years ago

Meet 3D Gaussian Splatting - a 3D AI breakthrough that’s changing the game for real-time 3D rendering. It generates 3D world with state-of-the-art visuals at fast speeds: 100+ fps at 1080p. Here’s the magic behind it:

Meet 3D Gaussian Splatting - a 3D AI breakthrough that’s changing the game for real-time 3D rendering. It generates 3D world with state-of-the-art visuals at fast speeds: 100+ fps at 1080p. Here’s the magic behind it:

el.cine

60,115 views • 1 year ago

Lightplane Highly-Scalable Components for Neural 3D Fields Contemporary 3D research, particularly in reconstruction and generation, heavily relies on 2D images for inputs or supervision. However, current designs for these 2D-3D mapping are memory-intensive, posing a

Lightplane Highly-Scalable Components for Neural 3D Fields Contemporary 3D research, particularly in reconstruction and generation, heavily relies on 2D images for inputs or supervision. However, current designs for these 2D-3D mapping are memory-intensive, posing a

AK

18,896 views • 2 years ago

📢We introduce “RefFusion”, a novel inpainting method for scenes reconstructed using 3D Gaussian Splatting. 🔗 TLDR: we personalize an image diffusion model to a given reference image and distill its knowledge to 3D through score distillation sampling.

📢We introduce “RefFusion”, a novel inpainting method for scenes reconstructed using 3D Gaussian Splatting. 🔗 TLDR: we personalize an image diffusion model to a given reference image and distill its knowledge to 3D through score distillation sampling.

Ashkan Mirzaei

34,683 views • 2 years ago

I always thought camera pose estimation is necessary for 3D reconstruction, until Zequn and Stephen proved me wrong! Introducing PreF3R, purely feed-forward 3D Gaussian Splatting without any intermediate pose estimation and COLMAP initialization. Video in, 3D Gaussians and novel view rendering out. Key technique: spatial memory network from Spann3R and Gaussian head supervised by pointmap loss plus photometric loss. See more at:

I always thought camera pose estimation is necessary for 3D reconstruction, until Zequn and Stephen proved me wrong! Introducing PreF3R, purely feed-forward 3D Gaussian Splatting without any intermediate pose estimation and COLMAP initialization. Video in, 3D Gaussians and novel view rendering out. Key technique: spatial memory network from Spann3R and Gaussian head supervised by pointmap loss plus photometric loss. See more at:

Heng Yang

20,160 views • 1 year ago

Coding some WebGL shaders for 3D Gaussian Splatting 😀 Check the quality of clothes at 9 seconds! We are going to push things for fashion. live demo :

Coding some WebGL shaders for 3D Gaussian Splatting 😀 Check the quality of clothes at 9 seconds! We are going to push things for fashion. live demo :

Alexandre Devaux

327,044 views • 2 years ago

Nuvo: Neural UV Mapping! It's super difficult to UV map/texture atlas geometry produced by 3D reconstruction and generation pipelines. Nuvo works on all kinds of "unruly" 3D representations (NeRF, DreamFusion, etc.) and enables easy appearance editing! 1/3

Nuvo: Neural UV Mapping! It's super difficult to UV map/texture atlas geometry produced by 3D reconstruction and generation pipelines. Nuvo works on all kinds of "unruly" 3D representations (NeRF, DreamFusion, etc.) and enables easy appearance editing! 1/3

Pratul Srinivasan

20,842 views • 2 years ago

LGTM from Apple: 4K feed-forward 3D Gaussian Splatting. instant 4K 3D scenes without massive GPUs.. - predicts a few lightweight 3D shapes, wraps them in ultra-high-res 2D textures. - low Memory usage You take two normal photos of a room. Instantly walk around it in flawless 3D.

LGTM from Apple: 4K feed-forward 3D Gaussian Splatting. instant 4K 3D scenes without massive GPUs.. - predicts a few lightweight 3D shapes, wraps them in ultra-high-res 2D textures. - low Memory usage You take two normal photos of a room. Instantly walk around it in flawless 3D.

Wildminder

46,278 views • 2 months ago

Introducing pixelSplat: feed-forward Gaussian splats from image pairs! Led by David Charatan and Lester Li, collaborating with Andrea Tagliasacchi @CVPR! We propose a memory-efficient, fast and editable alternative to pixelNeRF based on 3D Gaussian Splatting! 1/n

Introducing pixelSplat: feed-forward Gaussian splats from image pairs! Led by David Charatan and Lester Li, collaborating with Andrea Tagliasacchi @CVPR! We propose a memory-efficient, fast and editable alternative to pixelNeRF based on 3D Gaussian Splatting! 1/n

Vincent Sitzmann

47,594 views • 2 years ago

Tune in @ ICCV on Mon @ 10.30am where I talk about everything 3D + Realism: - Hyperscape: Gaussian Splatting in VR - FlowR: Flowing from Sparse-2-Dense 3D Recon - BulletGen: Improving 4D Recon with Bullet-Time Gen - MapAnything: Universal Feed-Forward Metric 3D Recon 🧵👇

Tune in @ ICCV on Mon @ 10.30am where I talk about everything 3D + Realism: - Hyperscape: Gaussian Splatting in VR - FlowR: Flowing from Sparse-2-Dense 3D Recon - BulletGen: Improving 4D Recon with Bullet-Time Gen - MapAnything: Universal Feed-Forward Metric 3D Recon 🧵👇

Jonathon Luiten

15,386 views • 8 months ago

PlayCanvas(PlayCanvas) + 3D Gaussian Splatting始めました。このアプリはWebブラウザ上で動いています！

PlayCanvas(PlayCanvas) + 3D Gaussian Splatting始めました。このアプリはWebブラウザ上で動いています！

ダックビル＠STUDIO DUCKBILL LLC

47,954 views • 1 year ago

I have written a tutorial on how to train your own "3D Gaussian Splatting" models! #GaussianSplatting Write me here if you're facing any issues. ⬇️⬇️

I have written a tutorial on how to train your own "3D Gaussian Splatting" models! #GaussianSplatting Write me here if you're facing any issues. ⬇️⬇️

Alex Carlier

103,203 views • 2 years ago

Segment Any 3D Gaussians paper page: Interactive 3D segmentation in radiance fields is an appealing task since its importance in 3D scene understanding and manipulation. However, existing methods face challenges in either achieving fine-grained, multi-granularity segmentation or contending with substantial computational overhead, inhibiting real-time interaction. In this paper, we introduce Segment Any 3D GAussians (SAGA), a novel 3D interactive segmentation approach that seamlessly blends a 2D segmentation foundation model with 3D Gaussian Splatting (3DGS), a recent breakthrough of radiance fields. SAGA efficiently embeds multi-granularity 2D segmentation results generated by the segmentation foundation model into 3D Gaussian point features through well-designed contrastive training. Evaluation on existing benchmarks demonstrates that SAGA can achieve competitive performance with state-of-the-art methods. Moreover, SAGA achieves multi-granularity segmentation and accommodates various prompts, including points, scribbles, and 2D masks. Notably, SAGA can finish the 3D segmentation within milliseconds, achieving nearly 1000x acceleration compared to previous SOTA.

Segment Any 3D Gaussians paper page: Interactive 3D segmentation in radiance fields is an appealing task since its importance in 3D scene understanding and manipulation. However, existing methods face challenges in either achieving fine-grained, multi-granularity segmentation or contending with substantial computational overhead, inhibiting real-time interaction. In this paper, we introduce Segment Any 3D GAussians (SAGA), a novel 3D interactive segmentation approach that seamlessly blends a 2D segmentation foundation model with 3D Gaussian Splatting (3DGS), a recent breakthrough of radiance fields. SAGA efficiently embeds multi-granularity 2D segmentation results generated by the segmentation foundation model into 3D Gaussian point features through well-designed contrastive training. Evaluation on existing benchmarks demonstrates that SAGA can achieve competitive performance with state-of-the-art methods. Moreover, SAGA achieves multi-granularity segmentation and accommodates various prompts, including points, scribbles, and 2D masks. Notably, SAGA can finish the 3D segmentation within milliseconds, achieving nearly 1000x acceleration compared to previous SOTA.

AK

69,542 views • 2 years ago

Fourier Integral & 3D Gaussian Wave

Fourier Integral & 3D Gaussian Wave

ₕₐₘₚₜₒₙ

84,368 views • 1 year ago

DroneSplat: 3D Gaussian Splatting for Robust 3D Reconstruction from In-the-Wild Drone Imagery Abstract: Drones have become essential tools for reconstructing wild scenes due to their outstanding maneuverability. Recent advances in radiance field methods have achieved remarkable rendering quality, providing a new avenue for 3D reconstruction from drone imagery. However, dynamic distractors in wild environments challenge the static scene assumption in radiance fields, while limited view constraints hinder the accurate capture of underlying scene geometry. To address these challenges, we introduce DroneSplat, a novel framework designed for robust 3D reconstruction from in-the-wild drone imagery. Our method adaptively adjusts masking thresholds by integrating local-global segmentation heuristics with statistical approaches, enabling precise identification and elimination of dynamic distractors in static scenes. We enhance 3D Gaussian Splatting with multi-view stereo predictions and a voxel-guided optimization strategy, supporting high-quality rendering under limited view constraints. For comprehensive evaluation, we provide a drone-captured 3D reconstruction dataset encompassing both dynamic and static scenes. Extensive experiments demonstrate that DroneSplat outperforms both 3DGS and NeRF baselines in handling in-the-wild drone imagery.

DroneSplat: 3D Gaussian Splatting for Robust 3D Reconstruction from In-the-Wild Drone Imagery Abstract: Drones have become essential tools for reconstructing wild scenes due to their outstanding maneuverability. Recent advances in radiance field methods have achieved remarkable rendering quality, providing a new avenue for 3D reconstruction from drone imagery. However, dynamic distractors in wild environments challenge the static scene assumption in radiance fields, while limited view constraints hinder the accurate capture of underlying scene geometry. To address these challenges, we introduce DroneSplat, a novel framework designed for robust 3D reconstruction from in-the-wild drone imagery. Our method adaptively adjusts masking thresholds by integrating local-global segmentation heuristics with statistical approaches, enabling precise identification and elimination of dynamic distractors in static scenes. We enhance 3D Gaussian Splatting with multi-view stereo predictions and a voxel-guided optimization strategy, supporting high-quality rendering under limited view constraints. For comprehensive evaluation, we provide a drone-captured 3D reconstruction dataset encompassing both dynamic and static scenes. Extensive experiments demonstrate that DroneSplat outperforms both 3DGS and NeRF baselines in handling in-the-wild drone imagery.

MrNeRF

21,340 views • 1 year ago