Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Experimenting with Gaussian splatting as a scene representation for monocular autonomous driving. visualization using Rerun running through a custom Apple ML-Sharp based pipeline..

Jaskirat Singh

1,376 subscribers

18,719 görüntüleme • 7 ay önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Video Contribution quote from the paper: In summary, our main contributions are • a comprehensive pipeline for reconstructing the shape, appearance, and behavior of real-world garments using Gaussian splatting, • an algorithm for registering garment meshes to multi- view videos with an optimization procedure based on Gaussian splatting, and • a Gaussian Garment representation that combines triangle meshes with Gaussian textures to capture photorealistic appearance and can be used as a fully controllable 3D asset.

Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Video Contribution quote from the paper: In summary, our main contributions are • a comprehensive pipeline for reconstructing the shape, appearance, and behavior of real-world garments using Gaussian splatting, • an algorithm for registering garment meshes to multi- view videos with an optimization procedure based on Gaussian splatting, and • a Gaussian Garment representation that combines triangle meshes with Gaussian textures to capture photorealistic appearance and can be used as a fully controllable 3D asset.

MrNeRF

27,345 görüntüleme • 1 yıl önce

Gaussian Shell Maps are a new neural scene representation that connects fields and 3D Gaussians. This representation unlocks the full potential of 3D Gaussian splatting for generative AI applications, such as 3D avatar generation. 1/2

Gordon Wetzstein

52,480 görüntüleme • 2 yıl önce

Taking “AR” videos and running them through a NeRF pipeline is a fun thing to do. Turns out things look even more 🔥 with Gaussian Splatting. But don’t take my word for it. Behold this beauty:

Taking “AR” videos and running them through a NeRF pipeline is a fun thing to do. Turns out things look even more 🔥 with Gaussian Splatting. But don’t take my word for it. Behold this beauty:

Bilawal Sidhu

136,876 görüntüleme • 3 yıl önce

From 2 second (faster on higher end GPUs) Gaussian splat generations, using Apple's "Sharp Monocular View Synthesis in Less Than a Second" code, running locally on my system. Imported into Octane Render 2026 which has fully path traced rendering of Gaussian splats. Stuff like this is why I've been so excited to see where Gaussian splatting research would go. Combined with a real-time Generative AI "renderer" this stuff could start to get quite insane. Combined with Dreams it would be even better ofc :D Code to the Apple code in the comments below.

From 2 second (faster on higher end GPUs) Gaussian splat generations, using Apple's "Sharp Monocular View Synthesis in Less Than a Second" code, running locally on my system. Imported into Octane Render 2026 which has fully path traced rendering of Gaussian splats. Stuff like this is why I've been so excited to see where Gaussian splatting research would go. Combined with a real-time Generative AI "renderer" this stuff could start to get quite insane. Combined with Dreams it would be even better ofc :D Code to the Apple code in the comments below.

Martin Nebelong

36,774 görüntüleme • 8 ay önce

New paper from Apple - Sharp Monocular View Synthesis in Less than a Second Mescheder et al. @ Apple just released a very impressive paper (congrats! 🎉🥳). You give it an image and it generates a really great looking 3d Gaussian representation. Uses depth pro. It's really good. The model is about 3GB, and takes ~5-10s on my M1 Max per image. Single feedforward pass network. The video is using "Metal Splatter" to view the .ply from ml-sharp on Apple Vision Pro. Some really wow moments today trying it on different scenes.

New paper from Apple - Sharp Monocular View Synthesis in Less than a Second Mescheder et al. @ Apple just released a very impressive paper (congrats! 🎉🥳). You give it an image and it generates a really great looking 3d Gaussian representation. Uses depth pro. It's really good. The model is about 3GB, and takes ~5-10s on my M1 Max per image. Single feedforward pass network. The video is using "Metal Splatter" to view the .ply from ml-sharp on Apple Vision Pro. Some really wow moments today trying it on different scenes.

Tim Davison ᯅ

132,924 görüntüleme • 8 ay önce

RT-Splatting: Joint Reflection-Transmission Modeling with Gaussian Splatting Contributions: • We introduce a unified surface-volume Gaussian scene representation for jointly modeling sharp specular reflections and clear transmission in real-world scenes containing thin semi-transparent surfaces. • We propose Specular-Aware Gradient Gating to suppress misleading gradients from complex specular regions, substantially reducing floaters in the transmission branch. • Extensive experiments demonstrate that RT-Splatting significantly outperforms prior methods while maintaining real-time rendering and enabling flexible scene editing.

RT-Splatting: Joint Reflection-Transmission Modeling with Gaussian Splatting Contributions: • We introduce a unified surface-volume Gaussian scene representation for jointly modeling sharp specular reflections and clear transmission in real-world scenes containing thin semi-transparent surfaces. • We propose Specular-Aware Gradient Gating to suppress misleading gradients from complex specular regions, substantially reducing floaters in the transmission branch. • Extensive experiments demonstrate that RT-Splatting significantly outperforms prior methods while maintaining real-time rendering and enabling flexible scene editing.

MrNeRF

28,330 görüntüleme • 3 ay önce

VAST AI releases Triplane Meets Gaussian Splatting on Hugging Face Fast and Generalizable Single-View 3D Reconstruction with Transformers demo: TGS enables fast reconstruction from single-view image. It builds the 3D representation upon a hybrid Triplane-Gaussian representation by evaluating a transformer-based framework, from which 3D Gaussians would be decoded

VAST AI releases Triplane Meets Gaussian Splatting on Hugging Face Fast and Generalizable Single-View 3D Reconstruction with Transformers demo: TGS enables fast reconstruction from single-view image. It builds the 3D representation upon a hybrid Triplane-Gaussian representation by evaluating a transformer-based framework, from which 3D Gaussians would be decoded

AK

113,242 görüntüleme • 2 yıl önce

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.

MrNeRF

34,782 görüntüleme • 1 yıl önce

$DreamScene360 Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splatting The increasing demand for virtual reality applications has highlighted the significance of crafting immersive 3D assets. We present a text-to-3D 360^{circ} scene generation pipeline$

DreamScene360 Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splatting The increasing demand for virtual reality applications has highlighted the significance of crafting immersive 3D assets. We present a text-to-3D 360^{circ} scene generation pipeline

AK

24,910 görüntüleme • 2 yıl önce

Testing the Insta360 X5 for gaussian splatting. Kensal Green Cemetery, London. Trained in Brush and running around with a PS5 controller in Unity using Aras Pranckevičius 🇺🇦🇱🇹 plugin. #gaussiansplatting #3DGS

Testing the Insta360 X5 for gaussian splatting. Kensal Green Cemetery, London. Trained in Brush and running around with a PS5 controller in Unity using Aras Pranckevičius 🇺🇦🇱🇹 plugin. #gaussiansplatting #3DGS

grade eterna

37,190 görüntüleme • 1 yıl önce

[SIGGRAPH Asia '25] Editable Physically-based Reflections in Raytraced Gaussian Radiance Fields Contributions: • A reconstruction method for radiance fields, with distinct optimization for diffuse and specular components, using path tracing for the latter. • An efficient and accurate training method that reconstructs the diffuse and specular components of the scene in a single representation. • An efficient ray tracer for Gaussian primitives, fast enough to enable treatment of multiple bounces with minimal computational overhead.

[SIGGRAPH Asia '25] Editable Physically-based Reflections in Raytraced Gaussian Radiance Fields Contributions: • A reconstruction method for radiance fields, with distinct optimization for diffuse and specular components, using path tracing for the latter. • An efficient and accurate training method that reconstructs the diffuse and specular components of the scene in a single representation. • An efficient ray tracer for Gaussian primitives, fast enough to enable treatment of multiple bounces with minimal computational overhead.

MrNeRF

15,810 görüntüleme • 10 ay önce

Excited to release our first public AI web app, powered by Apple’s open-source ML-SHARP model, capable of transforming a single image into a 3D Gaussian Splat with real depth understanding in seconds. Try it here → #AI #Apple #VR #GaussianSplatting

Excited to release our first public AI web app, powered by Apple’s open-source ML-SHARP model, capable of transforming a single image into a 3D Gaussian Splat with real depth understanding in seconds. Try it here → #AI #Apple #VR #GaussianSplatting

Niccolò Miranda

201,586 görüntüleme • 7 ay önce

As Hamburg is covered in a thick layer of snow, Jörg Kahlhöfer took the opportunity to capture this street scene with 3D Gaussian Splatting. Watch him try out different lighting setups:

As Hamburg is covered in a thick layer of snow, Jörg Kahlhöfer took the opportunity to capture this street scene with 3D Gaussian Splatting. Watch him try out different lighting setups:

80 LEVEL

48,450 görüntüleme • 7 ay önce

📢Happy to present Convex Splatting, a novel way for 3D reconstruction based on 3D smooth convexes. For the first time, a splatting-based method reaches the quality of NeRF sota methods but with real-time rendering and few primitives!! I expect this to replace Gaussian splatting for 3D in the coming months. CODE RELEASED TODAY! joint work with collaborators from Université de Liège Visual Geometry Group (VGG) , KAUST Computer Vision Lab (IVUL) a thread 🧵 1/n

📢Happy to present Convex Splatting, a novel way for 3D reconstruction based on 3D smooth convexes. For the first time, a splatting-based method reaches the quality of NeRF sota methods but with real-time rendering and few primitives!! I expect this to replace Gaussian splatting for 3D in the coming months. CODE RELEASED TODAY! joint work with collaborators from Université de Liège Visual Geometry Group (VGG) , KAUST Computer Vision Lab (IVUL) a thread 🧵 1/n

Abdullah Hamdi

71,661 görüntüleme • 1 yıl önce

From 2D video to a very rough 4D Gaussian Splatting POC on the Apple Vision Pro 👀 Took Big Buck Bunny frames, turned them into Gaussian splats with Sharp (Apple), compressed them with PlayCanvas splat transform to SOG, and played all frames via WebXR + spark Still rough and not fully optimized, but wow, dynamic GS in AVP feels wild 🤯 Second part of the video is on my laptop #threejs

From 2D video to a very rough 4D Gaussian Splatting POC on the Apple Vision Pro 👀 Took Big Buck Bunny frames, turned them into Gaussian splats with Sharp (Apple), compressed them with PlayCanvas splat transform to SOG, and played all frames via WebXR + spark Still rough and not fully optimized, but wow, dynamic GS in AVP feels wild 🤯 Second part of the video is on my laptop #threejs

Stijn Spanhove

20,127 görüntüleme • 7 ay önce

Behind the scenes VFX for @logitech with Karen X. Cheng! Had a blast getting experimental with scale juxtaposition using 3D Gaussian Splatting and irrealix plugin in After Effects! 🤯✨ #karenception #vfx #bts

Behind the scenes VFX for @logitech with Karen X. Cheng! Had a blast getting experimental with scale juxtaposition using 3D Gaussian Splatting and irrealix plugin in After Effects! 🤯✨ #karenception #vfx #bts

Kody Kurth

16,316 görüntüleme • 1 yıl önce

VPS tracking on smart glasses Experimenting with MultiSet AI’s Visual Positioning System (VPS) to determine a person’s position relative to a 3d scanned environment, using just an image Works with compatible smartglasses, in this case Omi AI’s Omi Glass (based on Seeed Studio’s ESP32-S3 Sense) running our custom firmware

VPS tracking on smart glasses Experimenting with MultiSet AI’s Visual Positioning System (VPS) to determine a person’s position relative to a 3d scanned environment, using just an image Works with compatible smartglasses, in this case Omi AI’s Omi Glass (based on Seeed Studio’s ESP32-S3 Sense) running our custom firmware

Brilliant Wear

13,437 görüntüleme • 5 ay önce

I'm testing tree generation using only two reference images. Basically, my Gaussian Splatting generation technique works with almost anything. I can't wait to see the final result once I’ve gathered a huge library of realistic assets for my game. For optimization, I’m planning to use a LOD system that dynamically reduces the number of Gaussian splats based on the distance from the camera.

I'm testing tree generation using only two reference images. Basically, my Gaussian Splatting generation technique works with almost anything. I can't wait to see the final result once I’ve gathered a huge library of realistic assets for my game. For optimization, I’m planning to use a LOD system that dynamically reduces the number of Gaussian splats based on the distance from the camera.

Alex

30,539 görüntüleme • 3 ay önce

Experimenting with wrist controls on smart glasses Using our Wearables SDK, we can display a 3D wireframe A-Frame scene on compatible smartglasses (in this case the Brilliant Labs Frame running our custom firmware) We’re also using our SDK to control a virtual cursor with our Ukaton motion module strapped to our wrist, running an Edge Impulse model to detect pinches There’s a lot of third party hand control accessories we could use (TapWithUs, Doublepoint, and Mudra Band), but using our own hardware/software is great for creating custom gestures

Experimenting with wrist controls on smart glasses Using our Wearables SDK, we can display a 3D wireframe A-Frame scene on compatible smartglasses (in this case the Brilliant Labs Frame running our custom firmware) We’re also using our SDK to control a virtual cursor with our Ukaton motion module strapped to our wrist, running an Edge Impulse model to detect pinches There’s a lot of third party hand control accessories we could use (TapWithUs, Doublepoint, and Mudra Band), but using our own hardware/software is great for creating custom gestures

Brilliant Wear

29,397 görüntüleme • 10 ay önce