The first public dynamic Gaussian Splatting implementation in VR... is here from San Francisco based Gmix, compatible with WebXR and Apple Vision Pro. 🔗show more

splaTV, developed by Kevin Kwok, is a browser-based implementation... for experiencing dynamic Gaussian Splatting with Spacetime Gaussians. This newly released application supports both WebGL and WebXR technologies and is available under the MIT License. 🔗show more

MoE-GS: Mixture of Experts for Dynamic Gaussian Splatting Contributions:... • MoE-GS: the first dynamic Gaussian splatting framework employing a Mixture-of-Experts architecture, enabling robust and adaptive reconstruction across diverse dynamic scenes. • A novel Volume-aware Pixel Router integrates expert outputs through differentiable weight splatting, achieving spatially and temporally coherent adaptive blending. • Efficiency of MoE-GS is improved through single-pass multi-expert rendering and gate-aware Gaussian pruning. A separate knowledge distillation strategy trains individual experts with pseudo-labels from the MoE model, enhancing quality without modifying the architecture.show more

Some personal news! 📣 Recently, I joined Niantic as... the product lead to work on MR apps and Gaussian splatting! Today, we’re launching “Into The Scaniverse”—A way of viewing Scaniverse splats in VR (yours and the ones posted to the map) 📸 It’s available as a webxr app via Quest Browser (for Q3 and Q3S), soon to launch in the Meta Horizon store. Honored to be working with quite a lot of brilliant folks at Niantic 🙌 Link to the webxr app 👇show more

This is Alibaba's TaoAvatar. A Full-Body, Photorealistic Virtual Avatar... for Apple Vision Pro. It's real-time. Spatial Telepresence will be the killer use-case of VR and AR, Period.show more

The first rain of the season is here—and San... Francisco Public Works is working hard to keep our drains clear and our streets safe. You can do your part too by joining SF Water Power Sewer’s Adopt-a-Drain program. It’s a rewarding way to help the environment and prevent flooding. Stay safe and dry, San Francisco!show more

Apple just trained a 3D Gaussian head reconstruction model... on 10,000+ subjects. Feed-forward. No test-time optimization. New identity in, reconstructed Gaussian head out. The UV-parameterized Gaussian representation decouples the number of Gaussians from the number and resolution of input images, making it practical to train with many high resolution views. And the heads are not just static either: text-conditioned identity generation, plus blendshape-driven latent animation across identities. We've been building in the 3D Gaussian Splatting space for a while. The gap between "research demo" and "works on real people at scale" is closing fast.show more

Public service announcement: The new Metallica Immersive Concert Experience... is out early and available now on Vision Pro! 25 minutes of stunning visuals and incredible spatial audio. Apple is cooking!show more

It’s almost here... Check out #NintendoSwitch2 during events at... Nintendo NEW YORK and Nintendo SAN FRANCISCO tomorrow, June 4! 📍Nintendo NEW YORK: 3pm–8pm ET 📍Nintendo SAN FRANCISCO: 1–6pm PT The events are first come first serve. Capacity is limited.show more

Apple is adding a new feature to iOS that... is designed to counteract motion sickness while looking at a screen in a moving vehicle. I'm very curious if they're thinking of the implications of this for VR experiences on Vision Pro.show more

Muckingham is in dire need of a clean! 💦... Wash 3 new levels in PowerWash Simulator Apple Arcade: - Food Truck - Solar Station - Satellite Dish The Muckingham Files 1 is available now on iPhone, iPad, Mac, Apple TV and Apple Vision! 🔗show more

visionOS 26.4 is now available. NVIDIA CloudXR 6.0 is... here. Stream directly from an RTX PC to Vision Pro at 4K 120Hz with foveated streaming. Apple Podcasts now supports video. Spatial audio that remembers your room acoustics. AirPods Max 2 support. And more.show more

Why do we have barely any of our team... in San Francisco today? "Our team in San Francisco is maybe 4% of the company, even though that's where we're founded and where I'm based. Cost and loyalty. Retention of good people has always been really hard." Ryan Petersenshow more

Self-Calibrating Gaussian Splatting for Large Field of View Reconstruction... Note: Check below for full video. Abstract (cited): "In this paper, we present a self-calibrating framework that jointly optimizes camera parameters, lens distortion, and 3D Gaussian representations, enabling accurate and efficient scene reconstruction. Our technique is particularly effective for high-quality scene reconstruction from large field-of-view (FOV) imagery taken with wide-angle lenses, allowing the scene to be modeled from a smaller number of images. We introduce a novel method for modeling complex lens distortions using a hybrid network that combines invertible residual networks with explicit grids. This design effectively regularizes the optimization process, achieving greater accuracy than conventional camera models. Additionally, we propose a cubemap-based resampling strategy to support large FOV images without sacrificing resolution or introducing distortion artifacts. Our method is compatible with the fast rasterization of Gaussian Splatting, adaptable to a wide variety of camera lens distortions, and demonstrates state-of-the-art performance on both synthetic and real-world datasets."show more

DroneSplat: 3D Gaussian Splatting for Robust 3D Reconstruction from... In-the-Wild Drone Imagery Abstract: Drones have become essential tools for reconstructing wild scenes due to their outstanding maneuverability. Recent advances in radiance field methods have achieved remarkable rendering quality, providing a new avenue for 3D reconstruction from drone imagery. However, dynamic distractors in wild environments challenge the static scene assumption in radiance fields, while limited view constraints hinder the accurate capture of underlying scene geometry. To address these challenges, we introduce DroneSplat, a novel framework designed for robust 3D reconstruction from in-the-wild drone imagery. Our method adaptively adjusts masking thresholds by integrating local-global segmentation heuristics with statistical approaches, enabling precise identification and elimination of dynamic distractors in static scenes. We enhance 3D Gaussian Splatting with multi-view stereo predictions and a voxel-guided optimization strategy, supporting high-quality rendering under limited view constraints. For comprehensive evaluation, we provide a drone-captured 3D reconstruction dataset encompassing both dynamic and static scenes. Extensive experiments demonstrate that DroneSplat outperforms both 3DGS and NeRF baselines in handling in-the-wild drone imagery.show more

Single video → a reframeable 4D Gaussian Splatting scene.... Not a sequence of separately built 3D frames played back like a video. This is one continuous space-time scene, reconstructed from a single clip shot on an iPhone 16. We combine feed-forward Gaussian generation, 3D tracking, and 4D Gaussian Splatting, aiming to deliver it as a compact 4D video file that runs on your phone. Still early R&D. The goal: make 4DGS something anyone can create and experience, not just researchers with a camera rig.show more

You’ll be able to watch the WWDC livestream on... Apple Vision Pro with spatial backdrops, interactive 3D models and the ability to post reactions in real-time. Source: Sceneryshow more

PackUV: Packed Gaussian UV Maps for 4D Volumetric Video... - PackUV — A new volumetric video representation that packs 3D Gaussian attributes into a sequence of UV atlases for efficient streaming and storage, making it readily compatible with existing video coding infrastructure. - PackUV-GS — An efficient method to fit PackUV directly from multiview videos using optical-flow-based keyframing and Gaussian labeling to handle large motions, disocclusions, and temporal consistency. - PackUV-2B — The largest multi-view 4D dataset with 2B frames, large motions, and disocclusions. It provides 360° coverage from 50+ synchronized cameras.show more

🙌 Apple Vision Pro seems to perfectly support direct... touch, which is a *fantastic* way to interact with digital content up close and personal, as well as typing with the keyboard. #WWDC23show more