Single video → a reframeable 4D Gaussian Splatting scene.... Not a sequence of separately built 3D frames played back like a video. This is one continuous space-time scene, reconstructed from a single clip shot on an iPhone 16. We combine feed-forward Gaussian generation, 3D tracking, and 4D Gaussian Splatting, aiming to deliver it as a compact 4D video file that runs on your phone. Still early R&D. The goal: make 4DGS something anyone can create and experience, not just researchers with a camera rig.show more

F3D-Gaus: Feed-forward 3D-aware Generation on ImageNet with Cycle-Consistent Gaussian... Splatting Contributions: • We pioneer 3D-aware generation using generalizable feed-forward Gaussian Splatting representation, achieving significant efficiency and favorable rendering quality on monocular datasets. • We significantly advance the capability of pixel-aligned Gaussian Splatting representations by designing a self-supervised cycle training strategy specifically tailored for monocular datasets. • We further mitigate the artifacts of 3D-aware representations caused by large viewpoint shifts by introducing geometry-aware video priors.show more

PackUV: Packed Gaussian UV Maps for 4D Volumetric Video... - PackUV — A new volumetric video representation that packs 3D Gaussian attributes into a sequence of UV atlases for efficient streaming and storage, making it readily compatible with existing video coding infrastructure. - PackUV-GS — An efficient method to fit PackUV directly from multiview videos using optical-flow-based keyframing and Gaussian labeling to handle large motions, disocclusions, and temporal consistency. - PackUV-2B — The largest multi-view 4D dataset with 2B frames, large motions, and disocclusions. It provides 360° coverage from 50+ synchronized cameras.show more

Apple just trained a 3D Gaussian head reconstruction model... on 10,000+ subjects. Feed-forward. No test-time optimization. New identity in, reconstructed Gaussian head out. The UV-parameterized Gaussian representation decouples the number of Gaussians from the number and resolution of input images, making it practical to train with many high resolution views. And the heads are not just static either: text-conditioned identity generation, plus blendshape-driven latent animation across identities. We've been building in the 3D Gaussian Splatting space for a while. The gap between "research demo" and "works on real people at scale" is closing fast.show more

DimensionX: Create Any 3D and 4D Scenes from a... Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.show more

Huawei's new 3D lock screen wallpaper is actually pretty... interesting. Unlike iPhone's Spatial Photo, which creates a fixed stereoscopic depth effect from a photo, this lets you scan an object 360°, generate a full 3D Gaussian Splatting asset, and use that as your lock screen. Tilt the phone and you're moving through a reconstructed 3D scene, not a fixed stereo image. The part worth paying attention to: 3D capture moving out of creative pipelines and into everyday phone personalization. That's a different kind of adoption curve.show more

IronSight turns 2D range footage into a replayable 4D... world -- with shot detection, 3D target tracking, and a camera you can move anywhere. Seeing my own runs reconstructed from every angle is ridiculously fun. Can't wait till we can run this tech in real time.show more

48 consistent camera feeds of me all from one... casual video capture. This is 4DAnyone. I am testing it out over the holiday week, the goal is 4D Gaussian Splatting from a single camera. Should out to Pablo Vela' for inspiring me to use Rerun to visualize my progress. Right now you see 6 cameras, but I can cycle through all 48. I hacked together this visualization using ChatGPT's new Astra model in VS Code. Literally took less than 10 minutes. More coming soon. Learn more about 4DAnyone Here: You can also try Pablo's 6 view 4DAnyone generator on Hugging Face here: Jeremy Park, PhD Robert Scoble #Computervision #3D #4DGSshow more

We just open-sourced 3DGS render blender addon v4.0 (... It has been a while, but we are still updating this 3D Gaussian Splatting tool! In this version, we added a real-time rendering mode, so say goodbye to those laggy displays in the viewport! Here is an example: I imported a file that contains one million gaussians, super smooth.show more

Wonderland: Navigating 3D Scenes from a Single Image Contributions:... • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.show more

ReViV reconstructs viewer-centric human motion (body, hand, and gaze)... and view-centric scene geometry (camera and depth) from a single egocentric RGB video in a unified feed-forward model. It formulates the task as learning the full joint probability distribution over multimodal signals, including RGB video, camera trajectory, gaze direction, full-body motion, hand motion, and depth. Powered by a Masked Generative Egocentric Transformer, ReViV operates within a single feed-forward architecture to simultaneously reconstruct the temporally consistent 4D reconstruction across the viewer and the view with fast inference speed. Paper Title: ReViV: Reconstructing the Viewer and the View in 4D from Monocular Project: Link:show more

WeatherEdit: Controllable Weather Editing with 4D Gaussian Field Contributions:... 1. Based on our analysis of weather editing characteristics, we introduce WeatherEdit, a comprehensive and efficient framework for realistic and controllable weather generation. Compared with existing methods that focus on either background editing or static weather effects, a progressive 2D-to-4D transformation process in WeatherEdit enhances adaptability across a wider range of scenarios. 2. We introduce an all-in-one adapter to enable a diffusion model for multi-weather (snowy, rainy, and fog) synthesis, along with a Temporal-View attention to ensure consistent editing across multi-frame and multi-view. 3. We design a 4D Gaussian field for weather particle modeling, enabling plausible simulation of raindrops, snowflakes, and fog with controllable severity. 4. We demonstrate WeatherEdit’s effectiveness in generating realistic, consistent, and controllable weather effects in 3D driving scenes, showcasing its applicability to real-world scenarios.show more

China just dropped an open source model that turns... any phone into a 3D scanner you point it at a room and walk through it the whole space builds itself into a 3D point cloud in real time as the camera moves, no LiDAR and no depth sensor anywhere it held a full 13 minute walkthrough at 25,000 frames without the map drifting or collapsing > ~20 fps on a single GPU > works indoors and outdoors from plain footage > the scan normally needs a $50,000 rig and a technician > Apache 2.0 and runs fully offline on your machine repo in the replyshow more

Take a Gaussian splat, a cutting-edge way of capturing... real-world scenes in 3D, and transform it into an entirely new form using custom-built tools. Then render it in OTOY Octane on The Render Network, that's "Hyperdimensional Breach" by Matteo Scappin (Machina-Infinitum). Using the ARTK plugin for Maxon Cinema 4D, Matteo layers audio-reactive fractal patterns (mesmerizing, endlessly repeating patterns) onto the captured scene, combining multiple custom workflows into visuals that pulse and evolve with the music.show more

First fully ML-framework-free 3D Gaussian Splatting implementation in LichtFeld... Studio. I’ve completed the migration of the full training pipeline to a custom CUDA-based tensor library. No PyTorch, no LibTorch, no autograd. Every gradient is implemented by hand, either through CUDA kernels or minimal abstractions on top. This makes it the first full training setup for 3D Gaussian Splatting with zero dependencies on existing ML frameworks. It’s not just about independence, it's about control! We now manage every byte of GPU memory, which opens the door to tighter optimization and finer performance tuning. The framework footprint is minimal, without pulling in gigabytes of ML runtime code that was never designed for real-time or graphics-driven applications. A few modules, such as the metrics and 3DGUT interfaces, are still being ported, and some operations are temporarily naïve, so performance is not yet on par with master. But this refactor lays the groundwork for: - A fully self-contained binary - Fine-grained memory optimization - Easier experimentation without the weight of an ML stack We’re getting close.show more

GSTAR: Gaussian Surface Tracking and Reconstruction Contributions: • A... new framework for tracking and reconstructing dynamic scenes, combining 3D Gaussians and meshes to effectively manage changes in topology. • A method for Gaussian unbinding and surface re-meshing, allowing for the generation of new surfaces as topologies evolve. • A method for handling large or fast deformations of surfaces between frames using scene flow warping. Abstract (excerpt): However, tracking dynamic surfaces with 3D Gaussians remains challenging due to complex topology changes, such as surfaces appearing, disappearing, or splitting. To address these challenges, we propose GSTAR, a novel method that achieves photo-realistic rendering, accurate surface reconstruction, and reliable 3D tracking for general dynamic scenes with changing topology. Given multi-view captures as input, GSTAR binds Gaussians to mesh faces to represent dynamic objects. For surfaces with consistent topology, GSTAR maintains the mesh topology and tracks the meshes using Gaussians.show more