Loading video...

Video Failed to Load

Go Home

Lift4D Harmonizing Single-View 3D Estimation for 4D Reconstruction In-the-Wild

_akhaliq's profile picture

AK

530,898 subscribers

31,896 views • 3 months ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Discover the right 3D Geometric Foundation Model for your task—whether it’s stereo matching, multi-view depth estimation, video depth, pose estimation, semantic understanding, or novel view synthesis. Explore more insights in our #E3DBench #FoundationModel #3D #GaussianSplatting. Project Webpage:
0:14

Discover the right 3D Geometric Foundation Model for your task—whether it’s stereo matching, multi-view depth estimation, video depth, pose estimation, semantic understanding, or novel view synthesis. Explore more insights in our #E3DBench #FoundationModel #3D #GaussianSplatting. Project Webpage:

Zhiwen(Aaron) Fan

30,181 views • 1 year ago

DroneSplat: 3D Gaussian Splatting for Robust 3D Reconstruction from In-the-Wild Drone Imagery Abstract: Drones have become essential tools for reconstructing wild scenes due to their outstanding maneuverability. Recent advances in radiance field methods have achieved remarkable rendering quality, providing a new avenue for 3D reconstruction from drone imagery. However, dynamic distractors in wild environments challenge the static scene assumption in radiance fields, while limited view constraints hinder the accurate capture of underlying scene geometry. To address these challenges, we introduce DroneSplat, a novel framework designed for robust 3D reconstruction from in-the-wild drone imagery. Our method adaptively adjusts masking thresholds by integrating local-global segmentation heuristics with statistical approaches, enabling precise identification and elimination of dynamic distractors in static scenes. We enhance 3D Gaussian Splatting with multi-view stereo predictions and a voxel-guided optimization strategy, supporting high-quality rendering under limited view constraints. For comprehensive evaluation, we provide a drone-captured 3D reconstruction dataset encompassing both dynamic and static scenes. Extensive experiments demonstrate that DroneSplat outperforms both 3DGS and NeRF baselines in handling in-the-wild drone imagery.
0:19

DroneSplat: 3D Gaussian Splatting for Robust 3D Reconstruction from In-the-Wild Drone Imagery Abstract: Drones have become essential tools for reconstructing wild scenes due to their outstanding maneuverability. Recent advances in radiance field methods have achieved remarkable rendering quality, providing a new avenue for 3D reconstruction from drone imagery. However, dynamic distractors in wild environments challenge the static scene assumption in radiance fields, while limited view constraints hinder the accurate capture of underlying scene geometry. To address these challenges, we introduce DroneSplat, a novel framework designed for robust 3D reconstruction from in-the-wild drone imagery. Our method adaptively adjusts masking thresholds by integrating local-global segmentation heuristics with statistical approaches, enabling precise identification and elimination of dynamic distractors in static scenes. We enhance 3D Gaussian Splatting with multi-view stereo predictions and a voxel-guided optimization strategy, supporting high-quality rendering under limited view constraints. For comprehensive evaluation, we provide a drone-captured 3D reconstruction dataset encompassing both dynamic and static scenes. Extensive experiments demonstrate that DroneSplat outperforms both 3DGS and NeRF baselines in handling in-the-wild drone imagery.

MrNeRF

21,391 views • 1 year ago

Learn how to combine a deep learning model for pose estimation to perform a 3D reconstruction using two cameras 📸
0:13

Learn how to combine a deep learning model for pose estimation to perform a 3D reconstruction using two cameras 📸

MATLAB

15,704 views • 1 year ago

How can we use test-time compute for spatial understanding? 🤔 In InterPose, we propose to repeatedly sample generative video models to help two-view pose estimation and reconstruction, by leveraging the video models' keyframe interpolation abilities. A 🧵... (1/8)
0:17

How can we use test-time compute for spatial understanding? 🤔 In InterPose, we propose to repeatedly sample generative video models to help two-view pose estimation and reconstruction, by leveraging the video models' keyframe interpolation abilities. A 🧵... (1/8)

Ricardo Martin-Brualla

21,685 views • 1 year ago

ReViV reconstructs viewer-centric human motion (body, hand, and gaze) and view-centric scene geometry (camera and depth) from a single egocentric RGB video in a unified feed-forward model. It formulates the task as learning the full joint probability distribution over multimodal signals, including RGB video, camera trajectory, gaze direction, full-body motion, hand motion, and depth. Powered by a Masked Generative Egocentric Transformer, ReViV operates within a single feed-forward architecture to simultaneously reconstruct the temporally consistent 4D reconstruction across the viewer and the view with fast inference speed. Paper Title: ReViV: Reconstructing the Viewer and the View in 4D from Monocular Project: Link:
0:14

ReViV reconstructs viewer-centric human motion (body, hand, and gaze) and view-centric scene geometry (camera and depth) from a single egocentric RGB video in a unified feed-forward model. It formulates the task as learning the full joint probability distribution over multimodal signals, including RGB video, camera trajectory, gaze direction, full-body motion, hand motion, and depth. Powered by a Masked Generative Egocentric Transformer, ReViV operates within a single feed-forward architecture to simultaneously reconstruct the temporally consistent 4D reconstruction across the viewer and the view with fast inference speed. Paper Title: ReViV: Reconstructing the Viewer and the View in 4D from Monocular Project: Link:

AI Bites | YouTube Channel

18,590 views • 2 months ago

random model find; might be the best 3D multi-person pose estimation I have ever seen; it's Apache-2.0 Multi-HMR: Regressing Whole-Body Human Meshes for Multiple Persons in a Single Shot; ECCV2024
0:17

random model find; might be the best 3D multi-person pose estimation I have ever seen; it's Apache-2.0 Multi-HMR: Regressing Whole-Body Human Meshes for Multiple Persons in a Single Shot; ECCV2024

SkalskiP

31,150 views • 1 year ago

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.
0:12

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

MrNeRF

52,849 views • 1 year ago

Announcing 2 new members of the *St3R family for human-centric 3D vision tasks! Meet HAMst3R & HOSt3R #ICCV2025 #iccv2025 - HAMSt3R (Human-Aware Multi-view Stereo 3D Reconstruction) extends MASt3R to handle scenes involving people. Paper: 1/2 🧵
0:10

Announcing 2 new members of the *St3R family for human-centric 3D vision tasks! Meet HAMst3R & HOSt3R #ICCV2025 #iccv2025 - HAMSt3R (Human-Aware Multi-view Stereo 3D Reconstruction) extends MASt3R to handle scenes involving people. Paper: 1/2 🧵

NAVER LABS Europe

14,094 views • 1 year ago

MoVieS Motion-Aware 4D Dynamic View Synthesis in One Second
0:14

MoVieS Motion-Aware 4D Dynamic View Synthesis in One Second

AK

29,624 views • 1 year ago

Made it in one single prompt in Cinema 4D 😎
0:12

Made it in one single prompt in Cinema 4D 😎

Anton Skvortsov

53,189 views • 4 months ago

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.
0:18

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

MrNeRF

17,062 views • 1 year ago

Apple Vision Pro - 3D Product Commercial made in Cinema 4D & Redshift.
0:19

Apple Vision Pro - 3D Product Commercial made in Cinema 4D & Redshift.

DarkSun

18,595 views • 1 year ago

(1/2) Intrinsic Image Diffusion for Single-view Material Estimation! We propose a probabilistic diffusion model to handle material & lighting ambiguities. We obtain sharp material estimates and facilitate high-fidelity relighting.
0:19

(1/2) Intrinsic Image Diffusion for Single-view Material Estimation! We propose a probabilistic diffusion model to handle material & lighting ambiguities. We obtain sharp material estimates and facilitate high-fidelity relighting.

Matthias Niessner

69,400 views • 2 years ago

AI is so trippy. This entire scene, including the exterior, was created from a single image of the ovens in the interior. This is a multi-view reconstruction, not a video. No v2v used. Prompt image in reply.
0:18

AI is so trippy. This entire scene, including the exterior, was created from a single image of the ovens in the interior. This is a multi-view reconstruction, not a video. No v2v used. Prompt image in reply.

martin_casado

54,371 views • 1 year ago

Do 3D reconstruction transformers really need a billion parameters, or are most of those layers just doing the same thing over and over? Introducing Déjà View: a single transformer block, looped K times, that matches or beats models 8–10× its size with lower compute. 🧵
0:13

Do 3D reconstruction transformers really need a billion parameters, or are most of those layers just doing the same thing over and over? Introducing Déjà View: a single transformer block, looped K times, that matches or beats models 8–10× its size with lower compute. 🧵

Tobias Fischer

112,876 views • 4 months ago