Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

A video is compressed 4D data. 🌍 Today, we’re sharing a preview of our latest research: 4D reconstruction from a single monocular video. 🎬 From one casually captured video, our system reconstructs a dynamic, freely viewable 4D Gaussian scene—recovering geometry, appearance, camera motion, and object motion across space and...

InSpatio_AI's profile picture

InSpatio｜影溯

1,486 subscribers

64,486 görüntüleme • 1 ay önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

18 Yorum

Liam | AI Tools & News profil fotoğrafı
Liam | AI Tools & News1 ay önce

Turning a single monocular video into a freely viewable 4D scene is a remarkable achievement.

inspire369 profil fotoğrafı
inspire3691 ay önce

Link to GitHub repo?

InSpatio｜影溯 profil fotoğrafı
InSpatio｜影溯1 ay önce

we will open source it, we’re preparing more technical details and results; stay tuned!

Toby VirtualFilmer profil fotoğrafı
Toby VirtualFilmer1 ay önce

Very interesting. I’d be interested in seeing the results with two-actor drama scenes, captured by a single camera. Does it do a good job with faces that are obscured? Does it fill in the lip sync?

InSpatio｜影溯 profil fotoğrafı
InSpatio｜影溯1 ay önce

Great question. multi-person interactions, occlusion, facial detail, and lip synchronization are among the hardest cases. We’re actively testing these scenarios and will share more results as the system improves :)

mr.tipton profil fotoğrafı
mr.tipton1 ay önce

This is awesome. I’ve always assumed it could be done with way less cameras than what I’ve typically seen for video capture, but didn’t realize it’d be just one. I bet a stereo configuration with two cameras would clean the noise right up.

Prannay Hebbar✧ profil fotoğrafı
Prannay Hebbar✧1 ay önce

braindance?

lost in latency profil fotoğrafı
lost in latency1 ay önce

this is incredible

Damian Morley profil fotoğrafı
Damian Morley1 ay önce

I’m working on this right now! Haha looks great!

InSpatio｜影溯 profil fotoğrafı
InSpatio｜影溯1 ay önce

Love to hear that:) would be great to compare notes. Monocular 4D reconstruction in the wild still has so many interesting challenges😆

hanqing profil fotoğrafı
hanqing1 ay önce

以后拍vlog直接给AI，它连你下次在哪里摔倒都算好了。

revolver ocelot profil fotoğrafı
revolver ocelot1 ay önce

This is phenomenal.

Aryan profil fotoğrafı
Aryan1 ay önce

woah, very interesting

Isaac Bratzel profil fotoğrafı
Isaac Bratzel1 ay önce

Wild

L$ profil fotoğrafı
L$1 ay önce

Great

Andrew M profil fotoğrafı
Andrew M1 ay önce

give it a year or two @tokyomegaplex !

Breakdown profil fotoğrafı
Breakdown1 ay önce

WOW

Film4EVER profil fotoğrafı
Film4EVER1 ay önce

Insane...

Benzer Videolar

ReViV: Reconstructing the Viewer and the View in 4D from Monocular Egocentric Video(ECCV2026) ReViV reconstructs viewer-centric human motion (body, hand, and gaze) and view-centric scene geometry (camera and depth) from a single egocentric RGB video in a unified feed-forward model. For visualization, the initial camera pose and camera intrinsics are estimated with GeoCalib.
0:54

ReViV: Reconstructing the Viewer and the View in 4D from Monocular Egocentric Video(ECCV2026) ReViV reconstructs viewer-centric human motion (body, hand, and gaze) and view-centric scene geometry (camera and depth) from a single egocentric RGB video in a unified feed-forward model. For visualization, the initial camera pose and camera intrinsics are estimated with GeoCalib.

Ryohei Sasaki@engineer

15,096 görüntüleme • 21 gün önce

We’ve upgraded Stable Video Diffusion 4D to Stable Video 4D 2.0 (SV4D 2.0), improving the quality of 4D outputs generated from a single object-centric video. While 3D provides a static view of an object’s shape and size; 4D extends this by including time, showing how the object moves. This multi-view video diffusion model generates a 4D output in three steps: 1️⃣ Starts with an input video of a moving person or object 2️⃣ Generates novel views of the subject from unseen angles 3️⃣ Constructs a single dynamic 4D video output with spatial and temporal consistency You can learn more here: (1/4)
0:30

We’ve upgraded Stable Video Diffusion 4D to Stable Video 4D 2.0 (SV4D 2.0), improving the quality of 4D outputs generated from a single object-centric video. While 3D provides a static view of an object’s shape and size; 4D extends this by including time, showing how the object moves. This multi-view video diffusion model generates a 4D output in three steps: 1️⃣ Starts with an input video of a moving person or object 2️⃣ Generates novel views of the subject from unseen angles 3️⃣ Constructs a single dynamic 4D video output with spatial and temporal consistency You can learn more here: (1/4)

Stability AI

36,059 görüntüleme • 1 yıl önce

Excited to share MonST3R! -- a simple way to estimate geometry from unposed video of dynamic scene We achieve competitive results on several downstreams (video depth, camera pose) and believe this is a promising step toward feed-forward 4D reconstruction
1:03

Excited to share MonST3R! -- a simple way to estimate geometry from unposed video of dynamic scene We achieve competitive results on several downstreams (video depth, camera pose) and believe this is a promising step toward feed-forward 4D reconstruction

Junyi Zhang

131,690 görüntüleme • 1 yıl önce

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.
5:19

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.

MrNeRF

34,782 görüntüleme • 1 yıl önce

Thats awesome! A developer used Al-powered 4D Gaussian Splatting to convert flat video footage into dynamic 3D spatial scenes. The system reconstructs different camera angles and depth information from ordinary footage, making it possible to navigate scenes in three dimensions. We're moving from recording video to digitally recreating reality itself.
1:10

Thats awesome! A developer used Al-powered 4D Gaussian Splatting to convert flat video footage into dynamic 3D spatial scenes. The system reconstructs different camera angles and depth information from ordinary footage, making it possible to navigate scenes in three dimensions. We're moving from recording video to digitally recreating reality itself.

SciTech Era

73,502 görüntüleme • 3 ay önce

4D videos, are we cooked? 😳 uses 4D Gaussian Splatting to reconstruct real footage into fully explorable scenes, letting you move the camera freely through a moment after it’s been filmed. Combine it with tools like Seedance for character replacement and motion control, and Beeble for relighting, and you unlock entire cinematic workflows. You can even step inside your scenes in augmented reality with the Apple Vision Pro. #4DVAI #GaussianSplatting #AIVideoWorkflow #CreativeAI #SpatialVideo
0:40

4D videos, are we cooked? 😳 uses 4D Gaussian Splatting to reconstruct real footage into fully explorable scenes, letting you move the camera freely through a moment after it’s been filmed. Combine it with tools like Seedance for character replacement and motion control, and Beeble for relighting, and you unlock entire cinematic workflows. You can even step inside your scenes in augmented reality with the Apple Vision Pro. #4DVAI #GaussianSplatting #AIVideoWorkflow #CreativeAI #SpatialVideo

Rourke Heath

69,646 görüntüleme • 2 ay önce

4DGS volumetric video captured with iPhones. 📱 Until now, capturing 4D Gaussian Splats meant million-dollar studios and fixed camera rigs. Radiant just changed the game - genlock-syncing multiple iPhone 17 Pro Max cameras with Tilta Chronos cage and Blackmagic ProDock into one fully wireless mobile capture rig. Pixel-level sync, frame by frame, real time. Apple Vision Pro native playback? We are working on it. But the capture side? Solved (affordable-ish). And it travels. The next great VolCap film could be coming from someone like you! #4DGS #volumetricvideo
0:58

4DGS volumetric video captured with iPhones. 📱 Until now, capturing 4D Gaussian Splats meant million-dollar studios and fixed camera rigs. Radiant just changed the game - genlock-syncing multiple iPhone 17 Pro Max cameras with Tilta Chronos cage and Blackmagic ProDock into one fully wireless mobile capture rig. Pixel-level sync, frame by frame, real time. Apple Vision Pro native playback? We are working on it. But the capture side? Solved (affordable-ish). And it travels. The next great VolCap film could be coming from someone like you! #4DGS #volumetricvideo

Hugh Hou

12,619 görüntüleme • 5 ay önce

This will change how sports content gets made and delivered. The method is Gaussian splatting. It takes video from multiple camera positions and rebuilds the whole thing in three dimensions, viewable from any angle. Instead of polygons (the way games render), it builds the scene from millions of tiny glowing splats of color and depth. Arcturus runs it across time (4D), so a live game becomes a 3D scene, shown from field level, beside a player, or above the play.
0:23

This will change how sports content gets made and delivered. The method is Gaussian splatting. It takes video from multiple camera positions and rebuilds the whole thing in three dimensions, viewable from any angle. Instead of polygons (the way games render), it builds the scene from millions of tiny glowing splats of color and depth. Arcturus runs it across time (4D), so a live game becomes a 3D scene, shown from field level, beside a player, or above the play.

James Morrison Rubin

68,751 görüntüleme • 16 gün önce

We're helping AI to see the 3D world in motion as humans do. 🌐 Enter D4RT: a unified model that turns video into 4D representations faster than previous methods - enabling it to understand space and time. This is how it works 🧵
0:29

We're helping AI to see the 3D world in motion as humans do. 🌐 Enter D4RT: a unified model that turns video into 4D representations faster than previous methods - enabling it to understand space and time. This is how it works 🧵

Google DeepMind

369,707 görüntüleme • 7 ay önce

We don’t generate videos. 🎬 We generate worlds from videos. 🌍 Introducing InSpatio-World — the world's first open-source real-time 4D world model‼️ Your input: a video clip Our output: a dynamic, navigable, persistent world 🕹️ explore freely across viewpoints ⏪ control time forward and backward 🔓 open-source and ready to build on :) Live demo: 🔗 Code & weights: 🔗 Project page: 🔗
1:32

We don’t generate videos. 🎬 We generate worlds from videos. 🌍 Introducing InSpatio-World — the world's first open-source real-time 4D world model‼️ Your input: a video clip Our output: a dynamic, navigable, persistent world 🕹️ explore freely across viewpoints ⏪ control time forward and backward 🔓 open-source and ready to build on :) Live demo: 🔗 Code & weights: 🔗 Project page: 🔗

InSpatio｜影溯

110,107 görüntüleme • 6 ay önce

"More Doggo than Doggo" Motion scanned @ 60fps. Playing back @ 20fps. Meet our inhouse tech Doggo "Rolo". Since July we've been redesigning our scanning pipeline to work with the new 3D Gaussian Splatting for Real-Time Radiance Field Rendering method from Inria. GS has reignited our passion for 3D/4D scanning. Thank you to the team at Inria IR's AeonX capture system has been designed to capture "Spatial" memories for the future. We've built on a custom version of Inria's Sibr viewer to handle the playback of our multiple processed 4D frames. Our next goal is to optimize the player to be multi-threaded to improve performance over 60fps. GS has reignited our passion for 3D/4D scanning. Thank you to the team at Inria More content to come. #guassiansplatting #ir #aeonx #4d #motionscanning #inria #ximea #idatronic #spatial #youth83
0:41

"More Doggo than Doggo" Motion scanned @ 60fps. Playing back @ 20fps. Meet our inhouse tech Doggo "Rolo". Since July we've been redesigning our scanning pipeline to work with the new 3D Gaussian Splatting for Real-Time Radiance Field Rendering method from Inria. GS has reignited our passion for 3D/4D scanning. Thank you to the team at Inria IR's AeonX capture system has been designed to capture "Spatial" memories for the future. We've built on a custom version of Inria's Sibr viewer to handle the playback of our multiple processed 4D frames. Our next goal is to optimize the player to be multi-threaded to improve performance over 60fps. GS has reignited our passion for 3D/4D scanning. Thank you to the team at Inria More content to come. #guassiansplatting #ir #aeonx #4d #motionscanning #inria #ximea #idatronic #spatial #youth83

Infinite-Realities

48,330 görüntüleme • 3 yıl önce

From 2D video to a very rough 4D Gaussian Splatting POC on the Apple Vision Pro 👀 Took Big Buck Bunny frames, turned them into Gaussian splats with Sharp (Apple), compressed them with PlayCanvas splat transform to SOG, and played all frames via WebXR + spark Still rough and not fully optimized, but wow, dynamic GS in AVP feels wild 🤯 Second part of the video is on my laptop #threejs
0:29

From 2D video to a very rough 4D Gaussian Splatting POC on the Apple Vision Pro 👀 Took Big Buck Bunny frames, turned them into Gaussian splats with Sharp (Apple), compressed them with PlayCanvas splat transform to SOG, and played all frames via WebXR + spark Still rough and not fully optimized, but wow, dynamic GS in AVP feels wild 🤯 Second part of the video is on my laptop #threejs

Stijn Spanhove

20,258 görüntüleme • 8 ay önce

Motion capture is about to get easy. This is captured with a single iPhone using Move One, our new mobile app that uses AI to extract motion data from video. No suits or markers required. We are in invite only beta right now, waitlist link in the comments📱
0:24

Motion capture is about to get easy. This is captured with a single iPhone using Move One, our new mobile app that uses AI to extract motion data from video. No suits or markers required. We are in invite only beta right now, waitlist link in the comments📱

Move AI

200,998 görüntüleme • 3 yıl önce

SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes abs: paper page: Existing methods for the 4D reconstruction of general, non-rigidly deforming objects focus on novel-view synthesis and neglect correspondences. However, time consistency enables advanced downstream tasks like 3D editing, motion analysis, or virtual-asset creation. We propose SceNeRFlow to reconstruct a general, non-rigid scene in a time-consistent manner. Our dynamic-NeRF method takes multi-view RGB videos and background images from static cameras with known camera parameters as input. It then reconstructs the deformations of an estimated canonical model of the geometry and appearance in an online fashion. Since this canonical model is time-invariant, we obtain correspondences even for long-term, long-range motions. We employ neural scene representations to parametrize the components of our method. Like prior dynamic-NeRF methods, we use a backwards deformation model. We find non-trivial adaptations of this model necessary to handle larger motions: We decompose the deformations into a strongly regularized coarse component and a weakly regularized fine component, where the coarse component also extends the deformation field into the space surrounding the object, which enables tracking over time. We show experimentally that, unlike prior work that only handles small motion, our method enables the reconstruction of studio-scale motions.
0:26

SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes abs: paper page: Existing methods for the 4D reconstruction of general, non-rigidly deforming objects focus on novel-view synthesis and neglect correspondences. However, time consistency enables advanced downstream tasks like 3D editing, motion analysis, or virtual-asset creation. We propose SceNeRFlow to reconstruct a general, non-rigid scene in a time-consistent manner. Our dynamic-NeRF method takes multi-view RGB videos and background images from static cameras with known camera parameters as input. It then reconstructs the deformations of an estimated canonical model of the geometry and appearance in an online fashion. Since this canonical model is time-invariant, we obtain correspondences even for long-term, long-range motions. We employ neural scene representations to parametrize the components of our method. Like prior dynamic-NeRF methods, we use a backwards deformation model. We find non-trivial adaptations of this model necessary to handle larger motions: We decompose the deformations into a strongly regularized coarse component and a weakly regularized fine component, where the coarse component also extends the deformation field into the space surrounding the object, which enables tracking over time. We show experimentally that, unlike prior work that only handles small motion, our method enables the reconstruction of studio-scale motions.

AK

76,380 görüntüleme • 3 yıl önce

We're proud to showcase some recent real-time work with 3D / 4D Gaussian splatting. In January of 2024 we acquired a full 3DGS commercial license with Inria to allow us to offer 4DGS services to clients around the world. We've spent the last 10 months developing our custom 4DGS software pipeline, utilizing our own Nvidia GPU cluster to batch process 100,000's of 4DGS frames rapidly, combined with our state-of-the-art custom built volumetric capture system(s). We can capture anything from complex character interactions for visual effects shots, or dynamic fast fighting scenes, sports, acting, even capturing and storing memories of family and friends for the future! The possibilities are endless. Our focus is always on high-quality and high-fidelity. We don't cut corners and we're continually striving to learn and improve. The below video showcases some 6-dof spatial captures we took of Henry Pearce's family. These scenes are played back in real-time @ 30fps inside a 3rd party engine running in a 120fps virtual environment. This is a culmination of years of hard work and dedication and we are keen to commercialize this technology with you. Please do reach out if you're interested in our capture systems or our 3D/4DGS processing services. HQ YouTube Link: HQ Vimeo Link: #spatialcapture #vr #virtualmemories #inria #3dgs #4dgs #aeonx #ir #ximea Music "Vision" by megiddo music Special thanks to Aras Pranckevičius 🇺🇦🇱🇹
2:20

We're proud to showcase some recent real-time work with 3D / 4D Gaussian splatting. In January of 2024 we acquired a full 3DGS commercial license with Inria to allow us to offer 4DGS services to clients around the world. We've spent the last 10 months developing our custom 4DGS software pipeline, utilizing our own Nvidia GPU cluster to batch process 100,000's of 4DGS frames rapidly, combined with our state-of-the-art custom built volumetric capture system(s). We can capture anything from complex character interactions for visual effects shots, or dynamic fast fighting scenes, sports, acting, even capturing and storing memories of family and friends for the future! The possibilities are endless. Our focus is always on high-quality and high-fidelity. We don't cut corners and we're continually striving to learn and improve. The below video showcases some 6-dof spatial captures we took of Henry Pearce's family. These scenes are played back in real-time @ 30fps inside a 3rd party engine running in a 120fps virtual environment. This is a culmination of years of hard work and dedication and we are keen to commercialize this technology with you. Please do reach out if you're interested in our capture systems or our 3D/4DGS processing services. HQ YouTube Link: HQ Vimeo Link: #spatialcapture #vr #virtualmemories #inria #3dgs #4dgs #aeonx #ir #ximea Music "Vision" by megiddo music Special thanks to Aras Pranckevičius 🇺🇦🇱🇹

Infinite-Realities

204,036 görüntüleme • 2 yıl önce

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:
0:50

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

Sherwin Bahmani

66,865 görüntüleme • 11 ay önce

The greatest Gaussian Splatting VR demo ever? Captured live at NVIDIA GTC — Mariya Sha experiencing 4D Gaussian Splatting in VR for the very first time. You can tell just from her reaction that this was something else! The scene was optimized using QUEEN, NVIDIA AI Developer's advanced research project for fast radiance field rendering. It’s an incredible look at what happens when high fidelity static and dynamic gaussian splats meets VR. Watch the full video on YouTube: Robert Scoble Did you get to do this one? #VR #3D #VFX
0:47

The greatest Gaussian Splatting VR demo ever? Captured live at NVIDIA GTC — Mariya Sha experiencing 4D Gaussian Splatting in VR for the very first time. You can tell just from her reaction that this was something else! The scene was optimized using QUEEN, NVIDIA AI Developer's advanced research project for fast radiance field rendering. It’s an incredible look at what happens when high fidelity static and dynamic gaussian splats meets VR. Watch the full video on YouTube: Robert Scoble Did you get to do this one? #VR #3D #VFX

Jonathan Stephens

28,114 görüntüleme • 1 yıl önce

GOOGLE DEEPMIND JUST TURNED A RANDOM OFFICE WALKTHROUGH INTO A SEARCHABLE 4D WORLD YOU CAN QUERY IN PLAIN ENGLISH. You feed it raw video. Nothing tagged, nothing labeled, no manual annotation. GenCeption tears that footage apart into four layers at once: a depth map, surface normals, a camera raymap, and segmentation masks. Then it rebuilds the whole space as a 4D scene you can actually talk to. The demo is stupid simple and that's the point. Type "I want to find a comfortable place to sit" and the sofa lights up. "I need a whiteboard." Whiteboard, found. "Is there a monitor I can use?" Highlighted. "Where's my colleague?" It grounds the person. No bounding boxes drawn by hand. No labels typed in advance. The model figured out what a sofa IS and where it sits in 3D space, from pixels alone. Every video you've ever recorded is now a database you can search with a sentence. Robots, AR, security, indexing. Same primitive. The camera roll just became queryable.
0:22

GOOGLE DEEPMIND JUST TURNED A RANDOM OFFICE WALKTHROUGH INTO A SEARCHABLE 4D WORLD YOU CAN QUERY IN PLAIN ENGLISH. You feed it raw video. Nothing tagged, nothing labeled, no manual annotation. GenCeption tears that footage apart into four layers at once: a depth map, surface normals, a camera raymap, and segmentation masks. Then it rebuilds the whole space as a 4D scene you can actually talk to. The demo is stupid simple and that's the point. Type "I want to find a comfortable place to sit" and the sofa lights up. "I need a whiteboard." Whiteboard, found. "Is there a monitor I can use?" Highlighted. "Where's my colleague?" It grounds the person. No bounding boxes drawn by hand. No labels typed in advance. The model figured out what a sofa IS and where it sits in 3D space, from pixels alone. Every video you've ever recorded is now a database you can search with a sentence. Robots, AR, security, indexing. Same primitive. The camera roll just became queryable.

Marre

23,671 görüntüleme • 2 ay önce