Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

We’ve upgraded Stable Video Diffusion 4D to Stable Video 4D 2.0 (SV4D 2.0), improving the quality of 4D outputs generated from a single object-centric video. While 3D provides a static view of an object’s shape and size; 4D extends this by including time, showing how the object moves. This... show more

Stability AI

256,462 subscribers

35,974 Aufrufe • vor 1 Jahr •via X (Twitter)

Kunst Wissenschaft & Technologie Bildung

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Stability AI

Stability AIvor 1 Jahr

Our analysis shows that SV4D 2.0 achieves state-of-the-art results in 4D generation, ranking first across all major benchmarks. (2/4)

Profilbild von Stability AI

Stability AIvor 1 Jahr

This upgrade marks progress toward 4D asset generation for professional production workflows, from generating sprite sheets for in-game characters, to supporting assets for film and virtual worlds. Multi-view generation, however, remains complex due to the inherent ambiguity of visualizing 3D objects from unseen views. As a result, occasional artifacts may still appear with dynamic motion. We invite the community to explore SV4D 2.0 and contribute to its ongoing development. (3/4)

Profilbild von Stability AI

Stability AIvor 1 Jahr

Weights, code, and paper 👇 @HuggingFace: @Github: @arXiv: (4/4)

Profilbild von zazoum

zazoumvor 1 Jahr

Can the output be a 3D animated file in any case? Like having blend shapes embed?

Profilbild von Stability AI

Stability AIvor 1 Jahr

The current output is a Dynamic NeRF representation, which doesn’t support explicit formats like FBX or blend shapes. However, one can use the multi-view videos as reference to optimize a different 4D representation.

Profilbild von Sam

Samvor 1 Jahr

@ai_for_success fyi

Profilbild von Mayank

Mayankvor 1 Jahr

This looks great, testing it out to build 3D Games for or should I call it 4D Games?

Profilbild von Dhaval Makwana

Dhaval Makwanavor 1 Jahr

This seems cool! Let’s connect over dm @StabilityAI

Profilbild von Ben Pielstick

Ben Pielstickvor 1 Jahr

How about FBX output?

Profilbild von Stability AI

Stability AIvor 1 Jahr

The current output is a Dynamic NeRF representation, which doesn’t support explicit formats like FBX or blend shapes. However, one can use the multi-view videos as reference to optimize a different 4D representation.

Ähnliche Videos

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:

Chuang Gan

43,265 Aufrufe • vor 1 Jahr

InsertAnywhere Bridging 4D Scene Geometry and Diffusion Models for Realistic Video Object Insertion

InsertAnywhere Bridging 4D Scene Geometry and Diffusion Models for Realistic Video Object Insertion

AK

26,123 Aufrufe • vor 6 Monaten

Google presents CAT4D Create Anything in 4D with Multi-View Video Diffusion Models

Google presents CAT4D Create Anything in 4D with Multi-View Video Diffusion Models

AK

61,949 Aufrufe • vor 1 Jahr

4D LangSplat: 4D Language Gaussian Splatting via Multimodal Large Language Models Contributions: • We introduce 4D LangSplat for open-vocabulary 4D spatial-temporal queries. To the best of our knowledge, we are the first to construct 4D language fields with object textual captions generated by MLLMs. • To model smooth transitions across states for objects in 4D scenes, we propose a status deformable network to capture continuous temporal changes. • Experiential results show that our method attains state-of-the-art performance for both time-agnostic and time-sensitive open-vocabulary queries.

4D LangSplat: 4D Language Gaussian Splatting via Multimodal Large Language Models Contributions: • We introduce 4D LangSplat for open-vocabulary 4D spatial-temporal queries. To the best of our knowledge, we are the first to construct 4D language fields with object textual captions generated by MLLMs. • To model smooth transitions across states for objects in 4D scenes, we propose a status deformable network to capture continuous temporal changes. • Experiential results show that our method attains state-of-the-art performance for both time-agnostic and time-sensitive open-vocabulary queries.

MrNeRF

10,953 Aufrufe • vor 1 Jahr

Can we scale 4D pretraining to learn general space-time representations that reconstruct an object from a few views at any time to any view at any other time? Introducing 4D-LRM: a Large Space-Time Reconstruction Model that ... 🔹 Predicts 4D Gaussian primitives directly from multi-view tokens (no motion vectors, no HexPlane); 🔹 Uses a clean, minimal Transformer backbone; 🔹 Generalizes with fast, high-quality feedforward rendering at any view and infinite frame rate. Check out more interactive demos and scaling behaviors on our homepage/paper. 👉Website: 👉Paper:

Can we scale 4D pretraining to learn general space-time representations that reconstruct an object from a few views at any time to any view at any other time? Introducing 4D-LRM: a Large Space-Time Reconstruction Model that ... 🔹 Predicts 4D Gaussian primitives directly from multi-view tokens (no motion vectors, no HexPlane); 🔹 Uses a clean, minimal Transformer backbone; 🔹 Generalizes with fast, high-quality feedforward rendering at any view and infinite frame rate. Check out more interactive demos and scaling behaviors on our homepage/paper. 👉Website: 👉Paper:

Martin Ziqiao Ma

21,811 Aufrufe • vor 1 Jahr

Excited to share HOI-PAGE, to appear at #ICML2026! 🚀 Lei Li generates 4D human-object interactions zero-shot from text A part-affordance graph grounds interactions via LLM+video priors, enabling complex multi-person, multi-object interactions 👉

Excited to share HOI-PAGE, to appear at #ICML2026! 🚀 Lei Li generates 4D human-object interactions zero-shot from text A part-affordance graph grounds interactions via LLM+video priors, enabling complex multi-person, multi-object interactions 👉

Angela Dai

11,921 Aufrufe • vor 2 Monaten

Diffuman4D 4D Consistent Human View Synthesis from Sparse-View Videos with Spatio-Temporal Diffusion Models

Diffuman4D 4D Consistent Human View Synthesis from Sparse-View Videos with Spatio-Temporal Diffusion Models

AK

12,518 Aufrufe • vor 1 Jahr

"Fast 4D Mesh Generation by Spatio-Temporal Attention Chains" TL;DR: training-free spatio-temporal attention chains generate topology-consistent 4D meshes from video 13× faster while improving temporal correspondence quality

"Fast 4D Mesh Generation by Spatio-Temporal Attention Chains" TL;DR: training-free spatio-temporal attention chains generate topology-consistent 4D meshes from video 13× faster while improving temporal correspondence quality

Alexandre Morgand

13,643 Aufrufe • vor 1 Monat

🔥Unbounded 4D City Generation🔥 #CityDreamer4D is a generative model of unbounded 4D cities that decouples static and dynamic scenes with realism and controllability - Project: - Code: - CityTopia Dataset:

🔥Unbounded 4D City Generation🔥 #CityDreamer4D is a generative model of unbounded 4D cities that decouples static and dynamic scenes with realism and controllability - Project: - Code: - CityTopia Dataset:

Ziwei Liu

24,326 Aufrufe • vor 1 Jahr

Synthesia research releases HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion a 4D dynamic neural scene representation that captures full-body appearance in motion from multi-view video input, and enables playback from novel, unseen viewpoints project page: paper page:

Synthesia research releases HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion a 4D dynamic neural scene representation that captures full-body appearance in motion from multi-view video input, and enables playback from novel, unseen viewpoints project page: paper page:

AK

160,702 Aufrufe • vor 3 Jahren

VerseCrafter Dynamic Realistic Video World Model with 4D Geometric Control

VerseCrafter Dynamic Realistic Video World Model with 4D Geometric Control

AK

26,856 Aufrufe • vor 6 Monaten

V3D Video Diffusion Models are Effective 3D Generators Automatic 3D generation has recently attracted widespread attention. Recent methods have greatly accelerated the generation speed, but usually produce less-detailed objects due to limited model capacity or 3D data. Motivated by recent advancements in video diffusion models, we introduce V3D, which leverages the world simulation capacity of pre-trained video diffusion models to facilitate 3D generation. To fully unleash the potential of video diffusion to perceive the 3D world, we further introduce geometrical consistency prior and extend the video diffusion model to a multi-view consistent 3D generator. Benefiting from this, the state-of-the-art video diffusion model could be fine-tuned to generate 360degree orbit frames surrounding an object given a single image. With our tailored reconstruction pipelines, we can generate high-quality meshes or 3D Gaussians within 3 minutes. Furthermore, our method can be extended to scene-level novel view synthesis, achieving precise control over the camera path with sparse input views. Extensive experiments demonstrate the superior performance of the proposed approach, especially in terms of generation quality and multi-view consistency

V3D Video Diffusion Models are Effective 3D Generators Automatic 3D generation has recently attracted widespread attention. Recent methods have greatly accelerated the generation speed, but usually produce less-detailed objects due to limited model capacity or 3D data. Motivated by recent advancements in video diffusion models, we introduce V3D, which leverages the world simulation capacity of pre-trained video diffusion models to facilitate 3D generation. To fully unleash the potential of video diffusion to perceive the 3D world, we further introduce geometrical consistency prior and extend the video diffusion model to a multi-view consistent 3D generator. Benefiting from this, the state-of-the-art video diffusion model could be fine-tuned to generate 360degree orbit frames surrounding an object given a single image. With our tailored reconstruction pipelines, we can generate high-quality meshes or 3D Gaussians within 3 minutes. Furthermore, our method can be extended to scene-level novel view synthesis, achieving precise control over the camera path with sparse input views. Extensive experiments demonstrate the superior performance of the proposed approach, especially in terms of generation quality and multi-view consistency

AK

31,997 Aufrufe • vor 2 Jahren

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.

MrNeRF

34,782 Aufrufe • vor 1 Jahr

Interactive 4D Time-Lapse for #VDC in #Construction / #AEC Gauzilla Pro allows for creating a 4D time-lapse from a series of videos. You can convert drone footage into an interactive historical record in 4D running 100% on the web. ✅ 4D time-lapse drastically reduces the need for physical site visits and also enhances communications among teams and stakeholders.

Interactive 4D Time-Lapse for #VDC in #Construction / #AEC Gauzilla Pro allows for creating a 4D time-lapse from a series of videos. You can convert drone footage into an interactive historical record in 4D running 100% on the web. ✅ 4D time-lapse drastically reduces the need for physical site visits and also enhances communications among teams and stakeholders.

Gauzilla Pro

14,432 Aufrufe • vor 23 Tagen

🤖 wake up, new 4d tool! Take any normal 2D video → instantly turn it into fully explorable 3D space. You can now orbit, tilt, and view the scene from angles that literally never existed in the original footage. This is 4D Gaussian Splatting (from 4dv) and it’s next-level.

🤖 wake up, new 4d tool! Take any normal 2D video → instantly turn it into fully explorable 3D space. You can now orbit, tilt, and view the scene from angles that literally never existed in the original footage. This is 4D Gaussian Splatting (from 4dv) and it’s next-level.

filipe

66,085 Aufrufe • vor 1 Monat

Check out this BIM 4D sim by Thiery Peleias, owner of Virtuart4d 🚧 Imported into UE5 for the animation aspect, the project model was created in Revit, the 4D planning animation in Synchro 4D Pro and the environment captured from Cesium with Google Maps Photorealistic 3D Tiles!

Check out this BIM 4D sim by Thiery Peleias, owner of Virtuart4d 🚧 Imported into UE5 for the animation aspect, the project model was created in Revit, the 4D planning animation in Synchro 4D Pro and the environment captured from Cesium with Google Maps Photorealistic 3D Tiles!

Unreal Engine

20,772 Aufrufe • vor 1 Jahr

V-DPM 4D Video Reconstruction with Dynamic Point Maps

V-DPM 4D Video Reconstruction with Dynamic Point Maps

AK

19,274 Aufrufe • vor 5 Monaten

[NeurIPS '24] DreamMesh4D: Video-to-4D Generation with Sparse-Controlled Gaussian-Mesh Hybrid Representation Abstract (excerpt) We introduce DreamMesh4D, a novel framework that combines mesh representation with sparse-controlled deformation technique to generate high-quality 4D object from a monocular video. To overcome the limitation of classical texture representation, we bind Gaussian splats to the surface of the triangular mesh for differentiable optimization of both the texture and mesh vertices. In particular, DreamMesh4D begins with a coarse mesh provided by a single image based 3D generation method. Sparse points are then uniformly sampled across the surface of the mesh, and are used to build a deformation graph to drive the motion of the 3D object for the sake of computational efficiency and providing additional constraint. For each step, transformations of sparse control points are predicted using a deformation network, and the mesh vertices as well as the bound surface Gaussians are deformed via a geometric skinning algorithm. The skinning algorithm is a hybrid approach combining LBS (linear blending skinning) and DQS (dual-quaternion skinning), mitigating drawbacks associated with both approaches. The static surface Gaussians and mesh vertices as well as the dynamic deformation network are learned via reference view photometric loss, score distillation loss as well as other regularization losses in a two-stage manner. Extensive experiments demonstrate that our method outperforms prior video-to-4D generation methods in terms of rendering quality and spatial-temporal consistency.

[NeurIPS '24] DreamMesh4D: Video-to-4D Generation with Sparse-Controlled Gaussian-Mesh Hybrid Representation Abstract (excerpt) We introduce DreamMesh4D, a novel framework that combines mesh representation with sparse-controlled deformation technique to generate high-quality 4D object from a monocular video. To overcome the limitation of classical texture representation, we bind Gaussian splats to the surface of the triangular mesh for differentiable optimization of both the texture and mesh vertices. In particular, DreamMesh4D begins with a coarse mesh provided by a single image based 3D generation method. Sparse points are then uniformly sampled across the surface of the mesh, and are used to build a deformation graph to drive the motion of the 3D object for the sake of computational efficiency and providing additional constraint. For each step, transformations of sparse control points are predicted using a deformation network, and the mesh vertices as well as the bound surface Gaussians are deformed via a geometric skinning algorithm. The skinning algorithm is a hybrid approach combining LBS (linear blending skinning) and DQS (dual-quaternion skinning), mitigating drawbacks associated with both approaches. The static surface Gaussians and mesh vertices as well as the dynamic deformation network are learned via reference view photometric loss, score distillation loss as well as other regularization losses in a two-stage manner. Extensive experiments demonstrate that our method outperforms prior video-to-4D generation methods in terms of rendering quality and spatial-temporal consistency.

MrNeRF

12,323 Aufrufe • vor 1 Jahr