正在加载视频...

视频加载失败

We’ve upgraded Stable Video Diffusion 4D to Stable Video 4D 2.0 (SV4D 2.0), improving the quality of 4D outputs generated from a single object-centric video. While 3D provides a static view of an object’s shape and size; 4D extends this by including time, showing how the object moves. This...

35,974 次观看 • 1 年前 •via X (Twitter)

10 条评论

Stability AI 的头像
Stability AI1 年前

Our analysis shows that SV4D 2.0 achieves state-of-the-art results in 4D generation, ranking first across all major benchmarks. (2/4)

Stability AI 的头像
Stability AI1 年前

This upgrade marks progress toward 4D asset generation for professional production workflows, from generating sprite sheets for in-game characters, to supporting assets for film and virtual worlds. Multi-view generation, however, remains complex due to the inherent ambiguity of visualizing 3D objects from unseen views. As a result, occasional artifacts may still appear with dynamic motion. We invite the community to explore SV4D 2.0 and contribute to its ongoing development. (3/4)

Stability AI 的头像
Stability AI1 年前

Weights, code, and paper 👇 @HuggingFace: @Github: @arXiv: (4/4)

zazoum 的头像
zazoum1 年前

Can the output be a 3D animated file in any case? Like having blend shapes embed?

Stability AI 的头像
Stability AI1 年前

The current output is a Dynamic NeRF representation, which doesn’t support explicit formats like FBX or blend shapes. However, one can use the multi-view videos as reference to optimize a different 4D representation.

Sam 的头像
Sam1 年前

@ai_for_success fyi

Mayank 的头像
Mayank1 年前

This looks great, testing it out to build 3D Games for or should I call it 4D Games?

Dhaval Makwana 的头像
Dhaval Makwana1 年前

This seems cool! Let’s connect over dm @StabilityAI

Ben Pielstick 的头像
Ben Pielstick1 年前

How about FBX output?

Stability AI 的头像
Stability AI1 年前

The current output is a Dynamic NeRF representation, which doesn’t support explicit formats like FBX or blend shapes. However, one can use the multi-view videos as reference to optimize a different 4D representation.

相关视频

[NeurIPS '24] DreamMesh4D: Video-to-4D Generation with Sparse-Controlled Gaussian-Mesh Hybrid Representation Abstract (excerpt) We introduce DreamMesh4D, a novel framework that combines mesh representation with sparse-controlled deformation technique to generate high-quality 4D object from a monocular video. To overcome the limitation of classical texture representation, we bind Gaussian splats to the surface of the triangular mesh for differentiable optimization of both the texture and mesh vertices. In particular, DreamMesh4D begins with a coarse mesh provided by a single image based 3D generation method. Sparse points are then uniformly sampled across the surface of the mesh, and are used to build a deformation graph to drive the motion of the 3D object for the sake of computational efficiency and providing additional constraint. For each step, transformations of sparse control points are predicted using a deformation network, and the mesh vertices as well as the bound surface Gaussians are deformed via a geometric skinning algorithm. The skinning algorithm is a hybrid approach combining LBS (linear blending skinning) and DQS (dual-quaternion skinning), mitigating drawbacks associated with both approaches. The static surface Gaussians and mesh vertices as well as the dynamic deformation network are learned via reference view photometric loss, score distillation loss as well as other regularization losses in a two-stage manner. Extensive experiments demonstrate that our method outperforms prior video-to-4D generation methods in terms of rendering quality and spatial-temporal consistency.

MrNeRF

12,323 次观看 • 1 年前