正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Want to obtain time-consistent dynamic mesh from monocular videos? Introducing: Dynamic Gaussians Mesh: Consistent Mesh Reconstruction from Monocular Videos We reconstruct meshes with flexible topology change and build the corresp. across meshes. 🧵(1/n)

Isabella Liu

1,101 subscribers

71,819 次观看 • 2 年前 •via X (Twitter)

科学技术教育

Anya Rossi• Live Now

Private livecam show

15 条评论

Isabella Liu 的头像

Isabella Liu2 年前

We also apply our method on real-world captures. Our method is able to produce satisfying meshes given dynamic monocular input videos. 🧵(2/n)

Isabella Liu 的头像

Isabella Liu2 年前

With mesh-to-mesh correspondence, we support downstream applications such as texture editing on dynamic objects. 🧵(3/n)

Isabella Liu 的头像

Isabella Liu2 年前

In our method, we deform canonical 3D Gaussian points into different times and use the differentiable poisson solver and marching cubes to turn the oriented point cloud into the mesh. Because of this, we can reconstruct the mesh with flexible topology change. 🧵(4/n)

Isabella Liu 的头像

Isabella Liu2 年前

To acquire the mesh-to-mesh corresp., we decompose it into two separate corresps. that are easier to acquire: 1. the mesh-to-point corresp. achieved by Gaussian-Mesh Anchoring and 2. the points-to-canonical-point corresp. maintained by the Cycle-Consistent Deformation. 🧵(5/n)

Isabella Liu 的头像

Isabella Liu2 年前

ArXiv: Project page: GitHub: 🧵(6/n)

Hongwei Yi 的头像

Hongwei Yi2 年前

Nice results, I am wondering the training time, how long does it take for training for like 10 seconds video with 20 fps?

Isabella Liu 的头像

Isabella Liu2 年前

Hi Hongwei, thanks for asking! Currently it takes ~90min to train 200 frames

Blurry 的头像

Blurry2 年前

It looks awsome...great work!

You Jiacheng 的头像

You Jiacheng2 年前

How to compute the cycle consistency loss? G and G' don't have 1-1 correspondence (because of density control, avg&create), how to define F_f(G, t) = F_d(G', t) ? Also F should take (x, t) or their positional encoding as input, what does F(G, t) mean?

Isabella Liu 的头像

Isabella Liu2 年前

Hi Youjia, thanks for your question! F(G, t) means the same thing as F(x, t). G’ should be the form of G under the deformed space, so they have 1-1 corresp.

You Jiacheng 的头像

You Jiacheng2 年前

Oh, there are two G' in the paper. The first one is the G under the deformed space, the second one is the G under the deformed space and anchored by averaging properties & creating new Gaussian at face centroid.

Rui Chen 的头像

Rui Chen2 年前

Great work! Very high quality!

You Jiacheng 的头像

You Jiacheng2 年前

Awesome!

Dillon Claassens 的头像

Dillon Claassens1 年前

Did you work for crucible labs?

Leo@Yuhao 的头像

Leo@Yuhao2 年前

awesome work. Will you plan to release the raw iphone captured data and the script for generating the data? I tested the referred code but failed

相关视频

3D Gaussian Splats are cool, but they're static (Part 25). DG-Mesh can reconstruct high-quality and time-consistent 3D meshes from a single video and track mesh vertices over time, which enables texture editing on dynamic objects.

3D Gaussian Splats are cool, but they're static (Part 25). DG-Mesh can reconstruct high-quality and time-consistent 3D meshes from a single video and track mesh vertices over time, which enables texture editing on dynamic objects.

Dreaming Tulpa 🥓👑

79,033 次观看 • 2 年前

"DVD: Dynamic Video Depth" TL;DR: Recovers temporally consistent depth from monocular videos using diffusion priors + geometric constraints, handling dynamic scenes and motion robustly.

"DVD: Dynamic Video Depth" TL;DR: Recovers temporally consistent depth from monocular videos using diffusion priors + geometric constraints, handling dynamic scenes and motion robustly.

Alexandre Morgand

11,823 次观看 • 8 个月前

"Fast 4D Mesh Generation by Spatio-Temporal Attention Chains" TL;DR: training-free spatio-temporal attention chains generate topology-consistent 4D meshes from video 13× faster while improving temporal correspondence quality

"Fast 4D Mesh Generation by Spatio-Temporal Attention Chains" TL;DR: training-free spatio-temporal attention chains generate topology-consistent 4D meshes from video 13× faster while improving temporal correspondence quality

Alexandre Morgand

13,807 次观看 • 5 个月前

Introducing MegaSaM! 🎥 Accurate, fast, & robust structure + camera estimation from casual monocular videos of dynamic scenes! MegaSaM outputs camera parameters and consistent video depth, scaling to long videos with unconstrained camera paths and complex scene dynamics!

Introducing MegaSaM! 🎥 Accurate, fast, & robust structure + camera estimation from casual monocular videos of dynamic scenes! MegaSaM outputs camera parameters and consistent video depth, scaling to long videos with unconstrained camera paths and complex scene dynamics!

Zhengqi Li

57,067 次观看 • 1 年前

Tripo P1.0 is live. Introducing Smart Mesh in Tripo Studio — generate structured 3D meshes in ~2 seconds. ⚡ ~2s generation 🧠 Smart topology 🛠 Production-ready assets 🌍 Built for game pipelines, real-time rendering, web 3D tools, and scalable content production From prompt to usable mesh — almost instantly. Try it: #3DAI #3DCG #3dart #Tripo

Tripo P1.0 is live. Introducing Smart Mesh in Tripo Studio — generate structured 3D meshes in ~2 seconds. ⚡ ~2s generation 🧠 Smart topology 🛠 Production-ready assets 🌍 Built for game pipelines, real-time rendering, web 3D tools, and scalable content production From prompt to usable mesh — almost instantly. Try it: #3DAI #3DCG #3dart #Tripo

Tripo

45,131 次观看 • 8 个月前

(1/2) Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generation #ECCV2024! We show a theoretical derivation to create radiance fields directly from meshes. Thus, we can obtain GT training data for generative NeRF methods.

(1/2) Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generation #ECCV2024! We show a theoretical derivation to create radiance fields directly from meshes. Thus, we can obtain GT training data for generative NeRF methods.

Matthias Niessner

16,975 次观看 • 2 年前

We are thrilled to release the next leap in art-grade 3D generative models. Our single-click model pipeline gives unprecedented mesh outputs, with mesh parts-based topology. It is available now for all Cube tiers to start for free. ☑️ Our multi-stage hierarchical AI models produce a fully assembled 3D mesh with adaptive poly-counts, providing the clean, separated topology you need. ✅ A parts-based approach enables high-resolution meshes. 🌐 Quad and Triangle mesh support. Access now:

We are thrilled to release the next leap in art-grade 3D generative models. Our single-click model pipeline gives unprecedented mesh outputs, with mesh parts-based topology. It is available now for all Cube tiers to start for free. ☑️ Our multi-stage hierarchical AI models produce a fully assembled 3D mesh with adaptive poly-counts, providing the clean, separated topology you need. ✅ A parts-based approach enables high-resolution meshes. 🌐 Quad and Triangle mesh support. Access now:

Common Sense Machines

420,512 次观看 • 1 年前

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.

MrNeRF

34,782 次观看 • 1 年前

I'm excited to share our new work Align3R that estimates camera poses and consistent depth maps from a monocular video of a dynamic scene. Project page: Code: Paper:

I'm excited to share our new work Align3R that estimates camera poses and consistent depth maps from a monocular video of a dynamic scene. Project page: Code: Paper:

Yuan Liu

56,547 次观看 • 1 年前

"AgentSTAR: Agentic Shape Tracking and Reconstruction from Monocular Videos" VLM agent+optimization to reconstruct an object geometry+kinematic structure from RGB video. Pose+articulation through time including occlusions, transparent objects. No dense pixel correspondences.

"AgentSTAR: Agentic Shape Tracking and Reconstruction from Monocular Videos" VLM agent+optimization to reconstruct an object geometry+kinematic structure from RGB video. Pose+articulation through time including occlusions, transparent objects. No dense pixel correspondences.

Alexandre Morgand

25,213 次观看 • 1 个月前

MeshLRM Large Reconstruction Model for High-Quality Mesh We propose MeshLRM, a novel LRM-based approach that can reconstruct a high-quality mesh from merely four input images in less than one second. Different from previous large reconstruction models (LRMs) that focus on

MeshLRM Large Reconstruction Model for High-Quality Mesh We propose MeshLRM, a novel LRM-based approach that can reconstruct a high-quality mesh from merely four input images in less than one second. Different from previous large reconstruction models (LRMs) that focus on

AK

69,193 次观看 • 2 年前

"Bro trust me, AI-generated meshes will never be production-ready" The AI-generate mesh:

"Bro trust me, AI-generated meshes will never be production-ready" The AI-generate mesh:

Emm | scenario.com

942,731 次观看 • 1 年前

I prompted Omma to build a #threejs app with real dynamic lighting on a 3D mesh - generated from a single image, using Depth Anything v2 + Transformers from Hugging Face ⚡ Depth estimation in the browser 🫧 3D mesh from the depth map 💡 Dynamic lighting reacting to the geometry No manual code. Pure AI prompting. The future of 3D on the web feels wide open.

I prompted Omma to build a #threejs app with real dynamic lighting on a 3D mesh - generated from a single image, using Depth Anything v2 + Transformers from Hugging Face ⚡ Depth estimation in the browser 🫧 3D mesh from the depth map 💡 Dynamic lighting reacting to the geometry No manual code. Pure AI prompting. The future of 3D on the web feels wide open.

Joseph Azar

14,041 次观看 • 6 个月前

Tip for deforming chest mesh: If you build the mesh topology to follow the lines of the chest and bra textures, it deforms really well! #Vtuber #Live2D

Tip for deforming chest mesh: If you build the mesh topology to follow the lines of the chest and bra textures, it deforms really well! #Vtuber #Live2D

Guilin (Live2D Rigger)

32,617 次观看 • 4 个月前

📢TriFlow: Generating Artist-Like 3D Mesh Topology via Nearest-Vertex Vector Fields (ECCV'26)📢 Compact 3D meshes with clean, artist-like triangle topology - structured connectivity you'd expect a human modeler to make🙂 🌐 ▶️

📢TriFlow: Generating Artist-Like 3D Mesh Topology via Nearest-Vertex Vector Fields (ECCV'26)📢 Compact 3D meshes with clean, artist-like triangle topology - structured connectivity you'd expect a human modeler to make🙂 🌐 ▶️

Matthias Niessner

116,262 次观看 • 4 个月前

Flexible Isosurface Extraction for Gradient-Based Mesh Optimization paper page: This work considers gradient-based mesh optimization, where we iteratively optimize for a 3D surface mesh by representing it as the isosurface of a scalar field, an increasingly common paradigm in applications including photogrammetry, generative modeling, and inverse physics. Existing implementations adapt classic isosurface extraction algorithms like Marching Cubes or Dual Contouring; these techniques were designed to extract meshes from fixed, known fields, and in the optimization setting they lack the degrees of freedom to represent high-quality feature-preserving meshes, or suffer from numerical instabilities. We introduce FlexiCubes, an isosurface representation specifically designed for optimizing an unknown mesh with respect to geometric, visual, or even physical objectives. Our main insight is to introduce additional carefully-chosen parameters into the representation, which allow local flexible adjustments to the extracted mesh geometry and connectivity. These parameters are updated along with the underlying scalar field via automatic differentiation when optimizing for a downstream task. We base our extraction scheme on Dual Marching Cubes for improved topological properties, and present extensions to optionally generate tetrahedral and hierarchically-adaptive meshes. Extensive experiments validate FlexiCubes on both synthetic benchmarks and real-world applications, showing that it offers significant improvements in mesh quality and geometric fidelity.

Flexible Isosurface Extraction for Gradient-Based Mesh Optimization paper page: This work considers gradient-based mesh optimization, where we iteratively optimize for a 3D surface mesh by representing it as the isosurface of a scalar field, an increasingly common paradigm in applications including photogrammetry, generative modeling, and inverse physics. Existing implementations adapt classic isosurface extraction algorithms like Marching Cubes or Dual Contouring; these techniques were designed to extract meshes from fixed, known fields, and in the optimization setting they lack the degrees of freedom to represent high-quality feature-preserving meshes, or suffer from numerical instabilities. We introduce FlexiCubes, an isosurface representation specifically designed for optimizing an unknown mesh with respect to geometric, visual, or even physical objectives. Our main insight is to introduce additional carefully-chosen parameters into the representation, which allow local flexible adjustments to the extracted mesh geometry and connectivity. These parameters are updated along with the underlying scalar field via automatic differentiation when optimizing for a downstream task. We base our extraction scheme on Dual Marching Cubes for improved topological properties, and present extensions to optionally generate tetrahedral and hierarchically-adaptive meshes. Extensive experiments validate FlexiCubes on both synthetic benchmarks and real-world applications, showing that it offers significant improvements in mesh quality and geometric fidelity.

AK

38,455 次观看 • 3 年前

DeepMesh is out on Hugging Face Auto-Regressive Artist-mesh Creation with Reinforcement Learning Conditioned on point clouds and images, DeepMesh generates meshes with intricate details and precise topology, outperforming state-of-the-art methods in both precision and quality.

DeepMesh is out on Hugging Face Auto-Regressive Artist-mesh Creation with Reinforcement Learning Conditioned on point clouds and images, DeepMesh generates meshes with intricate details and precise topology, outperforming state-of-the-art methods in both precision and quality.

AK

109,272 次观看 • 1 年前

"MeshFlow: Efficient Artistic Mesh Generation via MeshVAE and Flow-based Diffusion Transformer" TL;DR: learns a continuous mesh latent space and generates vertices and connectivity in parallel with flow matching, producing quality 3D meshes up to 18× faster than autoregressive.

"MeshFlow: Efficient Artistic Mesh Generation via MeshVAE and Flow-based Diffusion Transformer" TL;DR: learns a continuous mesh latent space and generates vertices and connectivity in parallel with flow matching, producing quality 3D meshes up to 18× faster than autoregressive.

Alexandre Morgand

43,375 次观看 • 5 个月前

📢MeshArt: Generating Articulated Meshes with Structure-guided Transformers Daoyi Gao generates articulated meshes with a hierarchical transformer, modeling articulation-aware structures that guide mesh synthesis. w/ Yawar Siddiqui Lei Li Project:

📢MeshArt: Generating Articulated Meshes with Structure-guided Transformers Daoyi Gao generates articulated meshes with a hierarchical transformer, modeling articulation-aware structures that guide mesh synthesis. w/ Yawar Siddiqui Lei Li Project:

Angela Dai

18,864 次观看 • 1 年前