Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Want to obtain time-consistent dynamic mesh from monocular videos? Introducing: Dynamic Gaussians Mesh: Consistent Mesh Reconstruction from Monocular Videos We reconstruct meshes with flexible topology change and build the corresp. across meshes. 🧵(1/n)

Isabella Liu

1,078 subscribers

71,800 görüntüleme • 2 yıl önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

15 Yorum

Isabella Liu profil fotoğrafı

Isabella Liu2 yıl önce

We also apply our method on real-world captures. Our method is able to produce satisfying meshes given dynamic monocular input videos. 🧵(2/n)

Isabella Liu profil fotoğrafı

Isabella Liu2 yıl önce

With mesh-to-mesh correspondence, we support downstream applications such as texture editing on dynamic objects. 🧵(3/n)

Isabella Liu profil fotoğrafı

Isabella Liu2 yıl önce

In our method, we deform canonical 3D Gaussian points into different times and use the differentiable poisson solver and marching cubes to turn the oriented point cloud into the mesh. Because of this, we can reconstruct the mesh with flexible topology change. 🧵(4/n)

Isabella Liu profil fotoğrafı

Isabella Liu2 yıl önce

To acquire the mesh-to-mesh corresp., we decompose it into two separate corresps. that are easier to acquire: 1. the mesh-to-point corresp. achieved by Gaussian-Mesh Anchoring and 2. the points-to-canonical-point corresp. maintained by the Cycle-Consistent Deformation. 🧵(5/n)

Isabella Liu profil fotoğrafı

Isabella Liu2 yıl önce

ArXiv: Project page: GitHub: 🧵(6/n)

Hongwei Yi profil fotoğrafı

Hongwei Yi2 yıl önce

Nice results, I am wondering the training time, how long does it take for training for like 10 seconds video with 20 fps?

Isabella Liu profil fotoğrafı

Isabella Liu2 yıl önce

Hi Hongwei, thanks for asking! Currently it takes ~90min to train 200 frames

Blurry profil fotoğrafı

Blurry2 yıl önce

It looks awsome...great work!

You Jiacheng profil fotoğrafı

You Jiacheng2 yıl önce

How to compute the cycle consistency loss? G and G' don't have 1-1 correspondence (because of density control, avg&create), how to define F_f(G, t) = F_d(G', t) ? Also F should take (x, t) or their positional encoding as input, what does F(G, t) mean?

Isabella Liu profil fotoğrafı

Isabella Liu2 yıl önce

Hi Youjia, thanks for your question! F(G, t) means the same thing as F(x, t). G’ should be the form of G under the deformed space, so they have 1-1 corresp.

You Jiacheng profil fotoğrafı

You Jiacheng2 yıl önce

Oh, there are two G' in the paper. The first one is the G under the deformed space, the second one is the G under the deformed space and anchored by averaging properties & creating new Gaussian at face centroid.

Rui Chen profil fotoğrafı

Rui Chen2 yıl önce

Great work! Very high quality!

You Jiacheng profil fotoğrafı

You Jiacheng2 yıl önce

Awesome!

Dillon Claassens profil fotoğrafı

Dillon Claassens1 yıl önce

Did you work for crucible labs?

Leo@Yuhao profil fotoğrafı

Leo@Yuhao2 yıl önce

awesome work. Will you plan to release the raw iphone captured data and the script for generating the data? I tested the referred code but failed

Benzer Videolar

3D Gaussian Splats are cool, but they're static (Part 25). DG-Mesh can reconstruct high-quality and time-consistent 3D meshes from a single video and track mesh vertices over time, which enables texture editing on dynamic objects.

3D Gaussian Splats are cool, but they're static (Part 25). DG-Mesh can reconstruct high-quality and time-consistent 3D meshes from a single video and track mesh vertices over time, which enables texture editing on dynamic objects.

Dreaming Tulpa 🥓👑

79,033 görüntüleme • 2 yıl önce

"DVD: Dynamic Video Depth" TL;DR: Recovers temporally consistent depth from monocular videos using diffusion priors + geometric constraints, handling dynamic scenes and motion robustly.

"DVD: Dynamic Video Depth" TL;DR: Recovers temporally consistent depth from monocular videos using diffusion priors + geometric constraints, handling dynamic scenes and motion robustly.

Alexandre Morgand

11,823 görüntüleme • 7 ay önce

"Fast 4D Mesh Generation by Spatio-Temporal Attention Chains" TL;DR: training-free spatio-temporal attention chains generate topology-consistent 4D meshes from video 13× faster while improving temporal correspondence quality

"Fast 4D Mesh Generation by Spatio-Temporal Attention Chains" TL;DR: training-free spatio-temporal attention chains generate topology-consistent 4D meshes from video 13× faster while improving temporal correspondence quality

Alexandre Morgand

13,807 görüntüleme • 5 ay önce

Introducing MegaSaM! 🎥 Accurate, fast, & robust structure + camera estimation from casual monocular videos of dynamic scenes! MegaSaM outputs camera parameters and consistent video depth, scaling to long videos with unconstrained camera paths and complex scene dynamics!

Introducing MegaSaM! 🎥 Accurate, fast, & robust structure + camera estimation from casual monocular videos of dynamic scenes! MegaSaM outputs camera parameters and consistent video depth, scaling to long videos with unconstrained camera paths and complex scene dynamics!

Zhengqi Li

57,067 görüntüleme • 1 yıl önce

Tripo P1.0 is live. Introducing Smart Mesh in Tripo Studio — generate structured 3D meshes in ~2 seconds. ⚡ ~2s generation 🧠 Smart topology 🛠 Production-ready assets 🌍 Built for game pipelines, real-time rendering, web 3D tools, and scalable content production From prompt to usable mesh — almost instantly. Try it: #3DAI #3DCG #3dart #Tripo

Tripo P1.0 is live. Introducing Smart Mesh in Tripo Studio — generate structured 3D meshes in ~2 seconds. ⚡ ~2s generation 🧠 Smart topology 🛠 Production-ready assets 🌍 Built for game pipelines, real-time rendering, web 3D tools, and scalable content production From prompt to usable mesh — almost instantly. Try it: #3DAI #3DCG #3dart #Tripo

Tripo

45,088 görüntüleme • 7 ay önce

(1/2) Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generation #ECCV2024! We show a theoretical derivation to create radiance fields directly from meshes. Thus, we can obtain GT training data for generative NeRF methods.

(1/2) Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generation #ECCV2024! We show a theoretical derivation to create radiance fields directly from meshes. Thus, we can obtain GT training data for generative NeRF methods.

Matthias Niessner

16,835 görüntüleme • 2 yıl önce

We are thrilled to release the next leap in art-grade 3D generative models. Our single-click model pipeline gives unprecedented mesh outputs, with mesh parts-based topology. It is available now for all Cube tiers to start for free. ☑️ Our multi-stage hierarchical AI models produce a fully assembled 3D mesh with adaptive poly-counts, providing the clean, separated topology you need. ✅ A parts-based approach enables high-resolution meshes. 🌐 Quad and Triangle mesh support. Access now:

We are thrilled to release the next leap in art-grade 3D generative models. Our single-click model pipeline gives unprecedented mesh outputs, with mesh parts-based topology. It is available now for all Cube tiers to start for free. ☑️ Our multi-stage hierarchical AI models produce a fully assembled 3D mesh with adaptive poly-counts, providing the clean, separated topology you need. ✅ A parts-based approach enables high-resolution meshes. 🌐 Quad and Triangle mesh support. Access now:

Common Sense Machines

420,505 görüntüleme • 1 yıl önce

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.

MrNeRF

34,782 görüntüleme • 1 yıl önce

I'm excited to share our new work Align3R that estimates camera poses and consistent depth maps from a monocular video of a dynamic scene. Project page: Code: Paper:

I'm excited to share our new work Align3R that estimates camera poses and consistent depth maps from a monocular video of a dynamic scene. Project page: Code: Paper:

Yuan Liu

56,547 görüntüleme • 1 yıl önce

"AgentSTAR: Agentic Shape Tracking and Reconstruction from Monocular Videos" VLM agent+optimization to reconstruct an object geometry+kinematic structure from RGB video. Pose+articulation through time including occlusions, transparent objects. No dense pixel correspondences.

"AgentSTAR: Agentic Shape Tracking and Reconstruction from Monocular Videos" VLM agent+optimization to reconstruct an object geometry+kinematic structure from RGB video. Pose+articulation through time including occlusions, transparent objects. No dense pixel correspondences.

Alexandre Morgand

25,126 görüntüleme • 1 ay önce

MeshLRM Large Reconstruction Model for High-Quality Mesh We propose MeshLRM, a novel LRM-based approach that can reconstruct a high-quality mesh from merely four input images in less than one second. Different from previous large reconstruction models (LRMs) that focus on

MeshLRM Large Reconstruction Model for High-Quality Mesh We propose MeshLRM, a novel LRM-based approach that can reconstruct a high-quality mesh from merely four input images in less than one second. Different from previous large reconstruction models (LRMs) that focus on

AK

69,190 görüntüleme • 2 yıl önce

"Bro trust me, AI-generated meshes will never be production-ready" The AI-generate mesh:

"Bro trust me, AI-generated meshes will never be production-ready" The AI-generate mesh:

Emm | scenario.com

942,608 görüntüleme • 1 yıl önce

I prompted Omma to build a #threejs app with real dynamic lighting on a 3D mesh - generated from a single image, using Depth Anything v2 + Transformers from Hugging Face ⚡ Depth estimation in the browser 🫧 3D mesh from the depth map 💡 Dynamic lighting reacting to the geometry No manual code. Pure AI prompting. The future of 3D on the web feels wide open.

I prompted Omma to build a #threejs app with real dynamic lighting on a 3D mesh - generated from a single image, using Depth Anything v2 + Transformers from Hugging Face ⚡ Depth estimation in the browser 🫧 3D mesh from the depth map 💡 Dynamic lighting reacting to the geometry No manual code. Pure AI prompting. The future of 3D on the web feels wide open.

Joseph Azar

14,041 görüntüleme • 6 ay önce

Tip for deforming chest mesh: If you build the mesh topology to follow the lines of the chest and bra textures, it deforms really well! #Vtuber #Live2D

Tip for deforming chest mesh: If you build the mesh topology to follow the lines of the chest and bra textures, it deforms really well! #Vtuber #Live2D

Guilin (Live2D Rigger)

31,914 görüntüleme • 4 ay önce

📢TriFlow: Generating Artist-Like 3D Mesh Topology via Nearest-Vertex Vector Fields (ECCV'26)📢 Compact 3D meshes with clean, artist-like triangle topology - structured connectivity you'd expect a human modeler to make🙂 🌐 ▶️

📢TriFlow: Generating Artist-Like 3D Mesh Topology via Nearest-Vertex Vector Fields (ECCV'26)📢 Compact 3D meshes with clean, artist-like triangle topology - structured connectivity you'd expect a human modeler to make🙂 🌐 ▶️

Matthias Niessner

116,134 görüntüleme • 3 ay önce

Flexible Isosurface Extraction for Gradient-Based Mesh Optimization paper page: This work considers gradient-based mesh optimization, where we iteratively optimize for a 3D surface mesh by representing it as the isosurface of a scalar field, an increasingly common paradigm in applications including photogrammetry, generative modeling, and inverse physics. Existing implementations adapt classic isosurface extraction algorithms like Marching Cubes or Dual Contouring; these techniques were designed to extract meshes from fixed, known fields, and in the optimization setting they lack the degrees of freedom to represent high-quality feature-preserving meshes, or suffer from numerical instabilities. We introduce FlexiCubes, an isosurface representation specifically designed for optimizing an unknown mesh with respect to geometric, visual, or even physical objectives. Our main insight is to introduce additional carefully-chosen parameters into the representation, which allow local flexible adjustments to the extracted mesh geometry and connectivity. These parameters are updated along with the underlying scalar field via automatic differentiation when optimizing for a downstream task. We base our extraction scheme on Dual Marching Cubes for improved topological properties, and present extensions to optionally generate tetrahedral and hierarchically-adaptive meshes. Extensive experiments validate FlexiCubes on both synthetic benchmarks and real-world applications, showing that it offers significant improvements in mesh quality and geometric fidelity.

Flexible Isosurface Extraction for Gradient-Based Mesh Optimization paper page: This work considers gradient-based mesh optimization, where we iteratively optimize for a 3D surface mesh by representing it as the isosurface of a scalar field, an increasingly common paradigm in applications including photogrammetry, generative modeling, and inverse physics. Existing implementations adapt classic isosurface extraction algorithms like Marching Cubes or Dual Contouring; these techniques were designed to extract meshes from fixed, known fields, and in the optimization setting they lack the degrees of freedom to represent high-quality feature-preserving meshes, or suffer from numerical instabilities. We introduce FlexiCubes, an isosurface representation specifically designed for optimizing an unknown mesh with respect to geometric, visual, or even physical objectives. Our main insight is to introduce additional carefully-chosen parameters into the representation, which allow local flexible adjustments to the extracted mesh geometry and connectivity. These parameters are updated along with the underlying scalar field via automatic differentiation when optimizing for a downstream task. We base our extraction scheme on Dual Marching Cubes for improved topological properties, and present extensions to optionally generate tetrahedral and hierarchically-adaptive meshes. Extensive experiments validate FlexiCubes on both synthetic benchmarks and real-world applications, showing that it offers significant improvements in mesh quality and geometric fidelity.

AK

38,455 görüntüleme • 3 yıl önce

DeepMesh is out on Hugging Face Auto-Regressive Artist-mesh Creation with Reinforcement Learning Conditioned on point clouds and images, DeepMesh generates meshes with intricate details and precise topology, outperforming state-of-the-art methods in both precision and quality.

DeepMesh is out on Hugging Face Auto-Regressive Artist-mesh Creation with Reinforcement Learning Conditioned on point clouds and images, DeepMesh generates meshes with intricate details and precise topology, outperforming state-of-the-art methods in both precision and quality.

AK

109,272 görüntüleme • 1 yıl önce

"MeshFlow: Efficient Artistic Mesh Generation via MeshVAE and Flow-based Diffusion Transformer" TL;DR: learns a continuous mesh latent space and generates vertices and connectivity in parallel with flow matching, producing quality 3D meshes up to 18× faster than autoregressive.

"MeshFlow: Efficient Artistic Mesh Generation via MeshVAE and Flow-based Diffusion Transformer" TL;DR: learns a continuous mesh latent space and generates vertices and connectivity in parallel with flow matching, producing quality 3D meshes up to 18× faster than autoregressive.

Alexandre Morgand

43,375 görüntüleme • 4 ay önce