Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Want to obtain time-consistent dynamic mesh from monocular videos? Introducing: Dynamic Gaussians Mesh: Consistent Mesh Reconstruction from Monocular Videos We reconstruct meshes with flexible topology change and build the corresp. across meshes. 🧵(1/n)

Isabella Liu

1,078 subscribers

71,800 Aufrufe • vor 2 Jahren •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

15 Kommentare

Profilbild von Isabella Liu

Isabella Liuvor 2 Jahren

We also apply our method on real-world captures. Our method is able to produce satisfying meshes given dynamic monocular input videos. 🧵(2/n)

Profilbild von Isabella Liu

Isabella Liuvor 2 Jahren

With mesh-to-mesh correspondence, we support downstream applications such as texture editing on dynamic objects. 🧵(3/n)

Profilbild von Isabella Liu

Isabella Liuvor 2 Jahren

In our method, we deform canonical 3D Gaussian points into different times and use the differentiable poisson solver and marching cubes to turn the oriented point cloud into the mesh. Because of this, we can reconstruct the mesh with flexible topology change. 🧵(4/n)

Profilbild von Isabella Liu

Isabella Liuvor 2 Jahren

To acquire the mesh-to-mesh corresp., we decompose it into two separate corresps. that are easier to acquire: 1. the mesh-to-point corresp. achieved by Gaussian-Mesh Anchoring and 2. the points-to-canonical-point corresp. maintained by the Cycle-Consistent Deformation. 🧵(5/n)

Profilbild von Isabella Liu

Isabella Liuvor 2 Jahren

ArXiv: Project page: GitHub: 🧵(6/n)

Profilbild von Hongwei Yi

Hongwei Yivor 2 Jahren

Nice results, I am wondering the training time, how long does it take for training for like 10 seconds video with 20 fps?

Profilbild von Isabella Liu

Isabella Liuvor 2 Jahren

Hi Hongwei, thanks for asking! Currently it takes ~90min to train 200 frames

Profilbild von Blurry

Blurryvor 2 Jahren

It looks awsome...great work!

Profilbild von You Jiacheng

You Jiachengvor 2 Jahren

How to compute the cycle consistency loss? G and G' don't have 1-1 correspondence (because of density control, avg&create), how to define F_f(G, t) = F_d(G', t) ? Also F should take (x, t) or their positional encoding as input, what does F(G, t) mean?

Profilbild von Isabella Liu

Isabella Liuvor 2 Jahren

Hi Youjia, thanks for your question! F(G, t) means the same thing as F(x, t). G’ should be the form of G under the deformed space, so they have 1-1 corresp.

Profilbild von You Jiacheng

You Jiachengvor 2 Jahren

Oh, there are two G' in the paper. The first one is the G under the deformed space, the second one is the G under the deformed space and anchored by averaging properties & creating new Gaussian at face centroid.

Profilbild von Rui Chen

Rui Chenvor 2 Jahren

Great work! Very high quality!

Profilbild von You Jiacheng

You Jiachengvor 2 Jahren

Awesome!

Profilbild von Dillon Claassens

Dillon Claassensvor 1 Jahr

Did you work for crucible labs?

Profilbild von Leo@Yuhao

Leo@Yuhaovor 2 Jahren

awesome work. Will you plan to release the raw iphone captured data and the script for generating the data? I tested the referred code but failed

Ähnliche Videos

3D Gaussian Splats are cool, but they're static (Part 25). DG-Mesh can reconstruct high-quality and time-consistent 3D meshes from a single video and track mesh vertices over time, which enables texture editing on dynamic objects.

3D Gaussian Splats are cool, but they're static (Part 25). DG-Mesh can reconstruct high-quality and time-consistent 3D meshes from a single video and track mesh vertices over time, which enables texture editing on dynamic objects.

Dreaming Tulpa 🥓👑

79,033 Aufrufe • vor 2 Jahren

"DVD: Dynamic Video Depth" TL;DR: Recovers temporally consistent depth from monocular videos using diffusion priors + geometric constraints, handling dynamic scenes and motion robustly.

"DVD: Dynamic Video Depth" TL;DR: Recovers temporally consistent depth from monocular videos using diffusion priors + geometric constraints, handling dynamic scenes and motion robustly.

Alexandre Morgand

11,823 Aufrufe • vor 7 Monaten

"Fast 4D Mesh Generation by Spatio-Temporal Attention Chains" TL;DR: training-free spatio-temporal attention chains generate topology-consistent 4D meshes from video 13× faster while improving temporal correspondence quality

"Fast 4D Mesh Generation by Spatio-Temporal Attention Chains" TL;DR: training-free spatio-temporal attention chains generate topology-consistent 4D meshes from video 13× faster while improving temporal correspondence quality

Alexandre Morgand

13,807 Aufrufe • vor 5 Monaten

Introducing MegaSaM! 🎥 Accurate, fast, & robust structure + camera estimation from casual monocular videos of dynamic scenes! MegaSaM outputs camera parameters and consistent video depth, scaling to long videos with unconstrained camera paths and complex scene dynamics!

Introducing MegaSaM! 🎥 Accurate, fast, & robust structure + camera estimation from casual monocular videos of dynamic scenes! MegaSaM outputs camera parameters and consistent video depth, scaling to long videos with unconstrained camera paths and complex scene dynamics!

Zhengqi Li

57,067 Aufrufe • vor 1 Jahr

Tripo P1.0 is live. Introducing Smart Mesh in Tripo Studio — generate structured 3D meshes in ~2 seconds. ⚡ ~2s generation 🧠 Smart topology 🛠 Production-ready assets 🌍 Built for game pipelines, real-time rendering, web 3D tools, and scalable content production From prompt to usable mesh — almost instantly. Try it: #3DAI #3DCG #3dart #Tripo

Tripo P1.0 is live. Introducing Smart Mesh in Tripo Studio — generate structured 3D meshes in ~2 seconds. ⚡ ~2s generation 🧠 Smart topology 🛠 Production-ready assets 🌍 Built for game pipelines, real-time rendering, web 3D tools, and scalable content production From prompt to usable mesh — almost instantly. Try it: #3DAI #3DCG #3dart #Tripo

Tripo

45,088 Aufrufe • vor 7 Monaten

(1/2) Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generation #ECCV2024! We show a theoretical derivation to create radiance fields directly from meshes. Thus, we can obtain GT training data for generative NeRF methods.

(1/2) Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generation #ECCV2024! We show a theoretical derivation to create radiance fields directly from meshes. Thus, we can obtain GT training data for generative NeRF methods.

Matthias Niessner

16,835 Aufrufe • vor 2 Jahren

We are thrilled to release the next leap in art-grade 3D generative models. Our single-click model pipeline gives unprecedented mesh outputs, with mesh parts-based topology. It is available now for all Cube tiers to start for free. ☑️ Our multi-stage hierarchical AI models produce a fully assembled 3D mesh with adaptive poly-counts, providing the clean, separated topology you need. ✅ A parts-based approach enables high-resolution meshes. 🌐 Quad and Triangle mesh support. Access now:

We are thrilled to release the next leap in art-grade 3D generative models. Our single-click model pipeline gives unprecedented mesh outputs, with mesh parts-based topology. It is available now for all Cube tiers to start for free. ☑️ Our multi-stage hierarchical AI models produce a fully assembled 3D mesh with adaptive poly-counts, providing the clean, separated topology you need. ✅ A parts-based approach enables high-resolution meshes. 🌐 Quad and Triangle mesh support. Access now:

Common Sense Machines

420,512 Aufrufe • vor 1 Jahr

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.

MrNeRF

34,782 Aufrufe • vor 1 Jahr

I'm excited to share our new work Align3R that estimates camera poses and consistent depth maps from a monocular video of a dynamic scene. Project page: Code: Paper:

I'm excited to share our new work Align3R that estimates camera poses and consistent depth maps from a monocular video of a dynamic scene. Project page: Code: Paper:

Yuan Liu

56,547 Aufrufe • vor 1 Jahr

"AgentSTAR: Agentic Shape Tracking and Reconstruction from Monocular Videos" VLM agent+optimization to reconstruct an object geometry+kinematic structure from RGB video. Pose+articulation through time including occlusions, transparent objects. No dense pixel correspondences.

"AgentSTAR: Agentic Shape Tracking and Reconstruction from Monocular Videos" VLM agent+optimization to reconstruct an object geometry+kinematic structure from RGB video. Pose+articulation through time including occlusions, transparent objects. No dense pixel correspondences.

Alexandre Morgand

25,126 Aufrufe • vor 1 Monat

MeshLRM Large Reconstruction Model for High-Quality Mesh We propose MeshLRM, a novel LRM-based approach that can reconstruct a high-quality mesh from merely four input images in less than one second. Different from previous large reconstruction models (LRMs) that focus on

MeshLRM Large Reconstruction Model for High-Quality Mesh We propose MeshLRM, a novel LRM-based approach that can reconstruct a high-quality mesh from merely four input images in less than one second. Different from previous large reconstruction models (LRMs) that focus on

AK

69,190 Aufrufe • vor 2 Jahren

"Bro trust me, AI-generated meshes will never be production-ready" The AI-generate mesh:

"Bro trust me, AI-generated meshes will never be production-ready" The AI-generate mesh:

Emm | scenario.com

942,608 Aufrufe • vor 1 Jahr

I prompted Omma to build a #threejs app with real dynamic lighting on a 3D mesh - generated from a single image, using Depth Anything v2 + Transformers from Hugging Face ⚡ Depth estimation in the browser 🫧 3D mesh from the depth map 💡 Dynamic lighting reacting to the geometry No manual code. Pure AI prompting. The future of 3D on the web feels wide open.

I prompted Omma to build a #threejs app with real dynamic lighting on a 3D mesh - generated from a single image, using Depth Anything v2 + Transformers from Hugging Face ⚡ Depth estimation in the browser 🫧 3D mesh from the depth map 💡 Dynamic lighting reacting to the geometry No manual code. Pure AI prompting. The future of 3D on the web feels wide open.

Joseph Azar

14,041 Aufrufe • vor 6 Monaten

Tip for deforming chest mesh: If you build the mesh topology to follow the lines of the chest and bra textures, it deforms really well! #Vtuber #Live2D

Tip for deforming chest mesh: If you build the mesh topology to follow the lines of the chest and bra textures, it deforms really well! #Vtuber #Live2D

Guilin (Live2D Rigger)

31,914 Aufrufe • vor 4 Monaten

Flexible Isosurface Extraction for Gradient-Based Mesh Optimization paper page: This work considers gradient-based mesh optimization, where we iteratively optimize for a 3D surface mesh by representing it as the isosurface of a scalar field, an increasingly common paradigm in applications including photogrammetry, generative modeling, and inverse physics. Existing implementations adapt classic isosurface extraction algorithms like Marching Cubes or Dual Contouring; these techniques were designed to extract meshes from fixed, known fields, and in the optimization setting they lack the degrees of freedom to represent high-quality feature-preserving meshes, or suffer from numerical instabilities. We introduce FlexiCubes, an isosurface representation specifically designed for optimizing an unknown mesh with respect to geometric, visual, or even physical objectives. Our main insight is to introduce additional carefully-chosen parameters into the representation, which allow local flexible adjustments to the extracted mesh geometry and connectivity. These parameters are updated along with the underlying scalar field via automatic differentiation when optimizing for a downstream task. We base our extraction scheme on Dual Marching Cubes for improved topological properties, and present extensions to optionally generate tetrahedral and hierarchically-adaptive meshes. Extensive experiments validate FlexiCubes on both synthetic benchmarks and real-world applications, showing that it offers significant improvements in mesh quality and geometric fidelity.

Flexible Isosurface Extraction for Gradient-Based Mesh Optimization paper page: This work considers gradient-based mesh optimization, where we iteratively optimize for a 3D surface mesh by representing it as the isosurface of a scalar field, an increasingly common paradigm in applications including photogrammetry, generative modeling, and inverse physics. Existing implementations adapt classic isosurface extraction algorithms like Marching Cubes or Dual Contouring; these techniques were designed to extract meshes from fixed, known fields, and in the optimization setting they lack the degrees of freedom to represent high-quality feature-preserving meshes, or suffer from numerical instabilities. We introduce FlexiCubes, an isosurface representation specifically designed for optimizing an unknown mesh with respect to geometric, visual, or even physical objectives. Our main insight is to introduce additional carefully-chosen parameters into the representation, which allow local flexible adjustments to the extracted mesh geometry and connectivity. These parameters are updated along with the underlying scalar field via automatic differentiation when optimizing for a downstream task. We base our extraction scheme on Dual Marching Cubes for improved topological properties, and present extensions to optionally generate tetrahedral and hierarchically-adaptive meshes. Extensive experiments validate FlexiCubes on both synthetic benchmarks and real-world applications, showing that it offers significant improvements in mesh quality and geometric fidelity.

AK

38,455 Aufrufe • vor 3 Jahren

📢TriFlow: Generating Artist-Like 3D Mesh Topology via Nearest-Vertex Vector Fields (ECCV'26)📢 Compact 3D meshes with clean, artist-like triangle topology - structured connectivity you'd expect a human modeler to make🙂 🌐 ▶️

📢TriFlow: Generating Artist-Like 3D Mesh Topology via Nearest-Vertex Vector Fields (ECCV'26)📢 Compact 3D meshes with clean, artist-like triangle topology - structured connectivity you'd expect a human modeler to make🙂 🌐 ▶️

Matthias Niessner

116,176 Aufrufe • vor 3 Monaten

DeepMesh is out on Hugging Face Auto-Regressive Artist-mesh Creation with Reinforcement Learning Conditioned on point clouds and images, DeepMesh generates meshes with intricate details and precise topology, outperforming state-of-the-art methods in both precision and quality.

DeepMesh is out on Hugging Face Auto-Regressive Artist-mesh Creation with Reinforcement Learning Conditioned on point clouds and images, DeepMesh generates meshes with intricate details and precise topology, outperforming state-of-the-art methods in both precision and quality.

AK

109,272 Aufrufe • vor 1 Jahr

"MeshFlow: Efficient Artistic Mesh Generation via MeshVAE and Flow-based Diffusion Transformer" TL;DR: learns a continuous mesh latent space and generates vertices and connectivity in parallel with flow matching, producing quality 3D meshes up to 18× faster than autoregressive.

"MeshFlow: Efficient Artistic Mesh Generation via MeshVAE and Flow-based Diffusion Transformer" TL;DR: learns a continuous mesh latent space and generates vertices and connectivity in parallel with flow matching, producing quality 3D meshes up to 18× faster than autoregressive.

Alexandre Morgand

43,375 Aufrufe • vor 4 Monaten