Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

(1/2) Excited to share "Learning Neural Parametric Head Models" #CVPR2023! We capture over 5200 high-quality 3D human head scans from which we build a neural parametric head model that disentangles & expressions and deformations.

MattNiessner's profile picture

Matthias Niessner

44,676 subscribers

53,279 Aufrufe • vor 3 Jahren •via X (Twitter)

KunstWissenschaft & TechnologieBildung#CVPR2023
LIVE

Anya Rossi• Live Now

Private livecam show

2 Kommentare

Profilbild von Matthias Niessner
Matthias Niessnervor 3 Jahren

(2/2) Core of our representation is an ensemble of local MLPs to facilitate high geometric detail. We will release the entire data asap! Great work by @SGiebenhain. Also super proud by the collab with @synthesiaIO kicking off the first academic publication of our research team!

Profilbild von Reza Babaee
Reza Babaeevor 3 Jahren

Is ear also included?

Ähnliche Videos

HACK: Learning a Parametric Head and Neck Model for High-fidelity Animation abs: paper page:
2:41

HACK: Learning a Parametric Head and Neck Model for High-fidelity Animation abs: paper page:

AK

90,154 Aufrufe • vor 3 Jahren

(1/2) 📢📢𝐃𝐢𝐟𝐟𝐮𝐬𝐢𝐨𝐧𝐀𝐯𝐚𝐭𝐚𝐫𝐬 📢📢 High-fidelity 3D head avatars with precise control over viewpoint, expression, and pose. -> Our parametric 3D model enables control & consistency + 2D diffusion makes it photoreal.
0:32

(1/2) 📢📢𝐃𝐢𝐟𝐟𝐮𝐬𝐢𝐨𝐧𝐀𝐯𝐚𝐭𝐚𝐫𝐬 📢📢 High-fidelity 3D head avatars with precise control over viewpoint, expression, and pose. -> Our parametric 3D model enables control & consistency + 2D diffusion makes it photoreal.

Matthias Niessner

66,414 Aufrufe • vor 2 Jahren

Champ Controllable and Consistent Human Image Animation with 3D Parametric Guidance In this study, we introduce a methodology for human image animation by leveraging a 3D human parametric model within a latent diffusion framework to enhance shape alignment and motion
2:10

Champ Controllable and Consistent Human Image Animation with 3D Parametric Guidance In this study, we introduce a methodology for human image animation by leveraging a 3D human parametric model within a latent diffusion framework to enhance shape alignment and motion

AK

194,356 Aufrufe • vor 2 Jahren

We relived Michelangelo in neural network style!! Super excited to finally share Neuralangelo 🛠 -- 3D surface reconstruction from RGB videos with extremely high fidelity! 🤩 To appear at #CVPR2023 project page: paper: 🧵⬇️ (1/4)
0:24

We relived Michelangelo in neural network style!! Super excited to finally share Neuralangelo 🛠 -- 3D surface reconstruction from RGB videos with extremely high fidelity! 🤩 To appear at #CVPR2023 project page: paper: 🧵⬇️ (1/4)

Chen-Hsuan Lin

253,350 Aufrufe • vor 3 Jahren

[SIGGRAPH '25] EVA: Expressive Virtual Avatars from Multi-view Videos Contributions: 1. We introduce EVA, a novel method enabling full-body control with real-time, photo-realistic renderings, robustly handling loose clothing dynamics and various facial expressions. 2. We develop an expressive deformable template that generates a deformable human template mesh and employs a multi-stage tracking algorithm to faithfully capture facial expressions, body motions, and non-rigid deformations from multi-view videos. 3. We propose a disentangled 3D Gaussian appearance module that models the body and face independently, ensuring separated control and high-quality renderings.
1:22

[SIGGRAPH '25] EVA: Expressive Virtual Avatars from Multi-view Videos Contributions: 1. We introduce EVA, a novel method enabling full-body control with real-time, photo-realistic renderings, robustly handling loose clothing dynamics and various facial expressions. 2. We develop an expressive deformable template that generates a deformable human template mesh and employs a multi-stage tracking algorithm to faithfully capture facial expressions, body motions, and non-rigid deformations from multi-view videos. 3. We propose a disentangled 3D Gaussian appearance module that models the body and face independently, ensuring separated control and high-quality renderings.

MrNeRF

18,407 Aufrufe • vor 1 Jahr

📢BecomingLit: Relightable Gaussian Avatars with Hybrid Neural Shading We propose a hybrid neural shading scheme for creating intrinsically decomposed 3DGS head avatars, that allow real-time relighting and animation. 🌍 📷
0:24

📢BecomingLit: Relightable Gaussian Avatars with Hybrid Neural Shading We propose a hybrid neural shading scheme for creating intrinsically decomposed 3DGS head avatars, that allow real-time relighting and animation. 🌍 📷

Matthias Niessner

23,019 Aufrufe • vor 1 Jahr

Excited to share our work on Neural Assets: a new method for enabling 3D asset-level control in image diffusion models – scalable & without any 3D inductive biases. Neural Assets goes beyond text or pixel-based control & provides an interface inspired by 3D graphics tools. 🧵
0:30

Excited to share our work on Neural Assets: a new method for enabling 3D asset-level control in image diffusion models – scalable & without any 3D inductive biases. Neural Assets goes beyond text or pixel-based control & provides an interface inspired by 3D graphics tools. 🧵

Thomas Kipf

97,924 Aufrufe • vor 2 Jahren

Current 3D generative models are slow and low quality. We present GRM, a large-scale model that reconstructs 3D Gaussians in 0.1s and generates high-quality 3D assets from text or single images in a few seconds. Demo: 1/4
0:37

Current 3D generative models are slow and low quality. We present GRM, a large-scale model that reconstructs 3D Gaussians in 0.1s and generates high-quality 3D assets from text or single images in a few seconds. Demo: 1/4

Gordon Wetzstein

19,210 Aufrufe • vor 2 Jahren

Can we learn a 3D world model that predicts object dynamics directly from videos? Introducing Particle-Grid Neural Dynamics: a learning-based simulator for deformable objects that trains from real-world videos. Website: ArXiv: Code: Demo: To appear at #RSS2025
0:34

Can we learn a 3D world model that predicts object dynamics directly from videos? Introducing Particle-Grid Neural Dynamics: a learning-based simulator for deformable objects that trains from real-world videos. Website: ArXiv: Code: Demo: To appear at #RSS2025

Kaifeng Zhang

45,971 Aufrufe • vor 1 Jahr

(1/2) 📢📢📢NeRSemble #SIGGRAPH'23! We reconstruct dynamic radiance fields for high-quality novel view synthesis of human heads. Key is a deformation field and an ensemble of multi-resolution hash encodings to model coarse & fine-scale deformations.
0:35

(1/2) 📢📢📢NeRSemble #SIGGRAPH'23! We reconstruct dynamic radiance fields for high-quality novel view synthesis of human heads. Key is a deformation field and an ensemble of multi-resolution hash encodings to model coarse & fine-scale deformations.

Matthias Niessner

69,652 Aufrufe • vor 3 Jahren

🔥Text-to-3D Foundation Model🔥 We are excited to announce #3DTopia, a generalist 🧊text-to-3D🧊 foundation model, which produces ** high-quality 3D assets within 5 minutes ** - Code: - Video:
0:58

🔥Text-to-3D Foundation Model🔥 We are excited to announce #3DTopia, a generalist 🧊text-to-3D🧊 foundation model, which produces ** high-quality 3D assets within 5 minutes ** - Code: - Video:

Ziwei Liu

62,424 Aufrufe • vor 2 Jahren

📢📢 𝐀𝐯𝐚𝐭𝟑𝐫 📢📢 Avat3r creates high-quality 3D head avatars from just a few input images in a single forward pass with a new dynamic 3DGS reconstruction model. Video: Project: Our core idea is to make Gaussian Reconstruction Models animatable. We find that a simple cross-attention to an expression code sequence is already sufficient to model complex facial expressions. We then incorporate position maps from DUSt3R and feature maps from Sapiens to facilitate the prediction task. While DUSt3R's position maps act as a pixel-aligned initialization for the Gaussians' positions, the Sapiens feature maps help the cross-view transformer to match corresponding image tokens in the 4 input images. One major challenge in creating a 3D head avatar from smartphone images comes from inconsistent facial expressions when the subject could not remain perfectly static during the capture. We eliminate this static requirement by simply showing our model input images with different facial expressions during training. This technique makes our model robust to inconsistent input images later on. Finally, we show that despite the model has been trained with 4 input images, one can even create a 3D head avatar when only a single image is available. To achieve this, we employ a pre-trained 3D GAN to lift the single image to 3D and then render the 4 input images for our model. This allows us to create 3D head avatars from single images and even highly out-of-distribution examples like AI generated faces, paintings or statues. Great work by Tobias Kirschstein from his internship at Meta with Javier Romero, Artem Sevastopolsky, and Shunsuke Saito
0:38

📢📢 𝐀𝐯𝐚𝐭𝟑𝐫 📢📢 Avat3r creates high-quality 3D head avatars from just a few input images in a single forward pass with a new dynamic 3DGS reconstruction model. Video: Project: Our core idea is to make Gaussian Reconstruction Models animatable. We find that a simple cross-attention to an expression code sequence is already sufficient to model complex facial expressions. We then incorporate position maps from DUSt3R and feature maps from Sapiens to facilitate the prediction task. While DUSt3R's position maps act as a pixel-aligned initialization for the Gaussians' positions, the Sapiens feature maps help the cross-view transformer to match corresponding image tokens in the 4 input images. One major challenge in creating a 3D head avatar from smartphone images comes from inconsistent facial expressions when the subject could not remain perfectly static during the capture. We eliminate this static requirement by simply showing our model input images with different facial expressions during training. This technique makes our model robust to inconsistent input images later on. Finally, we show that despite the model has been trained with 4 input images, one can even create a 3D head avatar when only a single image is available. To achieve this, we employ a pre-trained 3D GAN to lift the single image to 3D and then render the 4 input images for our model. This allows us to create 3D head avatars from single images and even highly out-of-distribution examples like AI generated faces, paintings or statues. Great work by Tobias Kirschstein from his internship at Meta with Javier Romero, Artem Sevastopolsky, and Shunsuke Saito

Matthias Niessner

74,698 Aufrufe • vor 1 Jahr

📢ProcGen3D: Learning Neural Procedural Graphs for Image-to-3D Reconstruction Xinyi Zhang learns neural procedural graphs to generate high-fidelity 3D - MCTS-guided sampling maintains consistency with the input image, even from real images! Check it out:
0:38

📢ProcGen3D: Learning Neural Procedural Graphs for Image-to-3D Reconstruction Xinyi Zhang learns neural procedural graphs to generate high-fidelity 3D - MCTS-guided sampling maintains consistency with the input image, even from real images! Check it out:

Angela Dai

14,634 Aufrufe • vor 8 Monaten

Going to #CVPR2023 ? Checkout our tutorial on Efficient Neural Networks: From Algorithm Design to Practical Mobile Deployment. We’ll do a deep dive into mobile transformers, mobile neural rendering and SnapFusion--the fastest mobile text-to-image model. Our content in 32 seconds
0:32

Going to #CVPR2023 ? Checkout our tutorial on Efficient Neural Networks: From Algorithm Design to Practical Mobile Deployment. We’ll do a deep dive into mobile transformers, mobile neural rendering and SnapFusion--the fastest mobile text-to-image model. Our content in 32 seconds

Sergey Tulyakov

23,365 Aufrufe • vor 3 Jahren

We also present another paper at @SIGGRAPH 2023 on neural implicit 3D Morphable Models that can be used to create a dynamic 3D avatar from a single in-the-wild image. (Lead author Connor Lin).
0:39

We also present another paper at @SIGGRAPH 2023 on neural implicit 3D Morphable Models that can be used to create a dynamic 3D avatar from a single in-the-wild image. (Lead author Connor Lin).

Koki Nagano

12,775 Aufrufe • vor 3 Jahren

You can build your 3D models directly from a prompt into Claude code. All thanks to this beautiful python project, build123 a parametric modeling framework for 2D and 3D CAD. It is built on the Open Cascade geometric kernel and it is suitable for 3D printing.
0:36

You can build your 3D models directly from a prompt into Claude code. All thanks to this beautiful python project, build123 a parametric modeling framework for 2D and 3D CAD. It is built on the Open Cascade geometric kernel and it is suitable for 3D printing.

Marco Franzon

20,777 Aufrufe • vor 3 Monaten

capability != learning new benchtalks with Parth Asawa on continual learning, where we discuss teaching models to learn from experience, measuring learning ability, the bet on parametric models, and more 01:06 What is continual learning? 04:10 Why capability and learning are different 06:13 Why build a benchmark? 08:07 Continual Learning Bench launch and reception 09:13 Anthropic's Fable release and Continual Learning Bench 11:02 How to design tasks for continual learning 18:41 The gain metric 24:01 What good looks like on the leaderboard 29:13 Failure modes: why models can't update their beliefs 31:12 Parametric systems and future architectures 34:30 Open science and AI safety 45:42 Lightning round 49:08 How to contribute to Continual Learning Bench
50:35

capability != learning new benchtalks with Parth Asawa on continual learning, where we discuss teaching models to learn from experience, measuring learning ability, the bet on parametric models, and more 01:06 What is continual learning? 04:10 Why capability and learning are different 06:13 Why build a benchmark? 08:07 Continual Learning Bench launch and reception 09:13 Anthropic's Fable release and Continual Learning Bench 11:02 How to design tasks for continual learning 18:41 The gain metric 24:01 What good looks like on the leaderboard 29:13 Failure modes: why models can't update their beliefs 31:12 Parametric systems and future architectures 34:30 Open science and AI safety 45:42 Lightning round 49:08 How to contribute to Continual Learning Bench

vincent sunn chen

39,598 Aufrufe • vor 16 Tagen

Introducing Neural Jacobian Fields, robot 3D kinematic models learned only from vision! They can model & control robots from just a single RGB camera, even those w/ intractable kinematics & no embedded sensors such as soft, 3D-printed pneumatic hands! 1/n
0:55

Introducing Neural Jacobian Fields, robot 3D kinematic models learned only from vision! They can model & control robots from just a single RGB camera, even those w/ intractable kinematics & no embedded sensors such as soft, 3D-printed pneumatic hands! 1/n

Vincent Sitzmann

54,204 Aufrufe • vor 1 Jahr

3D scans of yourself are incredibly handy for cosplay crafting! We used a scan of my head and body to bring the Revenant from Apex Legends to life! 😎 Apex Legends #diycrafts
0:59

3D scans of yourself are incredibly handy for cosplay crafting! We used a scan of my head and body to bring the Revenant from Apex Legends to life! 😎 Apex Legends #diycrafts

Kamui 📚

27,290 Aufrufe • vor 1 Jahr

🚨 New Paper Hybrid Neural World Models. We build neural world models for discontinuous environments and our method discovers when they're making error, with no ground truth. Our neural models run physics fast, leaping many simulator steps in a single pass. How we find points where not to trust it: ask the model for the same future two ways, and the disagreement between the two answers marks where it's failing without needing labels. The same recipe holds on three systems with nothing in common: a chemical wave, compressible gas with shocks, rigid bodies colliding in 3D. 👇
0:46

🚨 New Paper Hybrid Neural World Models. We build neural world models for discontinuous environments and our method discovers when they're making error, with no ground truth. Our neural models run physics fast, leaping many simulator steps in a single pass. How we find points where not to trust it: ask the model for the same future two ways, and the disagreement between the two answers marks where it's failing without needing labels. The same recipe holds on three systems with nothing in common: a chemical wave, compressible gas with shocks, rigid bodies colliding in 3D. 👇

Lossfunk

14,828 Aufrufe • vor 1 Monat