Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

(1/2) We released our Neural Parametric Head Models (NPHM) dataset from our #CVPR2023 paper! It includes over 5600 high-fidelity 3D scans of human heads from 272 subjects - all publicly available! Check it out!

Matthias Niessner

44,676 subscribers

36,014 views • 3 years ago •via X (Twitter)

Science & Technology Education #CVPR2023

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

(1/2) MonoNPHM will be presented as a #CVPR2024 Highlight! Our Neural Parametric Head Model parametrizes both geometry and appearance. With the learned model, we can then 3D reconstruct and track human heads from images or videos.

(1/2) MonoNPHM will be presented as a #CVPR2024 Highlight! Our Neural Parametric Head Model parametrizes both geometry and appearance. With the learned model, we can then 3D reconstruct and track human heads from images or videos.

Matthias Niessner

17,209 views • 2 years ago

(1/2) 📢𝐍𝐏𝐆𝐀: 𝐍𝐞𝐮𝐫𝐚𝐥 𝐏𝐚𝐫𝐚𝐦𝐞𝐭𝐫𝐢𝐜 𝐆𝐚𝐮𝐬𝐬𝐢𝐚𝐧 𝐀𝐯𝐚𝐭𝐚𝐫𝐬 📢 #SIGGRAPHAsia We leverage a neural parametric representation to facilitate precise control over 3D Gaussians to obtain high-fidelity avatars.

(1/2) 📢𝐍𝐏𝐆𝐀: 𝐍𝐞𝐮𝐫𝐚𝐥 𝐏𝐚𝐫𝐚𝐦𝐞𝐭𝐫𝐢𝐜 𝐆𝐚𝐮𝐬𝐬𝐢𝐚𝐧 𝐀𝐯𝐚𝐭𝐚𝐫𝐬 📢 #SIGGRAPHAsia We leverage a neural parametric representation to facilitate precise control over 3D Gaussians to obtain high-fidelity avatars.

Matthias Niessner

30,741 views • 1 year ago

We have released the code and weights for our #CVPR2023 paper "Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model"! code: abs: project: The demo is below:

We have released the code and weights for our #CVPR2023 paper "Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model"! code: abs: project: The demo is below:

Artsiom Sanakoyeu

35,710 views • 3 years ago

(1/2) Check out 𝐆𝐚𝐮𝐬𝐬𝐢𝐚𝐧𝐀𝐯𝐚𝐭𝐚𝐫𝐬: Photorealistic Head Avatars with Rigged 3D Gaussians! We create photorealistic head avatars by animating 3D Gaussians on a parametric face model - edited and rendered in *real-time*!

(1/2) Check out 𝐆𝐚𝐮𝐬𝐬𝐢𝐚𝐧𝐀𝐯𝐚𝐭𝐚𝐫𝐬: Photorealistic Head Avatars with Rigged 3D Gaussians! We create photorealistic head avatars by animating 3D Gaussians on a parametric face model - edited and rendered in real-time!

Matthias Niessner

126,971 views • 2 years ago

📢📢𝐍𝐞𝐑𝐒𝐞𝐦𝐛𝐥𝐞 𝐯𝟐 𝐃𝐚𝐭𝐚𝐬𝐞𝐭 𝐑𝐞𝐥𝐞𝐚𝐬𝐞📢📢 Head captures of 7.1MP from 16 cameras at 73fps: * More recordings (425 people) * Better color calibration * Convenient download scripts The new version of our dataset adds 156 participants for a total of 425 different people. In its entirety, the dataset provides now 65 million images from over 15 hours of diverse human facial expression performances. We improved the color consistency of the recorded images with a better color calibration procedure. As a result, 3D reconstructions with images from the NeRSemble dataset should become better and look more realistic. Finally, we made it much easier to download the recordings with our new download repository. It now just takes a single command to download all frontal hair shake videos of all participants or to download all recordings of a single participant. Check it out: Awesome work by Tobias Kirschstein Simon Giebenhain !!!

📢📢𝐍𝐞𝐑𝐒𝐞𝐦𝐛𝐥𝐞 𝐯𝟐 𝐃𝐚𝐭𝐚𝐬𝐞𝐭 𝐑𝐞𝐥𝐞𝐚𝐬𝐞📢📢 Head captures of 7.1MP from 16 cameras at 73fps: * More recordings (425 people) * Better color calibration * Convenient download scripts The new version of our dataset adds 156 participants for a total of 425 different people. In its entirety, the dataset provides now 65 million images from over 15 hours of diverse human facial expression performances. We improved the color consistency of the recorded images with a better color calibration procedure. As a result, 3D reconstructions with images from the NeRSemble dataset should become better and look more realistic. Finally, we made it much easier to download the recordings with our new download repository. It now just takes a single command to download all frontal hair shake videos of all participants or to download all recordings of a single participant. Check it out: Awesome work by Tobias Kirschstein Simon Giebenhain !!!

Matthias Niessner

12,089 views • 1 year ago

📢Pix2NPHM: Learning to Regress NPHM Reconstructions From a Single Image📢 We directly regress neural parametric head models (NPHMs) from a single image — fast, stable, and significantly more expressive than classical 3DMMs such as FLAME. Face tracking & 3D reconstruction are often limited by the representational capacity of PCA-based face models. By lifting NPHMs to a first-class reconstruction primitive, we enable more accurate geometry, richer expressions, and finer animation control. Pix2NPHM obtains fast and reliable NPHM reconstructions on real-world data. Inference-time optimization against surface normals and canonical point maps can further increase fidelity. Key to successful and generalized training of our ViT-based network are: (1) large-scale registration of existing 3D head datasets, and (2) self-supervised training on vast in-the-wild 2D video datasets using pseudo ground-truth surface normals. Finally, we show that geometry-aware pretraining on pixel-aligned reconstruction tasks significantly outperforms generic visual pretraining (e.g., DINO-style features) in terms of generalization. 🌍 🎥 Great work by Simon Giebenhain, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Zhe Chen

📢Pix2NPHM: Learning to Regress NPHM Reconstructions From a Single Image📢 We directly regress neural parametric head models (NPHMs) from a single image — fast, stable, and significantly more expressive than classical 3DMMs such as FLAME. Face tracking & 3D reconstruction are often limited by the representational capacity of PCA-based face models. By lifting NPHMs to a first-class reconstruction primitive, we enable more accurate geometry, richer expressions, and finer animation control. Pix2NPHM obtains fast and reliable NPHM reconstructions on real-world data. Inference-time optimization against surface normals and canonical point maps can further increase fidelity. Key to successful and generalized training of our ViT-based network are: (1) large-scale registration of existing 3D head datasets, and (2) self-supervised training on vast in-the-wild 2D video datasets using pseudo ground-truth surface normals. Finally, we show that geometry-aware pretraining on pixel-aligned reconstruction tasks significantly outperforms generic visual pretraining (e.g., DINO-style features) in terms of generalization. 🌍 🎥 Great work by Simon Giebenhain, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Zhe Chen

Matthias Niessner

37,784 views • 6 months ago

Introducing Objaverse, a massive open dataset of text-paired 3D objects! Nearly 1 million annotated 3D objects to pave the way to build incredible large-scale 3D generative models: 🧵👇 🤗 Hugging Face: 📝ArXiv: #CVPR2023

Introducing Objaverse, a massive open dataset of text-paired 3D objects! Nearly 1 million annotated 3D objects to pave the way to build incredible large-scale 3D generative models: 🧵👇 🤗 Hugging Face: 📝ArXiv: #CVPR2023

Matt Deitke

1,600,840 views • 3 years ago

Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians paper page: Creating high-fidelity 3D head avatars has always been a research hotspot, but there remains a great challenge under lightweight sparse view setups. In this paper, we propose Gaussian Head Avatar represented by controllable 3D Gaussians for high-fidelity head avatar modeling. We optimize the neutral 3D Gaussians and a fully learned MLP-based deformation field to capture complex expressions. The two parts benefit each other, thereby our method can model fine-grained dynamic details while ensuring expression accuracy. Furthermore, we devise a well-designed geometry-guided initialization strategy based on implicit SDF and Deep Marching Tetrahedra for the stability and convergence of the training procedure. Experiments show our approach outperforms other state-of-the-art sparse-view methods, achieving ultra high-fidelity rendering quality at 2K resolution even under exaggerated expressions.

AK

65,834 views • 2 years ago

What is the best architecture for robotic manipulation? In our new CoRL 2023 paper Act3D, we efficiently produce high-resolution 3D action maps from pre-trained 2D backbones. (1/n)

What is the best architecture for robotic manipulation? In our new CoRL 2023 paper Act3D, we efficiently produce high-resolution 3D action maps from pre-trained 2D backbones. (1/n)

Theophile Gervet

44,863 views • 2 years ago

(1/2) Check out "𝐏𝐨𝐥𝐲𝐃𝐢𝐟𝐟: Generating 3D Polygonal Meshes with Diffusion Models"! Our model operates directly on the polygons of 3D meshes and generates novel shapes as output through an iterative diffusion process.

(1/2) Check out "𝐏𝐨𝐥𝐲𝐃𝐢𝐟𝐟: Generating 3D Polygonal Meshes with Diffusion Models"! Our model operates directly on the polygons of 3D meshes and generates novel shapes as output through an iterative diffusion process.

Matthias Niessner

57,941 views • 2 years ago

Diffusion models are great, but we can squeeze out so much more from them. The only problem is that it usually requires extra training or manual representation editing. In our new paper, we show that with the current capabilities of LLMs, it is much simpler than we thought!

Diffusion models are great, but we can squeeze out so much more from them. The only problem is that it usually requires extra training or manual representation editing. In our new paper, we show that with the current capabilities of LLMs, it is much simpler than we thought!

Yossi Gandelsman

34,813 views • 3 months ago

This is completely nuts. Can't wait until the paper is released! "SuperGaussian: Repurposing Video Models for 3D Super Resolution" Project: Paper video ⬇️ 1 I 2

This is completely nuts. Can't wait until the paper is released! "SuperGaussian: Repurposing Video Models for 3D Super Resolution" Project: Paper video ⬇️ 1 I 2

MrNeRF

78,219 views • 2 years ago

Just released a breakdown video on how I made Modeus from Helltaker 3D, check it out now! :) 💖✨ #blender3d #lowpoly #3danimation

Just released a breakdown video on how I made Modeus from Helltaker 3D, check it out now! :) 💖✨ #blender3d #lowpoly #3danimation

Crashsune

214,916 views • 1 year ago

Over the Reality just hit another major milestone in the AI era. We have now surpassed 200,000 mappings in our dataset. 🎉 These are not just images. They are high-quality 3D maps generated from the mapping activity of our global community, which has produced more than 94M images of real-world locations so far. The dataset also includes 900TB+ of spatial data that can power Visual AI models, Robotics VPS for real-world navigation and positioning, XR VPS for precise spatial anchoring and AR experiences, Spatial Computing applications, Digital Twins, and the next generation of real-world AI infrastructure. In just the past 2 weeks, our dataset grew by +17,955 mapped locations. That is nearly +10% growth in just 14 days. More maps. More data. More demand for $OVR.

Over the Reality just hit another major milestone in the AI era. We have now surpassed 200,000 mappings in our dataset. 🎉 These are not just images. They are high-quality 3D maps generated from the mapping activity of our global community, which has produced more than 94M images of real-world locations so far. The dataset also includes 900TB+ of spatial data that can power Visual AI models, Robotics VPS for real-world navigation and positioning, XR VPS for precise spatial anchoring and AR experiences, Spatial Computing applications, Digital Twins, and the next generation of real-world AI infrastructure. In just the past 2 weeks, our dataset grew by +17,955 mapped locations. That is nearly +10% growth in just 14 days. More maps. More data. More demand for $OVR.

Over the Reality 🌐

926,400 views • 3 months ago

‘I Don’t Want A Lover’ was our debut single, released in 1989 and taken from our first album Southside. It was also our first ever UK top 10. The video is now available in high res on our YouTube channel for the first time ever. Watch it now here:

‘I Don’t Want A Lover’ was our debut single, released in 1989 and taken from our first album Southside. It was also our first ever UK top 10. The video is now available in high res on our YouTube channel for the first time ever. Watch it now here:

Texas

12,818 views • 3 years ago

🤩 Those touchline reactions Check out our latest Bench Cam for all the key moments from our London derby win over Chelsea 👇

🤩 Those touchline reactions Check out our latest Bench Cam for all the key moments from our London derby win over Chelsea 👇

Arsenal

350,133 views • 3 years ago

We just dropped our new website Check it out

We just dropped our new website Check it out

Fluton

19,703 views • 3 months ago

Check out our latest project which is now available on Behance! By far our most ambitious project so far. Check it out here:

Check out our latest project which is now available on Behance! By far our most ambitious project so far. Check it out here:

North Studio

17,422 views • 2 years ago

A dream of our lab has been to image the full central dogma from a single endogenous gene, all live and with single molecule resolution. After many years we are happy to unveil a beautiful cell line that makes it possible. Check out our preprint ( (1/n)

A dream of our lab has been to image the full central dogma from a single endogenous gene, all live and with single molecule resolution. After many years we are happy to unveil a beautiful cell line that makes it possible. Check out our preprint ( (1/n)

Timothy

206,264 views • 2 years ago