正在加载视频...

视频加载失败

返回首页

🚀Announcing NeRSemble 3D Head Avatar Benchmark v2 Version 2 of the NeRSemble 3D Head Avatar Benchmark systematically evaluates several aspects of 3D head avatar creation. Our goal is to drive progress toward more realistic, robust, and generalizable avatar methods. 🔬Benchmark Tasks The NeRSemble Benchmark v2 features three core challenges:...

MattNiessner's profile picture

Matthias Niessner

47,901 subscribers

29,874 次观看 • 2 个月前 •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Want to create an avatar from a single image? FlexAvatar is a transformer model that creates full 360°, high-quality, and expressive 3D head avatar from just a single portrait image in minutes. Real-time Demo: FlexAvatar's lightweight architecture allows both animation and rendering in real-time, enabling interactive user experiences. To create a new 3D head avatar, only one image is required, e.g., from a webcam. The final avatar is ready after 2 minutes. Architecture: Under the hood, FlexAvatar adopts a transformer-based encoder-decoder design. The encoder maps the input image onto a latent avatar space, while the decoder produces 3D Gaussian attribute maps by incorporating the animation signal via cross-attention. The model learns all facial animations directly from the data without relying on pre-built 3D face models. This equips the avatars with realistic facial expressions. The internal avatar latent space can be conveniently used to integrate additional observations of a person via fitting. This enables use-cases where more than one image of a person is available, e.g., from a phone scan of the person. We train jointly on 2D monocular videos and multi-view data. However, in monocular videos, the animation signal leaks the target viewpoint, causing the model to produce incomplete 3D heads. We call this phenomenon entanglement of driving signal and target viewpoint. To prevent entanglement, we introduce bias sinks. These are learnable tokens that indicate whether a training sample stems from a monocular or a multi-view dataset. During training, the model learns to produce incomplete 3D heads only when the monocular token is present. During inference, FlexAvatar then always uses the multi-view token for which the model has learned to produce complete 3D heads. This simple design allows to combine the generalizability from monocular data with the quality of multi-view data. FlexAvatar summary: - Input: Single-image, phone scan, or monocular video - Output: Full 360° head avatar - Expressive animations - Real-time rendering and animation - Generalization to any portrait - Create a new avatar in 2 minutes - Use bias sinks to combine 2D and 3D data 🏠 🌍 🎥 Great work by Tobias Kirschstein and Simon Giebenhain!
0:30

Want to create an avatar from a single image? FlexAvatar is a transformer model that creates full 360°, high-quality, and expressive 3D head avatar from just a single portrait image in minutes. Real-time Demo: FlexAvatar's lightweight architecture allows both animation and rendering in real-time, enabling interactive user experiences. To create a new 3D head avatar, only one image is required, e.g., from a webcam. The final avatar is ready after 2 minutes. Architecture: Under the hood, FlexAvatar adopts a transformer-based encoder-decoder design. The encoder maps the input image onto a latent avatar space, while the decoder produces 3D Gaussian attribute maps by incorporating the animation signal via cross-attention. The model learns all facial animations directly from the data without relying on pre-built 3D face models. This equips the avatars with realistic facial expressions. The internal avatar latent space can be conveniently used to integrate additional observations of a person via fitting. This enables use-cases where more than one image of a person is available, e.g., from a phone scan of the person. We train jointly on 2D monocular videos and multi-view data. However, in monocular videos, the animation signal leaks the target viewpoint, causing the model to produce incomplete 3D heads. We call this phenomenon entanglement of driving signal and target viewpoint. To prevent entanglement, we introduce bias sinks. These are learnable tokens that indicate whether a training sample stems from a monocular or a multi-view dataset. During training, the model learns to produce incomplete 3D heads only when the monocular token is present. During inference, FlexAvatar then always uses the multi-view token for which the model has learned to produce complete 3D heads. This simple design allows to combine the generalizability from monocular data with the quality of multi-view data. FlexAvatar summary: - Input: Single-image, phone scan, or monocular video - Output: Full 360° head avatar - Expressive animations - Real-time rendering and animation - Generalization to any portrait - Create a new avatar in 2 minutes - Use bias sinks to combine 2D and 3D data 🏠 🌍 🎥 Great work by Tobias Kirschstein and Simon Giebenhain!

Matthias Niessner

95,431 次观看 • 6 个月前

📢Pixel3DMM: Versatile Screen-Space Priors for Single-Image 3D Face Reconstruction📢 -> highly accurate face reconstruction by training powerful VITs via surface normals and UV-coordinates estimation. The geometric cues from our 2D foundation model backbone constrain the 3DMM parameters, which allows us to achieve remarkable reconstruction accuracy - works for both single image and videos! In addition, we introduce a new 3D face reconstruction benchmark that evaluates both neutral and posed face geometry. 🌍 📷 Great work by Simon Giebenhain Tobias Kirschstein Martin Rünz Lourdes Agapito
0:32

📢Pixel3DMM: Versatile Screen-Space Priors for Single-Image 3D Face Reconstruction📢 -> highly accurate face reconstruction by training powerful VITs via surface normals and UV-coordinates estimation. The geometric cues from our 2D foundation model backbone constrain the 3DMM parameters, which allows us to achieve remarkable reconstruction accuracy - works for both single image and videos! In addition, we introduce a new 3D face reconstruction benchmark that evaluates both neutral and posed face geometry. 🌍 📷 Great work by Simon Giebenhain Tobias Kirschstein Martin Rünz Lourdes Agapito

Matthias Niessner

62,104 次观看 • 1 年前

📢📢 𝐀𝐯𝐚𝐭𝟑𝐫 📢📢 Avat3r creates high-quality 3D head avatars from just a few input images in a single forward pass with a new dynamic 3DGS reconstruction model. Video: Project: Our core idea is to make Gaussian Reconstruction Models animatable. We find that a simple cross-attention to an expression code sequence is already sufficient to model complex facial expressions. We then incorporate position maps from DUSt3R and feature maps from Sapiens to facilitate the prediction task. While DUSt3R's position maps act as a pixel-aligned initialization for the Gaussians' positions, the Sapiens feature maps help the cross-view transformer to match corresponding image tokens in the 4 input images. One major challenge in creating a 3D head avatar from smartphone images comes from inconsistent facial expressions when the subject could not remain perfectly static during the capture. We eliminate this static requirement by simply showing our model input images with different facial expressions during training. This technique makes our model robust to inconsistent input images later on. Finally, we show that despite the model has been trained with 4 input images, one can even create a 3D head avatar when only a single image is available. To achieve this, we employ a pre-trained 3D GAN to lift the single image to 3D and then render the 4 input images for our model. This allows us to create 3D head avatars from single images and even highly out-of-distribution examples like AI generated faces, paintings or statues. Great work by Tobias Kirschstein from his internship at Meta with Javier Romero, Artem Sevastopolsky, and Shunsuke Saito
0:38

📢📢 𝐀𝐯𝐚𝐭𝟑𝐫 📢📢 Avat3r creates high-quality 3D head avatars from just a few input images in a single forward pass with a new dynamic 3DGS reconstruction model. Video: Project: Our core idea is to make Gaussian Reconstruction Models animatable. We find that a simple cross-attention to an expression code sequence is already sufficient to model complex facial expressions. We then incorporate position maps from DUSt3R and feature maps from Sapiens to facilitate the prediction task. While DUSt3R's position maps act as a pixel-aligned initialization for the Gaussians' positions, the Sapiens feature maps help the cross-view transformer to match corresponding image tokens in the 4 input images. One major challenge in creating a 3D head avatar from smartphone images comes from inconsistent facial expressions when the subject could not remain perfectly static during the capture. We eliminate this static requirement by simply showing our model input images with different facial expressions during training. This technique makes our model robust to inconsistent input images later on. Finally, we show that despite the model has been trained with 4 input images, one can even create a 3D head avatar when only a single image is available. To achieve this, we employ a pre-trained 3D GAN to lift the single image to 3D and then render the 4 input images for our model. This allows us to create 3D head avatars from single images and even highly out-of-distribution examples like AI generated faces, paintings or statues. Great work by Tobias Kirschstein from his internship at Meta with Javier Romero, Artem Sevastopolsky, and Shunsuke Saito

Matthias Niessner

74,698 次观看 • 1 年前

From live video to 3D avatar in seconds. This #SIGGRAPH2025 paper from Adobe Research reconstructs a realistic head avatar on the fly, with no pre-cached data, and adapts seamlessly to facial motion for VR, animation, and online communication. 🔗
0:23

From live video to 3D avatar in seconds. This #SIGGRAPH2025 paper from Adobe Research reconstructs a realistic head avatar on the fly, with no pre-cached data, and adapts seamlessly to facial motion for VR, animation, and online communication. 🔗

Adobe Research

29,059 次观看 • 10 个月前

I'm excited to show a new avatar customization and creation tool I've been working on for #Vrchat -- Avatar Workshop! Stay tuned and follow for more!
1:04

I'm excited to show a new avatar customization and creation tool I've been working on for #Vrchat -- Avatar Workshop! Stay tuned and follow for more!

Tatsu

277,975 次观看 • 2 年前

New Puplic Lycanroc Drone Avatar A new public SDL drone avatar has been released. A Lycanroc drone avatar. The avatar has the following options: - Colour can be changed - Hypnovisor with effect Textured and modified by Roc Unit L-049 . We hope you all like this new avatar and have fun with it. The avatar is Adviable on our SDL avatar World. Links to it in the comments. A quest version of the avatar will be released in the coming days.
1:07

New Puplic Lycanroc Drone Avatar A new public SDL drone avatar has been released. A Lycanroc drone avatar. The avatar has the following options: - Colour can be changed - Hypnovisor with effect Textured and modified by Roc Unit L-049 . We hope you all like this new avatar and have fun with it. The avatar is Adviable on our SDL avatar World. Links to it in the comments. A quest version of the avatar will be released in the coming days.

SDL

18,528 次观看 • 6 个月前

We also present another paper at @SIGGRAPH 2023 on neural implicit 3D Morphable Models that can be used to create a dynamic 3D avatar from a single in-the-wild image. (Lead author Connor Lin).
0:39

We also present another paper at @SIGGRAPH 2023 on neural implicit 3D Morphable Models that can be used to create a dynamic 3D avatar from a single in-the-wild image. (Lead author Connor Lin).

Koki Nagano

12,758 次观看 • 3 年前

For the second free update of our 3D customizable vtuber avatar, I added in a new Live2D styled mouth graph with 12 new mouth forms! 🫧 You'll be able to blend between stylized and realistic mouth tracking! Yay!🩷 -^u^-
0:32

For the second free update of our 3D customizable vtuber avatar, I added in a new Live2D styled mouth graph with 12 new mouth forms! 🫧 You'll be able to blend between stylized and realistic mouth tracking! Yay!🩷 -^u^-

Pastell & Palette🤍 3D Customizable Model!

16,987 次观看 • 6 个月前

Personalized 3D Generative Avatars from a Single Portrait Contributions: 1. Generate a personalized 3D avatar from a reference portrait image with controllable facial attributes. 2. Create high-quality synthetic 2D video datasets with diverse attribute editing from a reference portrait image. 3. Use latent space regularization with face morphing supervision for a continuous and smooth latent space, enhancing the generative ability for unseen or interpolated attribute appearances. 4. Employ an efficient fine-tuning technique via Low-Rank Adaptation (LoRA) [26] to integrate any new facial attribute into the avatar model.
0:45

Personalized 3D Generative Avatars from a Single Portrait Contributions: 1. Generate a personalized 3D avatar from a reference portrait image with controllable facial attributes. 2. Create high-quality synthetic 2D video datasets with diverse attribute editing from a reference portrait image. 3. Use latent space regularization with face morphing supervision for a continuous and smooth latent space, enhancing the generative ability for unseen or interpolated attribute appearances. 4. Employ an efficient fine-tuning technique via Low-Rank Adaptation (LoRA) [26] to integrate any new facial attribute into the avatar model.

MrNeRF

20,507 次观看 • 1 年前

Alice in Wonderland was released 16 years ago today. Riding the 3D boom after Avatar, it became one of the first films to fully capitalize on the craze, crossing $1 billion worldwide during the post-Avatar wave of 3D blockbusters.
1:43

Alice in Wonderland was released 16 years ago today. Riding the 3D boom after Avatar, it became one of the first films to fully capitalize on the craze, crossing $1 billion worldwide during the post-Avatar wave of 3D blockbusters.

cinesthetic.

17,033 次观看 • 3 个月前

Axie Avatar Forge Just Leveled Up! • MOAR freedom to setup the angle for your avatar • Equip new accessories with updated animations • Get a higher resolution image and GIF Check out the UPGRADES and show us your new axie avatar now 👇 🔗 :
0:36

Axie Avatar Forge Just Leveled Up! • MOAR freedom to setup the angle for your avatar • Equip new accessories with updated animations • Get a higher resolution image and GIF Check out the UPGRADES and show us your new axie avatar now 👇 🔗 :

Axie Infinity

13,495 次观看 • 11 个月前

🚀Announcing the release of next-gen avatar platform, It’s now a matter of a few seconds to get your realistic, customizable 3D avatar from a 2d photo using Avaturn’s new generative AI. Developer? Integrate into your game or app, in 15 min. for free.
0:33

🚀Announcing the release of next-gen avatar platform, It’s now a matter of a few seconds to get your realistic, customizable 3D avatar from a 2d photo using Avaturn’s new generative AI. Developer? Integrate into your game or app, in 15 min. for free.

Avaturn

17,783 次观看 • 3 年前

Download UE5 AVATAR METAVANCE for Free! Create your personalized AVATAR with real-time CG movie-quality rendering, all powered by UE5. Modify 3D models, outfits, and appearances instantly—no need for Maya modeling or complex rigging. Future updates will support 3D model import/export and image-to-3D face generation, making 3D model creation accessible to everyone—no professional skills required! 🚀
2:20

Download UE5 AVATAR METAVANCE for Free! Create your personalized AVATAR with real-time CG movie-quality rendering, all powered by UE5. Modify 3D models, outfits, and appearances instantly—no need for Maya modeling or complex rigging. Future updates will support 3D model import/export and image-to-3D face generation, making 3D model creation accessible to everyone—no professional skills required! 🚀

CYANPUPPETS

28,742 次观看 • 1 年前

The world's first 3D customizable vtuber avatar will be released in four days! 🩷 Here are some of the ear and horn options (more on the way! <3) Just wanted to clear up some confusion, our vtuber model is not a software/program or a base, it is a single avatar built for Warudo with lots of customization options! -^u^-
1:21

The world's first 3D customizable vtuber avatar will be released in four days! 🩷 Here are some of the ear and horn options (more on the way! <3) Just wanted to clear up some confusion, our vtuber model is not a software/program or a base, it is a single avatar built for Warudo with lots of customization options! -^u^-

Pastell & Palette🤍 3D Customizable Model!

91,051 次观看 • 8 个月前

Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors paper page: present Magic123, a two-stage coarse-to-fine approach for high-quality, textured 3D meshes generation from a single unposed image in the wild using both2D and 3D priors. In the first stage, we optimize a neural radiance field to produce a coarse geometry. In the second stage, we adopt a memory-efficient differentiable mesh representation to yield a high-resolution mesh with a visually appealing texture. In both stages, the 3D content is learned through reference view supervision and novel views guided by a combination of 2D and 3D diffusion priors. We introduce a single trade-off parameter between the 2D and 3D priors to control exploration (more imaginative) and exploitation (more precise) of the generated geometry. Additionally, we employ textual inversion and monocular depth regularization to encourage consistent appearances across views and to prevent degenerate solutions, respectively. Magic123 demonstrates a significant improvement over previous image-to-3D techniques, as validated through extensive experiments on synthetic benchmarks and diverse real-world images.
0:34

Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors paper page: present Magic123, a two-stage coarse-to-fine approach for high-quality, textured 3D meshes generation from a single unposed image in the wild using both2D and 3D priors. In the first stage, we optimize a neural radiance field to produce a coarse geometry. In the second stage, we adopt a memory-efficient differentiable mesh representation to yield a high-resolution mesh with a visually appealing texture. In both stages, the 3D content is learned through reference view supervision and novel views guided by a combination of 2D and 3D diffusion priors. We introduce a single trade-off parameter between the 2D and 3D priors to control exploration (more imaginative) and exploitation (more precise) of the generated geometry. Additionally, we employ textual inversion and monocular depth regularization to encourage consistent appearances across views and to prevent degenerate solutions, respectively. Magic123 demonstrates a significant improvement over previous image-to-3D techniques, as validated through extensive experiments on synthetic benchmarks and diverse real-world images.

AK

305,643 次观看 • 3 年前

Here's all of the face customization sliders our 3D customizable Warudo vtuber avatar has! :D (More will be added soon!)💙 We are pushing the limits of 3D with its customizability! -^u^- 🩷
0:31

Here's all of the face customization sliders our 3D customizable Warudo vtuber avatar has! :D (More will be added soon!)💙 We are pushing the limits of 3D with its customizability! -^u^- 🩷

Pastell & Palette🤍 3D Customizable Model Soon!

14,648 次观看 • 8 个月前

Stop by the #ECCV2024 Google Booth at 4:30pm CEST where Googlers Francis Engelmann and Federico Tombari will demo language-guided 3D search ( and SceneFun3D, a new benchmark dataset for functional 3D scene understanding (
0:27

Stop by the #ECCV2024 Google Booth at 4:30pm CEST where Googlers Francis Engelmann and Federico Tombari will demo language-guided 3D search ( and SceneFun3D, a new benchmark dataset for functional 3D scene understanding (

Google AI

27,556 次观看 • 1 年前

Gaussian Shell Maps are a new neural scene representation that connects fields and 3D Gaussians. This representation unlocks the full potential of 3D Gaussian splatting for generative AI applications, such as 3D avatar generation. 1/2
0:22

Gaussian Shell Maps are a new neural scene representation that connects fields and 3D Gaussians. This representation unlocks the full potential of 3D Gaussian splatting for generative AI applications, such as 3D avatar generation. 1/2

Gordon Wetzstein

52,480 次观看 • 2 年前

#SynthEyes 2025 is out Check out the new features in BorisFX's 3D tracking app, including AI-based roto mask generation, a 3D head mesh for head tracking, and a new Multi-Export manager #matchmoving #VFX #motiongraphics Boris FX
0:23

#SynthEyes 2025 is out Check out the new features in BorisFX's 3D tracking app, including AI-based roto mask generation, a 3D head mesh for head tracking, and a new Multi-Export manager #matchmoving #VFX #motiongraphics Boris FX

CG Channel

12,665 次观看 • 1 年前