Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

🚀Exicited to present our new work "InvertAvatar: Incremental GAN Inversion for Generalized Head Avatars". Given single or more source images, our method rapidly reconstructs photorealistic 3D facial avatars under 1s. Paper: Project page comes soon!

JingxiangSun42's profile picture

Jingxiang Sun

1,169 subscribers

16,741 Aufrufe • vor 2 Jahren •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Jingxiang Sun
Jingxiang Sunvor 2 Jahren

Leveraging a 3D GAN prior, we introduce an effective recurrent inversion pipeline that aggregates temporal data and inherently supports flexible image inputs.

Profilbild von 田中義弘 | taziku CEO / AI × Creative
田中義弘 | taziku CEO / AI × Creativevor 2 Jahren

This is a promising technology. In Japan, there is a VTuber culture, so it will be greatly utilized!

Profilbild von Martin Shkreli (e/acc)
Martin Shkreli (e/acc)vor 2 Jahren

Nice job

Profilbild von Jingxiang Sun
Jingxiang Sunvor 2 Jahren

Thanks😄

Profilbild von Tonter_IAS
Tonter_IASvor 2 Jahren

Wow! 👏👏👏

Profilbild von BLENDER SUSHI 🫶 X - 24/7 Blenderian
BLENDER SUSHI 🫶 X - 24/7 Blenderianvor 2 Jahren

Pin the shoulders and it would be better.

Profilbild von Jingxiang Sun
Jingxiang Sunvor 2 Jahren

yeah that‘s exactly what we want to do next🤝

Profilbild von 210 Digital Marketing
210 Digital Marketingvor 2 Jahren

Nice 👍

Profilbild von wwwwg
wwwwgvor 2 Jahren

@Memdotai mem it

Profilbild von Mem
Memvor 2 Jahren

@JingxiangSun42 Saved! Here's the compiled thread: 🪄 AI-generated summary: "We are excited to present our new work, InvertAvatar, which rapidly reconstructs photorealistic 3D facial avatars from single or multiple source images in under 1 second....

Ähnliche Videos

Our new work PSHuman reconstructs a detailed 3D human mesh from a single-view image in ~1min. Codes have already been released! Welcome to try it! Project page: Codes: Paper:
0:39

Our new work PSHuman reconstructs a detailed 3D human mesh from a single-view image in ~1min. Codes have already been released! Welcome to try it! Project page: Codes: Paper:

Yuan Liu

53,252 Aufrufe • vor 1 Jahr

Drivable 3D Gaussian Avatars paper page: present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats. Current photorealistic drivable avatars require either accurate 3D registrations during training, dense input images during testing, or both. The ones based on neural radiance fields also tend to be prohibitively slow for telepresence applications. This work uses the recently presented 3D Gaussian Splatting (3DGS) technique to render realistic humans at real-time framerates, using dense calibrated multi-view videos as input. To deform those primitives, we depart from the commonly used point deformation method of linear blend skinning (LBS) and use a classic volumetric deformation method: cage deformations. Given their smaller size, we drive these deformations with joint angles and keypoints, which are more suitable for communication applications. Our experiments on nine subjects with varied body shapes, clothes, and motions obtain higher-quality results than state-of-the-art methods when using the same training and test data.
6:20

Drivable 3D Gaussian Avatars paper page: present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats. Current photorealistic drivable avatars require either accurate 3D registrations during training, dense input images during testing, or both. The ones based on neural radiance fields also tend to be prohibitively slow for telepresence applications. This work uses the recently presented 3D Gaussian Splatting (3DGS) technique to render realistic humans at real-time framerates, using dense calibrated multi-view videos as input. To deform those primitives, we depart from the commonly used point deformation method of linear blend skinning (LBS) and use a classic volumetric deformation method: cage deformations. Given their smaller size, we drive these deformations with joint angles and keypoints, which are more suitable for communication applications. Our experiments on nine subjects with varied body shapes, clothes, and motions obtain higher-quality results than state-of-the-art methods when using the same training and test data.

AK

327,373 Aufrufe • vor 2 Jahren

📢📢 𝐀𝐯𝐚𝐭𝟑𝐫 📢📢 Avat3r creates high-quality 3D head avatars from just a few input images in a single forward pass with a new dynamic 3DGS reconstruction model. Video: Project: Our core idea is to make Gaussian Reconstruction Models animatable. We find that a simple cross-attention to an expression code sequence is already sufficient to model complex facial expressions. We then incorporate position maps from DUSt3R and feature maps from Sapiens to facilitate the prediction task. While DUSt3R's position maps act as a pixel-aligned initialization for the Gaussians' positions, the Sapiens feature maps help the cross-view transformer to match corresponding image tokens in the 4 input images. One major challenge in creating a 3D head avatar from smartphone images comes from inconsistent facial expressions when the subject could not remain perfectly static during the capture. We eliminate this static requirement by simply showing our model input images with different facial expressions during training. This technique makes our model robust to inconsistent input images later on. Finally, we show that despite the model has been trained with 4 input images, one can even create a 3D head avatar when only a single image is available. To achieve this, we employ a pre-trained 3D GAN to lift the single image to 3D and then render the 4 input images for our model. This allows us to create 3D head avatars from single images and even highly out-of-distribution examples like AI generated faces, paintings or statues. Great work by Tobias Kirschstein from his internship at Meta with Javier Romero, Artem Sevastopolsky, and Shunsuke Saito
0:38

📢📢 𝐀𝐯𝐚𝐭𝟑𝐫 📢📢 Avat3r creates high-quality 3D head avatars from just a few input images in a single forward pass with a new dynamic 3DGS reconstruction model. Video: Project: Our core idea is to make Gaussian Reconstruction Models animatable. We find that a simple cross-attention to an expression code sequence is already sufficient to model complex facial expressions. We then incorporate position maps from DUSt3R and feature maps from Sapiens to facilitate the prediction task. While DUSt3R's position maps act as a pixel-aligned initialization for the Gaussians' positions, the Sapiens feature maps help the cross-view transformer to match corresponding image tokens in the 4 input images. One major challenge in creating a 3D head avatar from smartphone images comes from inconsistent facial expressions when the subject could not remain perfectly static during the capture. We eliminate this static requirement by simply showing our model input images with different facial expressions during training. This technique makes our model robust to inconsistent input images later on. Finally, we show that despite the model has been trained with 4 input images, one can even create a 3D head avatar when only a single image is available. To achieve this, we employ a pre-trained 3D GAN to lift the single image to 3D and then render the 4 input images for our model. This allows us to create 3D head avatars from single images and even highly out-of-distribution examples like AI generated faces, paintings or statues. Great work by Tobias Kirschstein from his internship at Meta with Javier Romero, Artem Sevastopolsky, and Shunsuke Saito

Matthias Niessner

74,863 Aufrufe • vor 1 Jahr

I'm excited to share our new work, VistaDream, which generates a 3D Gaussian field from a single-view image. The codes have already been released. Project page: (with interactive demos) Code: Paper:
0:27

I'm excited to share our new work, VistaDream, which generates a 3D Gaussian field from a single-view image. The codes have already been released. Project page: (with interactive demos) Code: Paper:

Yuan Liu

79,610 Aufrufe • vor 1 Jahr

📢📢 𝐆𝐚𝐮𝐬𝐬𝐢𝐚𝐧𝐒𝐩𝐞𝐞𝐜𝐡: Audio-Driven Gaussian Avatars 📢📢 We synthesize photorealistic and 3D-consistent talking human head avatars driven directly from spoken audio. More specifically, we introduce an efficient 3DGS-based representation, combined with an audio-conditioned transformer, to generate realistic and high-quality animations. Our method can easily generalize to a variety of in the wild scenarios like songs, foreign languages etc. We further introduce a new multiview dataset of native English speakers with overall recording time of ∼3.5 hours. Project: Video: Great work by Shivangi, Artem Sevastopolsky, Tobias Kirschstein, Justus Thies, Angela Dai
0:33

📢📢 𝐆𝐚𝐮𝐬𝐬𝐢𝐚𝐧𝐒𝐩𝐞𝐞𝐜𝐡: Audio-Driven Gaussian Avatars 📢📢 We synthesize photorealistic and 3D-consistent talking human head avatars driven directly from spoken audio. More specifically, we introduce an efficient 3DGS-based representation, combined with an audio-conditioned transformer, to generate realistic and high-quality animations. Our method can easily generalize to a variety of in the wild scenarios like songs, foreign languages etc. We further introduce a new multiview dataset of native English speakers with overall recording time of ∼3.5 hours. Project: Video: Great work by Shivangi, Artem Sevastopolsky, Tobias Kirschstein, Justus Thies, Angela Dai

Matthias Niessner

22,703 Aufrufe • vor 1 Jahr

🚀 Excited to announce the open-sourcing of Hallo3, a new avatar generative model! 🎨✨ Hallo3 is capable of creating highly realistic avatars with dynamic backgrounds, expressive foregrounds, and various head orientations. Perfect for bringing your ideas to life! 🛠️ Try it out and let us know your feedback or suggestions. Github 👉 Project page:
1:02

🚀 Excited to announce the open-sourcing of Hallo3, a new avatar generative model! 🎨✨ Hallo3 is capable of creating highly realistic avatars with dynamic backgrounds, expressive foregrounds, and various head orientations. Perfect for bringing your ideas to life! 🛠️ Try it out and let us know your feedback or suggestions. Github 👉 Project page:

Siyu ZHU

35,424 Aufrufe • vor 1 Jahr

📢📢 𝐏𝐞𝐫𝐜𝐇𝐞𝐚𝐝: 𝐏𝐞𝐫𝐜𝐞𝐩𝐭𝐮𝐚𝐥 𝐇𝐞𝐚𝐝 𝐌𝐨𝐝𝐞𝐥 𝐟𝐨𝐫 𝐒𝐢𝐧𝐠𝐥𝐞-𝐈𝐦𝐚𝐠𝐞 𝟑𝐃 𝐇𝐞𝐚𝐝 𝐑𝐞𝐜𝐨𝐧𝐬𝐭𝐫𝐮𝐜𝐭𝐢𝐨𝐧 & 𝐄𝐝𝐢𝐭𝐢𝐧𝐠📢📢 PercHead reconstructs realistic 3D heads from a single image and enables disentangled 3D editing via geometric controls and style inputs from images or text. At its core is a generalized 3D head decoder trained with perceptual supervision from DINOv2 and SAM 2.1. We find that our new perceptual loss formulation improves reconstruction fidelity compared to commonly-used methods such as LPIPS. Our trained reconstruction model is able to generate 3D-consistent heads from a single input image. Even with challenging side-view inputs, the model robustly infers missing regions for a coherent, high-fidelity output. In addition, our architecture seamlessly adapts to downstream tasks: by swapping the encoder, we can transform the model into a disentangled 3D editing pipeline. In this scenario, we can control geometry through - potentially hand-drawn - segmentation maps, and condition style via image or text prompt. We also provide an interactive GUI to enable the exploration of our editing pipeline. 🌍 📽️ Great work by Antonio Oroz and Tobias Kirschstein
0:30

📢📢 𝐏𝐞𝐫𝐜𝐇𝐞𝐚𝐝: 𝐏𝐞𝐫𝐜𝐞𝐩𝐭𝐮𝐚𝐥 𝐇𝐞𝐚𝐝 𝐌𝐨𝐝𝐞𝐥 𝐟𝐨𝐫 𝐒𝐢𝐧𝐠𝐥𝐞-𝐈𝐦𝐚𝐠𝐞 𝟑𝐃 𝐇𝐞𝐚𝐝 𝐑𝐞𝐜𝐨𝐧𝐬𝐭𝐫𝐮𝐜𝐭𝐢𝐨𝐧 & 𝐄𝐝𝐢𝐭𝐢𝐧𝐠📢📢 PercHead reconstructs realistic 3D heads from a single image and enables disentangled 3D editing via geometric controls and style inputs from images or text. At its core is a generalized 3D head decoder trained with perceptual supervision from DINOv2 and SAM 2.1. We find that our new perceptual loss formulation improves reconstruction fidelity compared to commonly-used methods such as LPIPS. Our trained reconstruction model is able to generate 3D-consistent heads from a single input image. Even with challenging side-view inputs, the model robustly infers missing regions for a coherent, high-fidelity output. In addition, our architecture seamlessly adapts to downstream tasks: by swapping the encoder, we can transform the model into a disentangled 3D editing pipeline. In this scenario, we can control geometry through - potentially hand-drawn - segmentation maps, and condition style via image or text prompt. We also provide an interactive GUI to enable the exploration of our editing pipeline. 🌍 📽️ Great work by Antonio Oroz and Tobias Kirschstein

Matthias Niessner

18,903 Aufrufe • vor 11 Monaten

2D is what the camera sees.👀 3D is what the world is. 🌍 Today, we’re proudly open-sourcing QuerySplat — an open-source feed-forward 3D model that turns a few unposed images into a navigable 3D scene in seconds (run on 4090🪶). Your input: a few unposed images Our output: a sharp, navigable 3D scene — generated in seconds! 🚀 🧠 organize scenes with 3D queries, not pixel-bound Gaussians 📷 no camera poses or manual calibration required ⚡ reconstruct new scenes in a single forward pass ✨ preserve sharp geometry and high-frequency appearance 🏆 achieve state-of-the-art results on DL3DV-Evaluation 🔓 code and model weights are now open-source From pixels to scenes, we are making 3D generation scalable. Try our free iOS app — it’s a lot of fun! 😆 🔗 Try it on the Web: 🔗 Paper: 🔗 Code & weights: 🔗 Project page: 🔗 #AIGC #3DGS #AI #Computervision #3D #artist #3dartist #3d
1:13

2D is what the camera sees.👀 3D is what the world is. 🌍 Today, we’re proudly open-sourcing QuerySplat — an open-source feed-forward 3D model that turns a few unposed images into a navigable 3D scene in seconds (run on 4090🪶). Your input: a few unposed images Our output: a sharp, navigable 3D scene — generated in seconds! 🚀 🧠 organize scenes with 3D queries, not pixel-bound Gaussians 📷 no camera poses or manual calibration required ⚡ reconstruct new scenes in a single forward pass ✨ preserve sharp geometry and high-frequency appearance 🏆 achieve state-of-the-art results on DL3DV-Evaluation 🔓 code and model weights are now open-source From pixels to scenes, we are making 3D generation scalable. Try our free iOS app — it’s a lot of fun! 😆 🔗 Try it on the Web: 🔗 Paper: 🔗 Code & weights: 🔗 Project page: 🔗 #AIGC #3DGS #AI #Computervision #3D #artist #3dartist #3d

InSpatio｜影溯

29,806 Aufrufe • vor 2 Monaten

Celebrating the first reveal of YogaPetz NFTs! A human type Petz fully 3D. Meet this AI-driven pink beauty, featuring AI Motion Capture & Body Tracking, fully animated and powered by WELLAIOS and marking the beginning of a new AI Identity revolution. A vast collection of EGGs 🥚 will be revealed soon—get ready for a wave of unique, intelligent avatars! 🌐 Your AI identity is coming to life—AI agent-driven, on-chain, and fully customizable. This is just a taste of how our platform connects AI agents with avatars, redefining digital identity. ✨ Beyond just avatars—unlock limitless possibilities: 🎥 Live streaming & reels with AI-powered motion capture 🕶 VR-ready 3D avatars for immersive interactions 🌍 Intelligent AI agents expanding virtual engagement 🚀 WELLAIOS is leading the 3D + AI evolution. Your AI Identity starts here. Let's cook! #WELLAIOS #Web3 #YogaPetz #AI #AIAgent #WELL3 #NJZ #NewJeans #3DHuman
0:22

Celebrating the first reveal of YogaPetz NFTs! A human type Petz fully 3D. Meet this AI-driven pink beauty, featuring AI Motion Capture & Body Tracking, fully animated and powered by WELLAIOS and marking the beginning of a new AI Identity revolution. A vast collection of EGGs 🥚 will be revealed soon—get ready for a wave of unique, intelligent avatars! 🌐 Your AI identity is coming to life—AI agent-driven, on-chain, and fully customizable. This is just a taste of how our platform connects AI agents with avatars, redefining digital identity. ✨ Beyond just avatars—unlock limitless possibilities: 🎥 Live streaming & reels with AI-powered motion capture 🕶 VR-ready 3D avatars for immersive interactions 🌍 Intelligent AI agents expanding virtual engagement 🚀 WELLAIOS is leading the 3D + AI evolution. Your AI Identity starts here. Let's cook! #WELLAIOS #Web3 #YogaPetz #AI #AIAgent #WELL3 #NJZ #NewJeans #3DHuman

WELL3

14,726 Aufrufe • vor 1 Jahr

🚀Announcing NeRSemble 3D Head Avatar Benchmark v2 Version 2 of the NeRSemble 3D Head Avatar Benchmark systematically evaluates several aspects of 3D head avatar creation. Our goal is to drive progress toward more realistic, robust, and generalizable avatar methods. 🔬Benchmark Tasks The NeRSemble Benchmark v2 features three core challenges: - Dynamic Novel View Synthesis - Monocular FLAME-driven Avatar Creation (updated) - Single-view 3D Face Reconstruction (new) 👉Explore the online leaderboard and submission system: 🆕What's new? 1. New Task: Single-view 3D Face Reconstruction Given a single portrait image, reconstruct an accurate 3D mesh either showing the input expression or a fully neutral one. Unlike prior benchmarks, the NeRSemble benchmark emphasizes diverse and challenging facial expressions, better reflecting real scenarios. For technical details, see the Pixel3DMM paper. 2. Updated task: Monocular FLAME-driven Avatar Creation We have improved the FLAME tracking that is used for both avatar creation from the monocular videos and avatar driving on the hidden test sequences. The updated benchmark task has: - more stable torso tracking - more expressive lip closures during speech - Improved mouth tracking for challenging facial expressions We hope that these improvements to the benchmark help drive the field forward. 🏆 CVPR 2026 Workshop & Prizes The NeRSemble benchmark will be featured at the CVPR 2026 Workshop on Photo-realistic 3D Head Avatars. Participants in the new and updated tasks have the opportunity to win: - 🎁RTX 5080 GPUs (sponsored by NVIDIA) - 🎤15-minute oral presentation at the workshop ⏰ Submission Deadline - May 26, 2026 Reach out to the amazing Tobias Kirschstein and Simon Giebenhain for more details :)
0:27

🚀Announcing NeRSemble 3D Head Avatar Benchmark v2 Version 2 of the NeRSemble 3D Head Avatar Benchmark systematically evaluates several aspects of 3D head avatar creation. Our goal is to drive progress toward more realistic, robust, and generalizable avatar methods. 🔬Benchmark Tasks The NeRSemble Benchmark v2 features three core challenges: - Dynamic Novel View Synthesis - Monocular FLAME-driven Avatar Creation (updated) - Single-view 3D Face Reconstruction (new) 👉Explore the online leaderboard and submission system: 🆕What's new? 1. New Task: Single-view 3D Face Reconstruction Given a single portrait image, reconstruct an accurate 3D mesh either showing the input expression or a fully neutral one. Unlike prior benchmarks, the NeRSemble benchmark emphasizes diverse and challenging facial expressions, better reflecting real scenarios. For technical details, see the Pixel3DMM paper. 2. Updated task: Monocular FLAME-driven Avatar Creation We have improved the FLAME tracking that is used for both avatar creation from the monocular videos and avatar driving on the hidden test sequences. The updated benchmark task has: - more stable torso tracking - more expressive lip closures during speech - Improved mouth tracking for challenging facial expressions We hope that these improvements to the benchmark help drive the field forward. 🏆 CVPR 2026 Workshop & Prizes The NeRSemble benchmark will be featured at the CVPR 2026 Workshop on Photo-realistic 3D Head Avatars. Participants in the new and updated tasks have the opportunity to win: - 🎁RTX 5080 GPUs (sponsored by NVIDIA) - 🎤15-minute oral presentation at the workshop ⏰ Submission Deadline - May 26, 2026 Reach out to the amazing Tobias Kirschstein and Simon Giebenhain for more details :)

Matthias Niessner

30,098 Aufrufe • vor 6 Monaten

Nvidia announces GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning paper page: Gaussian splatting has emerged as a powerful 3D representation that harnesses the advantages of both explicit (mesh) and implicit (NeRF) 3D representations. In this paper, we seek to leverage Gaussian splatting to generate realistic animatable avatars from textual descriptions, addressing the limitations (e.g., flexibility and efficiency) imposed by mesh or NeRF-based representations. However, a naive application of Gaussian splatting cannot generate high-quality animatable avatars and suffers from learning instability; it also cannot capture fine avatar geometries and often leads to degenerate body parts. To tackle these problems, we first propose a primitive-based 3D Gaussian representation where Gaussians are defined inside pose-driven primitives to facilitate animation. Second, to stabilize and amortize the learning of millions of Gaussians, we propose to use neural implicit fields to predict the Gaussian attributes (e.g., colors). Finally, to capture fine avatar geometries and extract detailed meshes, we propose a novel SDF-based implicit mesh learning approach for 3D Gaussians that regularizes the underlying geometries and extracts highly detailed textured meshes. Our proposed method, GAvatar, enables the large-scale generation of diverse animatable avatars using only text prompts. GAvatar significantly surpasses existing methods in terms of both appearance and geometry quality, and achieves extremely fast rendering (100 fps) at 1K resolution.
5:36

Nvidia announces GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning paper page: Gaussian splatting has emerged as a powerful 3D representation that harnesses the advantages of both explicit (mesh) and implicit (NeRF) 3D representations. In this paper, we seek to leverage Gaussian splatting to generate realistic animatable avatars from textual descriptions, addressing the limitations (e.g., flexibility and efficiency) imposed by mesh or NeRF-based representations. However, a naive application of Gaussian splatting cannot generate high-quality animatable avatars and suffers from learning instability; it also cannot capture fine avatar geometries and often leads to degenerate body parts. To tackle these problems, we first propose a primitive-based 3D Gaussian representation where Gaussians are defined inside pose-driven primitives to facilitate animation. Second, to stabilize and amortize the learning of millions of Gaussians, we propose to use neural implicit fields to predict the Gaussian attributes (e.g., colors). Finally, to capture fine avatar geometries and extract detailed meshes, we propose a novel SDF-based implicit mesh learning approach for 3D Gaussians that regularizes the underlying geometries and extracts highly detailed textured meshes. Our proposed method, GAvatar, enables the large-scale generation of diverse animatable avatars using only text prompts. GAvatar significantly surpasses existing methods in terms of both appearance and geometry quality, and achieves extremely fast rendering (100 fps) at 1K resolution.

AK

141,058 Aufrufe • vor 2 Jahren

We’ve just open-sourced JOSH, our method for 4D human–scene reconstruction in the wild. Huge credit to the first author, my PhD student Zhizheng, who will present this work at ICLR in Brazil next month 🇧🇷 🌐 Project page: 💻 Code: A trivia is that this work was finished more than one year ago, and has been rejected twice...Be resilient!
0:25

We’ve just open-sourced JOSH, our method for 4D human–scene reconstruction in the wild. Huge credit to the first author, my PhD student Zhizheng, who will present this work at ICLR in Brazil next month 🇧🇷 🌐 Project page: 💻 Code: A trivia is that this work was finished more than one year ago, and has been rejected twice...Be resilient!

Bolei Zhou

25,616 Aufrufe • vor 7 Monaten

Super excited for #ECCV'26 in Malmö! Our group will present a series of cutting-edge works, including two orals and a spotlight, around generative AI that cover 3D scene generation, photo-realistic avatars, agentic modeling, and much more! I'll also give several workshop talks about our recent research, including a few exciting updates what we're cooking at SpAItial AI :) 1. GaussianGPT: Towards Autoregressive 3D Gaussian Scene Generation *(oral)* 2. TriFlow: Generating Artist-Like 3D Mesh Topology via Nearest-Vertex Vector Fields *(oral)* 3. Face Anything: 4D Face Reconstruction from Any Image Sequence *(spotlight)* 4. Pix2NPHM: Learning to Regress NPHM Reconstructions From a Single Image 5. Seen2Scene: Completing Realistic 3D Scenes with Visibility-Guided Flow 6. DreamEdit3D: Personalization of Multi-View Diffusion Models for 3D Editing 7. WorldAgents: Can Foundation Image Models be Agents for 3D World Models? 8. World Reconstruction From Inconsistent Views 9. Head Avatars with Dynamic Explicit Hair Great work by Nicolas von Lützow Haoxuan Li Ziya Erkoç Umut Kocasarı Vanessa Sklyarova Lukas Höllein Quan Meng Jinxin Ai Simon Giebenhain and all amazing collaborators!
0:27

Super excited for #ECCV'26 in Malmö! Our group will present a series of cutting-edge works, including two orals and a spotlight, around generative AI that cover 3D scene generation, photo-realistic avatars, agentic modeling, and much more! I'll also give several workshop talks about our recent research, including a few exciting updates what we're cooking at SpAItial AI :) 1. GaussianGPT: Towards Autoregressive 3D Gaussian Scene Generation *(oral)* 2. TriFlow: Generating Artist-Like 3D Mesh Topology via Nearest-Vertex Vector Fields *(oral)* 3. Face Anything: 4D Face Reconstruction from Any Image Sequence *(spotlight)* 4. Pix2NPHM: Learning to Regress NPHM Reconstructions From a Single Image 5. Seen2Scene: Completing Realistic 3D Scenes with Visibility-Guided Flow 6. DreamEdit3D: Personalization of Multi-View Diffusion Models for 3D Editing 7. WorldAgents: Can Foundation Image Models be Agents for 3D World Models? 8. World Reconstruction From Inconsistent Views 9. Head Avatars with Dynamic Explicit Hair Great work by Nicolas von Lützow Haoxuan Li Ziya Erkoç Umut Kocasarı Vanessa Sklyarova Lukas Höllein Quan Meng Jinxin Ai Simon Giebenhain and all amazing collaborators!

Matthias Niessner

15,185 Aufrufe • vor 1 Monat

📢📢📢 Excited to share our new work *Autonomous Character-Scene Interaction Synthesis from Text Instruction* (Siggraph Asia 24). It presents a unified model for flexible scene-conditioned motion generation given text, scene, trajectory conditions. The results with smooth interaction look very impressive! 📰Paper: Project: Code and data will be released soon.
6:34

📢📢📢 Excited to share our new work *Autonomous Character-Scene Interaction Synthesis from Text Instruction* (Siggraph Asia 24). It presents a unified model for flexible scene-conditioned motion generation given text, scene, trajectory conditions. The results with smooth interaction look very impressive! 📰Paper: Project: Code and data will be released soon.

Siyuan Huang

11,451 Aufrufe • vor 2 Jahren

Google announces PALP Prompt Aligned Personalization of Text-to-Image Models paper page: Content creators often aim to create personalized images using personal subjects that go beyond the capabilities of conventional text-to-image models. Additionally, they may want the resulting image to encompass a specific location, style, ambiance, and more. Existing personalization methods may compromise personalization ability or the alignment to complex textual prompts. This trade-off can impede the fulfillment of user prompts and subject fidelity. We propose a new approach focusing on personalization methods for a single prompt to address this issue. We term our approach prompt-aligned personalization. While this may seem restrictive, our method excels in improving text alignment, enabling the creation of images with complex and intricate prompts, which may pose a challenge for current techniques. In particular, our method keeps the personalized model aligned with a target prompt using an additional score distillation sampling term. We demonstrate the versatility of our method in multi- and single-shot settings and further show that it can compose multiple subjects or use inspiration from reference images, such as artworks. We compare our approach quantitatively and qualitatively with existing baselines and state-of-the-art techniques.
1:16

Google announces PALP Prompt Aligned Personalization of Text-to-Image Models paper page: Content creators often aim to create personalized images using personal subjects that go beyond the capabilities of conventional text-to-image models. Additionally, they may want the resulting image to encompass a specific location, style, ambiance, and more. Existing personalization methods may compromise personalization ability or the alignment to complex textual prompts. This trade-off can impede the fulfillment of user prompts and subject fidelity. We propose a new approach focusing on personalization methods for a single prompt to address this issue. We term our approach prompt-aligned personalization. While this may seem restrictive, our method excels in improving text alignment, enabling the creation of images with complex and intricate prompts, which may pose a challenge for current techniques. In particular, our method keeps the personalized model aligned with a target prompt using an additional score distillation sampling term. We demonstrate the versatility of our method in multi- and single-shot settings and further show that it can compose multiple subjects or use inspiration from reference images, such as artworks. We compare our approach quantitatively and qualitatively with existing baselines and state-of-the-art techniques.

AK

90,212 Aufrufe • vor 2 Jahren

On deck in our #CVPR2025 series: ChatGarment 👚✨ By Siyuan Bian, Chenghao Xu, Yuliang Xiu, Artur Grigorev, Zhen Liu, Cewu Lu, Michael J. Black and Yao Feng. Feed it an image, video or text, and watch a tailored 3D outfit appear. The video walks through: 1️⃣ Image-based sewing pattern estimation 2️⃣ Text-to-garment generation 3️⃣ Text-based editing Why it matters • Designers preview new looks before cutting fabric • Game and film studios drop physics-ready clothes onto avatars in minutes • Researchers study cloth–body dynamics without manual labeling 🎯 Crafting catwalk looks or decking out game avatars? Just type a prompt and ChatGarment whips up, shrinks, or totally restyles the outfit—no scissors required! 👋 Swing by our #CVPR2026 booth 1333 to meet some of the minds behind ChatGarment, and stay tuned for more CVPR paper videos! 📄 Paper link in the thread/comment. #GarmentTech #3DFashion #DigitalHuman #ComputerVision #AI #MachineLearning
0:48

On deck in our #CVPR2025 series: ChatGarment 👚✨ By Siyuan Bian, Chenghao Xu, Yuliang Xiu, Artur Grigorev, Zhen Liu, Cewu Lu, Michael J. Black and Yao Feng. Feed it an image, video or text, and watch a tailored 3D outfit appear. The video walks through: 1️⃣ Image-based sewing pattern estimation 2️⃣ Text-to-garment generation 3️⃣ Text-based editing Why it matters • Designers preview new looks before cutting fabric • Game and film studios drop physics-ready clothes onto avatars in minutes • Researchers study cloth–body dynamics without manual labeling 🎯 Crafting catwalk looks or decking out game avatars? Just type a prompt and ChatGarment whips up, shrinks, or totally restyles the outfit—no scissors required! 👋 Swing by our #CVPR2026 booth 1333 to meet some of the minds behind ChatGarment, and stay tuned for more CVPR paper videos! 📄 Paper link in the thread/comment. #GarmentTech #3DFashion #DigitalHuman #ComputerVision #AI #MachineLearning

Meshcapade

17,619 Aufrufe • vor 1 Jahr

Today, we are open-sourcing Hunyuan World 1.1 (WorldMirror), a universal feed-forward 3D reconstruction model. 🚀🚀🚀 While our previously released Hunyuan World 1.0 (open-sourced, lite version deployable on consumer GPUs) focused on generating 3D worlds from text or single-view images, Hunyuan World 1.1 significantly expands the input scope by unlocking video-to-3D and multi-view-to-3D world creation. Highlights: 🔹Any Input, Maximized Flexibility and Fidelity: Flexibly integrates diverse geometric priors (camera poses, intrinsics, depth maps) to resolve structural ambiguities and ensure geometrically consistent 3D outputs. 🔹Any Output, SOTA Results：This elegant architecture simultaneously generates multiple 3D representations: dense point clouds, multi-view depth maps, camera parameters, surface normals, and 3D Gaussian Splattings. 🔹Single-GPU & Fast Inference: As an all-in-one, feed-forward model, Hunyuan World 1.1 runs on a single GPU and delivers all 3D attributes in a single forward pass, within seconds. 🌐Project Page: 🔗Github： 🤗Hugging Face： ✨Demo: 📄Technical Report:
1:20

Today, we are open-sourcing Hunyuan World 1.1 (WorldMirror), a universal feed-forward 3D reconstruction model. 🚀🚀🚀 While our previously released Hunyuan World 1.0 (open-sourced, lite version deployable on consumer GPUs) focused on generating 3D worlds from text or single-view images, Hunyuan World 1.1 significantly expands the input scope by unlocking video-to-3D and multi-view-to-3D world creation. Highlights: 🔹Any Input, Maximized Flexibility and Fidelity: Flexibly integrates diverse geometric priors (camera poses, intrinsics, depth maps) to resolve structural ambiguities and ensure geometrically consistent 3D outputs. 🔹Any Output, SOTA Results：This elegant architecture simultaneously generates multiple 3D representations: dense point clouds, multi-view depth maps, camera parameters, surface normals, and 3D Gaussian Splattings. 🔹Single-GPU & Fast Inference: As an all-in-one, feed-forward model, Hunyuan World 1.1 runs on a single GPU and delivers all 3D attributes in a single forward pass, within seconds. 🌐Project Page: 🔗Github： 🤗Hugging Face： ✨Demo: 📄Technical Report:

Tencent Hy

168,848 Aufrufe • vor 11 Monaten

Introducing MLX-Audio Studio 🚀 An open-source UI for audio gen. This new UI will allow you to easily generate and transcribe audio locally using MLX-Audio, Transformers or any other backend you prefer (i.e. OpenAI). We will be adding more tasks soon, stay tuned! Get started on our GH:
1:01

Introducing MLX-Audio Studio 🚀 An open-source UI for audio gen. This new UI will allow you to easily generate and transcribe audio locally using MLX-Audio, Transformers or any other backend you prefer (i.e. OpenAI). We will be adding more tasks soon, stay tuned! Get started on our GH:

Prince Canuma

51,071 Aufrufe • vor 11 Monaten