๐ข SHeaP: Self-Supervised Head Predictor Learned via 2D Gaussians ๐ข
Matthias Niessner
Visit on X@MattNiessner51K followers
28,629 views
๐ข SHeaP: Self-Supervised Head Predictor Learned via 2D Gaussians ๐ข
Given a single input image, we predict accurate 3D head geometry, pose, and expression.
Previous works (e.g. DECA, EMOCA) use differentiable mesh rasterization to learn a self-supervised head geometry predictor via a photometric reconstruction loss. We borrow these ideas, but our key insight is to replace the mesh rendering with 2D Gaussian Splatting. This leads to much higher accuracy of the underlying predicted geometry and thus more gradient signal during training.
๐
๐ฅ
Great work by Liam Schoneveld Davide Davoli Jiapeng Tang
4 Comments
Comments from the original post on X (Twitter)

@taubnerfelix
Always happy to see new work face trackers!
6 likes

@RainmakerTrade
Join me as I put several Machine Learning models head-to-head to see which one can beat the market and deliver strong returns. In this free Substack post I share several models that deliver better returns with much lower drawdown compared to Buy-and-Hold approach.
73 likes

@Michael_J_Black
Nice. Iโve been wanting to replace the old photometric loss with splatting. Results look great.
5 likes

@karlmehta
A fascinating step forward in precision and training efficiency.
1 like
