Skip navigation
๐Ÿ“ข SHeaP: Self-Supervised Head Predictor Learned via 2D Gaussians ๐Ÿ“ข
0:00 / 0:48

๐Ÿ“ข SHeaP: Self-Supervised Head Predictor Learned via 2D Gaussians ๐Ÿ“ข

Matthias Niessner

@MattNiessner51K followers

Visit on X
334 likesDownloadOriginal post

28,629 views

๐Ÿ“ข SHeaP: Self-Supervised Head Predictor Learned via 2D Gaussians ๐Ÿ“ข Given a single input image, we predict accurate 3D head geometry, pose, and expression. Previous works (e.g. DECA, EMOCA) use differentiable mesh rasterization to learn a self-supervised head geometry predictor via a photometric reconstruction loss. We borrow these ideas, but our key insight is to replace the mesh rendering with 2D Gaussian Splatting. This leads to much higher accuracy of the underlying predicted geometry and thus more gradient signal during training. ๐ŸŒ ๐ŸŽฅ Great work by Liam Schoneveld Davide Davoli Jiapeng Tang

4 Comments

Comments from the original post on X (Twitter)

  • Felix Taubner's profile picture

    @taubnerfelix

    Always happy to see new work face trackers!

    6 likes

  • Rainmaker's profile picture

    @RainmakerTrade

    Join me as I put several Machine Learning models head-to-head to see which one can beat the market and deliver strong returns. In this free Substack post I share several models that deliver better returns with much lower drawdown compared to Buy-and-Hold approach.

    73 likes

  • Michael Black's profile picture

    @Michael_J_Black

    Nice. Iโ€™ve been wanting to replace the old photometric loss with splatting. Results look great.

    5 likes

  • Karl Mehta's profile picture

    @karlmehta

    A fascinating step forward in precision and training efficiency.

    1 like