playing with new backbones this weekend.. impressive how far DINO3+mesh queries can get you.

This is nice. Been looking to pair something like this with lip reading models. For sports purposes...

whoa I've never seen this before. any recs for further reading?

haha of course @andrew_n_carr beats me to it :) Yes a good place to start is metro

dino3 with mesh queries is sick. are you doing the spatial lookups on raw point clouds or running it through a feature bank first? curious what your pipeline looks like

the latency looks crazy low, one of the many benefit of JEPA like architectures

It looks so good. Now I know what to play with later 😃

very interesting. how well does it handle off centre camera positions, and how well does it handle eye blinks ?

The reason I'm doing this is for blinks and off center. positions. For this, I actually have several different spurs, one is pretrtained specifically for neutral identity across all head rotations. The idea is that the model learns the canonical version of a given person. which then acts as a sort of self normalization. It makes mesh prediction more stable and accurate when it comes to blinks for a specific type of facial structure and expression.

I was able to animate a scan mesh

How does this compare to apple’s ARKit mesh?

The version of ARKit with the TrueDepth sensor on iPhone 11 and 12 will never be beat.

can you share more details?

this is the kind of weekend project that makes everyone else feel unproductive :)

Face recognition

DINOv3 is extremely overpowered

So much so

#266 9/30/2019 #SantaClara