New AI research from Meta – CoTracker3 Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. More details ➡️ Demo on Hugging Face ➡️ Building on our previous work on CoTracker, this new model demonstrates impressive tracking results where points can be tracked for a long time even when... they're occluded or leave the field of view. CoTracker3 achieves state-of-the-art, outperforming all recent point tracking approaches on standard benchmarks — often by a substantial margin. We've released the research paper, code and a demo on Hugging Face — along with models available under an A-NC license to support further research in this space.show more

@huggingface

The paper introduces a new point tracking model called CoTracker3 that builds upon recent point tracking models like PIPs, TAPIR, and CoTracker. Point tracking is an important task in video analysis for applications like 3D reconstruction and video editing. The CoTracker3 model, when trained only on synthetic data, already outperforms state-of-the-art trackers on several benchmarks. When further fine-tuned on just 15,000 real-world unlabeled videos using the proposed protocol, it significantly surpasses the performance of BootsTAPIR, which was trained on 15 million real videos. CoTracker3 also shows better handling of occluded points compared to other models. full paper:

@huggingface love how meta keeps open-sourcing these research

@huggingface Your things @Mbounge_

@huggingface That’s awesome!

@huggingface Exciting development! Meta's new CoTracker3 could revolutionize point tracking in real videos.

@huggingface This is wild.

@huggingface This new AI model from Meta sounds impressive! CoTracker3 could really push forward developments in point tracking technology

@huggingface When are you going to release co-tracker under an Open Source/OSI license? It is the 3rd release under Creative Commons.

@huggingface wow!