🧠🔬 Excited to share AnyLoc: Towards Universal Visual Place Recognition Foundation Models meet VPR - VPR anywhere🌍🌊🏙️, anytime🌌☁️🌄, and under anyview🚡🚗🛸 - no retraining/finetuning 🔁 - aimed at general-purpose localization & navigation 🧵👇

Why? - current VPR solutions are task-specific & fail outside training distribution - the “per-image” (CLS) features are suboptimal when used “as-is” for retrieval or VPR 💡Choose the right per-pixel features and aggregate them from “pixels” into “places” 👇

Which Models? We explore task-agnostic features from popular classes of self-supervised models: DINOv2, CLIP, MAE, and more DINOv2 learns long-range global patterns + invariant local features suited to VPR 🗺️

Which Features? Across multiple ViT layers and facets (query, key, value, token) of ViTs: - value has the largest contrast b/w keypoint & background - earlier layers (key & query) show high positional bias ⌖ - deeper layers (value) have the sharpest contrast

Feature aggregation type? We explore a number of aggregation techniques: GeM, GAP, GMP, Soft-VLAD, & Hard-VLAD, In our no-retrain setting, hard-assignment VLAD ranks the best, which (along with GeM) outperforms CLS descriptors used commonly in prior work

There’s more: While common vocab options for VLAD are global, map-specific, or learned, PCA over globally-pooled local features uncover distinct “domains”, enabling ‘domain-specific vocab’ (GeM -> VLAD) to better harness the local feature distribution in aggregation

We evaluate AnyLoc on an unprecedented diversity of VPR scenarios (urban, indoors, aerial, underwater, subterranean, day-night, and seasonal variations, opposing viewpoints), establishing a strong baseline for future research toward universal VPR solutions.

AnyLoc on visually degraded environments 🏚️

AnyLoc on aerial imagery 🛩️🚁

Amazing 🌍 Collab with @123avneesh, @JayKarhade, @_krishna_murthy, @smash0190 @AirLabCMU, Madhava Krishna, & @sourav_garg_ Thanks to @YaoHE09 & Ivan Cisneros for collecting cool drone imagery 🚁📷 to test AnyLoc!

Check out our cool demos: and 5-minute explainer video: