[1/n] Do distinct large models admit a simple map that aligns their embedding spaces? We show that across multimodal contrastive models—trained on different data and architectures—an orthogonal map aligns image embeddings. Strikingly, the same map also aligns text embeddings.

Interesting work! in some sense, relative representations ( already point at the map being mostly orthogonal, and follow-up work empirically showed exactly this also studied this for N > 2 models (

[2/n] Specifically, across model families, an orthogonal map aligns their mean-centered image embeddings (cosine up to ~0.9). Fitted on images alone, it transfers to text, raising text–text cosine from near zero to ~0.8, all while preserving task-relevant image–text geometry.

[3/n] Intuition: Despite the modality gap and the disjoint supports of the two models, the relative geometry stays strikingly stable, i.e., texts keep the same angular arrangement with respect to images. Equivalently, the models agree on the multimodal kernel.

[4/n] Theoretically, we show that if the multimodal kernels agree on a sufficiently rich yet small finite set of shared anchors, then a single global orthogonal map aligns the image representations across models, and the same map simultaneously aligns the text representations.

[5/n] This map can be learned from just ~30% of the data and also generalizes across datasets and distributions.

[6/n] Further, this map approximately commutes with cross-modal nearest-neighbor structure; direct image-to-image transfer across models and text-mediated transfer retrieve nearly the same semantic neighborhoods.

[7/n] Implications: avoids re-embedding billions of vectors on model upgrades as small anchor sets recover a geometry-preserving map across multimodal contrastive models. This enables mix-and-match of model encoders and text-free alignment, with important privacy implications.

[8/n] Project page: Link to paper: Link to code: Shoutout to my wonderful collaborators and advisors! Sanyam Kansal, @StefanieJegelka , @phillip_isola and @montsgarg

Interesting work! I'm studying in the same direction. We published a paper on single-modality backward-compatibility at NeurIPS 2025 — we use orthogonal mapping to align new and old models, building on assumptions from Platonic Representation Hypothesis.

Thanks for sharing this! This is quite relevant. Also fully agree with your premise that orthogonal maps preserve downstream-relevant geometry, making them quite powerful.

Nice ☺️, mini-vec2vec might be relevant here. The orthogonal mapping is learned in unsupervised manner between sentence embedding models (but probably could be adapted to other modalities!)

Thanks for sharing this, very relevant : )

Great! Happy that it is!!

this is kind of expected at this point no? - -

PRH and related work focus on kernel alignment; we’re (to our knowledge) the first to show this induces a strikingly simple map: an O(d) map, shared across modalities (cosine ~0.9 showing very strong alignment). This is non-trivial: aligning marginals need not identify the joint.

@BogdanIonutCir2 How close is this to a defacto confirmation of the Platonic Representation Hypothesis?

@BogdanIonutCir2 Thanks for your question! Our work theoretically and empirically confirms PRH for the case of multimodal contrastive models. PRH predicts unimodal kernel agreement; we show that (approx) multimodal kernel alignment implies the two models are related by a simple orthogonal map.

Excellent...... Great job done...

Thank you so much!

Fascinating result. Congrats on that... The striking part isn’t that models align, it’s that they align with a rigid rotation. That almost reads like evidence that representation learning is converging toward a shared geometric manifold rather than inventing new semantics. Some of us are exploring symbolic directional representations for exactly this reason.

I recently read about the Aristotelian perspective here from @mariabrbic , countering PRH. I would like to hear comments. ARH if I may call makes sense to me from a naive perspective. But that being said I have to read a lot.

@mariabrbic ARH focuses on local convergence, whereas PRH argues for global convergence across models. Our work demonstrates global convergence within a specific subset of models, i.e., contrastive multimodal models, and reveals a corresponding globally orthogonal relationship.

Really cool! Looking forward to reading the paper

Thanks a lot!

Exciting work! May i know how did you make this video? Which tool you used?

Thank you! I used Keynote animations and screen recording