Загрузка видео...

Не удалось загрузить видео

На главную

[1/n] Do distinct large models admit a simple map that aligns their embedding spaces? We show that across multimodal contrastive models—trained on different data and architectures—an orthogonal map aligns image embeddings. Strikingly, the same map also aligns text embeddings.

sharut_gupta's profile picture

Sharut Gupta

2,680 subscribers

36,746 просмотров • 3 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Consistency models, CTMs, shortcut models, align your flow, mean flow... What's the connection, and how should you learn them in practice? We show they're all different sides of the same coin connected by one central object: the flow map. 🧵(1/n)
2:55

Consistency models, CTMs, shortcut models, align your flow, mean flow... What's the connection, and how should you learn them in practice? We show they're all different sides of the same coin connected by one central object: the flow map. 🧵(1/n)

Nicholas Boffi

65,480 просмотров • 8 месяцев назад

Embeddings (and how to create them) are, perhaps, the most interesting idea behind Large Language Models. I built a simple model to help you understand embeddings from scratch. Here is a step-by-step video explanation:
51:43

Embeddings (and how to create them) are, perhaps, the most interesting idea behind Large Language Models. I built a simple model to help you understand embeddings from scratch. Here is a step-by-step video explanation:

Santiago

44,107 просмотров • 1 год назад

🌎⚠️ Doomsday map alert! Peggy Bolton's research shows Bill Gates' farmland aligns with dry land on the map. Is he preparing for a catastrophe? Meanwhile, Jeff Bezos is buying up farmland too. And Hawaii stays safe! How convenient for Mark Zuckerberg. 🤔
1:12

🌎⚠️ Doomsday map alert! Peggy Bolton's research shows Bill Gates' farmland aligns with dry land on the map. Is he preparing for a catastrophe? Meanwhile, Jeff Bezos is buying up farmland too. And Hawaii stays safe! How convenient for Mark Zuckerberg. 🤔

Red Pill USA

1,420,071 просмотров • 2 лет назад

Introducing — Hosted Embedding Marketplace 💈 We’re building a single destination to discover, evaluate, and access relevant embeddings. Move from large expensive models to leaner open-source models without reducing accuracy. Comment 👋 for early access
0:24

Introducing — Hosted Embedding Marketplace 💈 We’re building a single destination to discover, evaluate, and access relevant embeddings. Move from large expensive models to leaner open-source models without reducing accuracy. Comment 👋 for early access

vishal ✦

160,325 просмотров • 3 лет назад

.Tkay had to pull out the "mom-cam" to prove to Methodz & Zoomaa he's just different Tkay on the series (ELIMS): Map 1 - 58/29 Map 2 - 16/2 Map 3 - N/A Map 4 - 44/24 HIM 😤
0:37

.Tkay had to pull out the "mom-cam" to prove to Methodz & Zoomaa he's just different Tkay on the series (ELIMS): Map 1 - 58/29 Map 2 - 16/2 Map 3 - N/A Map 4 - 44/24 HIM 😤

Torn

181,408 просмотров • 1 месяц назад

I downloaded something like 300GB of open models and wrote a bunch of map-reduce style processing scripts to make this graph. It's plotting the distribution of weight values across a variety of popular open models, to show that models are almost entirely made up of small floats.
0:36

I downloaded something like 300GB of open models and wrote a bunch of map-reduce style processing scripts to make this graph. It's plotting the distribution of weight values across a variety of popular open models, to show that models are almost entirely made up of small floats.

Sam Rose

58,141 просмотров • 3 месяцев назад

I recorded this introduction to embeddings (perhaps the most essential concept behind how LLMs.) But this is different from everything else you've seen: We will generate embeddings from scratch using a Siamese Network and a contrastive loss. This video is for 5 year olds.
51:43

I recorded this introduction to embeddings (perhaps the most essential concept behind how LLMs.) But this is different from everything else you've seen: We will generate embeddings from scratch using a Siamese Network and a contrastive loss. This video is for 5 year olds.

Santiago

68,976 просмотров • 1 год назад

How do we build multimodal systems that work effectively across the globe? 🌍 Today we release the Aya Vision Technical Report, the detailed recipe behind Aya Vision models, unifying state-of-the-art multilingual capabilities in multimodal and text tasks across 23 languages!
1:55

How do we build multimodal systems that work effectively across the globe? 🌍 Today we release the Aya Vision Technical Report, the detailed recipe behind Aya Vision models, unifying state-of-the-art multilingual capabilities in multimodal and text tasks across 23 languages!

Cohere Labs

15,561 просмотров • 1 год назад

New short course Multimodal RAG: Chat with Videos, developed with Intel and taught by vasudevlal! In this course, you’ll work with LLaVA (Large Language and Vision Assistant), a Large Vision Language Model (LVLM) that can process both images and text. For example, given an image of a person doing a handstand on a skateboard at the beach, LLaVA doesn't just caption the scene, it’s able to predict possible outcomes, like the person losing balance or falling off. By understanding not just what's in a video frame, but what might happen next, your application can provide more insightful answers to questions about video. You'll build a full multimodal RAG pipeline that can chat about video content: - Use the BridgeTower model to create joint text-image embeddings in a 512-dimensional multimodal semantic space. - Learn video processing techniques to extract keyframes, generate transcripts using Whisper, and create captions. - Use the LanceDB vector database to store and retrieve high-dimensional multimodal embeddings. - Integrate the LLaVA model, combining CLIP's (Contrastive Language Image Pretraining) vision transformer with Llama, for advanced visual-textual reasoning. Your final system will ingest video data, generate embeddings for frames and text, perform similarity searches for relevant content, and use the retrieved multimodal context to inform LVLM-based response generation. The result is a system capable of answering nuanced questions about video content, effectively chatting about the video it has processed. Please sign up here!
2:40

New short course Multimodal RAG: Chat with Videos, developed with Intel and taught by vasudevlal! In this course, you’ll work with LLaVA (Large Language and Vision Assistant), a Large Vision Language Model (LVLM) that can process both images and text. For example, given an image of a person doing a handstand on a skateboard at the beach, LLaVA doesn't just caption the scene, it’s able to predict possible outcomes, like the person losing balance or falling off. By understanding not just what's in a video frame, but what might happen next, your application can provide more insightful answers to questions about video. You'll build a full multimodal RAG pipeline that can chat about video content: - Use the BridgeTower model to create joint text-image embeddings in a 512-dimensional multimodal semantic space. - Learn video processing techniques to extract keyframes, generate transcripts using Whisper, and create captions. - Use the LanceDB vector database to store and retrieve high-dimensional multimodal embeddings. - Integrate the LLaVA model, combining CLIP's (Contrastive Language Image Pretraining) vision transformer with Llama, for advanced visual-textual reasoning. Your final system will ingest video data, generate embeddings for frames and text, perform similarity searches for relevant content, and use the retrieved multimodal context to inform LVLM-based response generation. The result is a system capable of answering nuanced questions about video content, effectively chatting about the video it has processed. Please sign up here!

Andrew Ng

107,548 просмотров • 1 год назад

Sturge: We stand ready. Our personnel are highly trained and our doctrine aligns with yours. We require assets in the interim.
0:47

Sturge: We stand ready. Our personnel are highly trained and our doctrine aligns with yours. We require assets in the interim.

Kejan Haynes

21,204 просмотров • 3 месяцев назад

A civil engineer is working on building a Roller Coaster that travels across the map
0:24

A civil engineer is working on building a Roller Coaster that travels across the map

Rooster

653,612 просмотров • 6 месяцев назад

I Created The FIRST Ever Simple Edit Practice Map! 🤯 Map Code: 8942-4322-3496 or Search "Crosshair Training" Tag someone below that NEEDS this map ⬇️
0:37

I Created The FIRST Ever Simple Edit Practice Map! 🤯 Map Code: 8942-4322-3496 or Search "Crosshair Training" Tag someone below that NEEDS this map ⬇️

Xen Quinn

37,951 просмотров • 11 месяцев назад

Putin says that he intends to change out Ukraine's leadership. Interesting that this ALSO aligns with Trump's goals.
0:31

Putin says that he intends to change out Ukraine's leadership. Interesting that this ALSO aligns with Trump's goals.

Jay in Kyiv

76,164 просмотров • 1 год назад

New Feature: Ping System Showcase Players can now ping on the mini-map to place a pin on the map. Pings placed on the map will also appear on the mini map. #callofdutymobile #codm #codmobile
0:34

New Feature: Ping System Showcase Players can now ping on the mini-map to place a pin on the map. Pings placed on the map will also appear on the mini map. #callofdutymobile #codm #codmobile

Leakers On Duty

42,248 просмотров • 1 год назад

Launching the 1st Arena for Embedding Models: MTEB Arena🏟️ Vote @ ⚔️ 15 Models: OpenAI Google Cohere Voyage_AI_ Jina AI Salesforce AI Research Nomic E5 GritLM BGE.. 3 Tasks: Retrieval/Clustering/STS Deep dive with me on embeddings & the arena👇 🧵1/13
0:35

Launching the 1st Arena for Embedding Models: MTEB Arena🏟️ Vote @ ⚔️ 15 Models: OpenAI Google Cohere Voyage_AI_ Jina AI Salesforce AI Research Nomic E5 GritLM BGE.. 3 Tasks: Retrieval/Clustering/STS Deep dive with me on embeddings & the arena👇 🧵1/13

Niklas Muennighoff

58,942 просмотров • 1 год назад

Meta announces Movie Gen A Cast of Media Foundation Models We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user’s image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models
0:30

Meta announces Movie Gen A Cast of Media Foundation Models We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user’s image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models

AK

62,702 просмотров • 1 год назад

interpolating image embeddings across arbitrary numbers of vertices. using polygons as lenses through n-dimensional latent space.
1:05

interpolating image embeddings across arbitrary numbers of vertices. using polygons as lenses through n-dimensional latent space.

harley turan

22,949 просмотров • 9 месяцев назад

Ever had trouble navigating your city because of outdated map data? Meet Hivemapper: a global, real-time, crowdsourced map that you can build and own. STFU on Solana 🦾
2:47

Ever had trouble navigating your city because of outdated map data? Meet Hivemapper: a global, real-time, crowdsourced map that you can build and own. STFU on Solana 🦾

Superteam

138,781 просмотров • 2 лет назад

A model that aligns perfectly what we see in reality 👀 Something you will never find on a globe 🌎💦
1:00

A model that aligns perfectly what we see in reality 👀 Something you will never find on a globe 🌎💦

Rufus_2688

359,231 просмотров • 2 лет назад

Master the Grapple Hook. Own the map. Simple as that. #StrayShotMobile
0:30

Master the Grapple Hook. Own the map. Simple as that. #StrayShotMobile

StrayShot

49,946 просмотров • 5 месяцев назад